I am not a physicist and do not know all those topics, but I am a mathematician, and have taught manifolds, covering spaces, and discrete groups for years. In my opinion one reason you are having a hard time is the ridiculously large amount of material you are trying to cram in. So you should not feel bad that it did not all take at once.
If I may suggest some reasonable sources for some of those topics, first of all I suggest the first 9 chapters of the book Algebra, by Michael Artin. It is an undergraduate introduction to algebra for MIT students, hence no nonsense and high level, but very well explained. Moreover he takes the attitude, appropriate for you, that infinite groups are more important than finite ones, so begins with the basic such examples, namely matrices.
Later he discusses the various special structures one can put on a space where matrices act, namely metrics defined by bilinear forms, and then, uniquely at this elementary level, he discusses an intoduction to representation theory, including a few important examples of matrix groups, like SU(2) and SO(3), I believe.
Another good book, more oriented to topology and manifolds, is Foundations of differentiable manifolds and Lie groups, by Frank Warner. Here you will find a treatment of tensors, missing from Artin's book, as well as manifolds and Lie algebras.
Another succinct and well written but challenging book is Lectures on Lie Groups, by J. Frank Adams. Here there are Stiefel diagrams, and exponential maps, assuming you know basic existence theorems for diffrential equations, but still no Dynkin diagrams. he gives a nice geometric version of the exponential map, a map from the lie algebra to the lie group, and it is nice to know also that this map, for matrix groups, is actually given by the marvellous exponential series! I.e. if M is any matrix, then e^M is an invertible matrix.
[digression: The only compact abelian Lie groups I believe are the tori, a quotient of euclidean space by a discrete lattice, e.g. an elliptic curve E = C/(Z+iZ), where C is the complex numbers, and Z is the integers. Here no doubt the exponential map is just the universal covering (quotient) map C-->E. No other compact real surface can have a continuous group structure I believe, by computing the Euler characteristic. Just as the circle has a group structure inherited as the length one complex numbers, so presumably does the 3-sphere inherit one from being viewed as length one quaternions. I thought then the Cayley numbers ("octonions") would give one on S^7, but they are not associative, so all this structure gives us is a non zero vector field, apparently a necessary but not sufficient condition for a group structure. So most spheres are not themselves topological groups, and yet they give rise to them by looking at their groups of rotations, the matrix groups SO(n).]
Edit: I found this linked discussion on stackexchange rather enlightening and fun (read the comments too).
https://math.stackexchange.com/ques...h-spheres-can-be-lie-groups?noredirect=1&lq=1
Mike Spivak's Differential Geometry vol. 1, is another comprehensive source for manifolds, tensors, and has a short introdution to Lie groups in chapter 9, which he obswrves uses all the previous material in the book, which is quite a long and detailed treatment of smooth manifold theory.
Many people also like books by William Fulton and Joe Harris. who have a book on representation theory which I have not seen.
None of these yet classify lie algebras, for which I do not know what to recommend, but I recall James Humphreys was well regarded for his books.
the basic idea is to understand an abstract group in terms of its action as a group of motions on some geometric space. To compare two groups you have to understand homomorphisms of groups, and continuous or differentiable maps for continuous groups. Finite groups are thus represented by mapping them into permutation groups S(n). Continuous groups are studied by mapping them into matrix groups, subgroups of GL(n), such as those that preserve some notion of length, like SO(n), or SU(n).
As always in calculus, differentiable maps are studied by approximating them by linear objects, and hence a manifold is studied using its tangent spaces. the basic fact here is that the tangent spaces of a smooth group are all essentially the same since translation in the group defines isomorphisms between any two of them, so it suffices to understand the tangent space at the origin. Now fundamentally, that tangent space has a structure of algebra, and the "Lie algebra" asociated to a lie group is nothing but the tangent space at the origin of that lie group, equipped with its lie bracket multiplication. Viewing tangent vectors as differential operators, this lie bracket structure can be defined using composition of differential operators, as a "commutator". (I mention as a warning that there apparently exist books on lie algebras which do not even mention this basic connection between lie algebras and lie groups.)
Finally, one tries to understand all lie groups by understanding all lie algebras, and learning to what extent a lie group is actually determined by its lie algebra.
This is a lot of stuff, good luck, and hang in there. It is rather beautiful.
