Yes, Lie groups, infinitesimal generators, invariants, and other 'strange' words are the language and method of many branches of modern mathematics and all emerged from considering a special type of functions (transformations) having the geometrical properties of symmetry. Sophus Lie himself started to study a 'point' transformations something around ~1860 and then moved to the more complicated 'contact' transformations.
The point transformations concerns with a transformation of the point, let say (x,y,z)
in the 3D space by a continuous parameter a, producing the transformation
x'=X(x,y,z,a), y'=Y(x,y,z,a), z'=Z(x,y,z,a). There are many such transformations in physics(!), for example translations ( x'=x+a, y'=y, z'=z ), rotations (x'=x*cos(a)+y*sin(a), y'=-x*sin(a)+y*cos(a), z'=z), scaling ( x'=x*a, y'=y, z'=z ),
etc.,etc.,etc.
All of them have some special properties which was discovered 40 yr. before Lie by Abel and Galous. They are called group properties. The difference between the groups discovered by Abel and Galous and the groups of Lie is that the last groups are continious. But the group's properties are almost the same.
Lie found that many functions x'=X(x,a) (here x means a point in n-D space) are differentiable and have the following group properties:
(1) Two sequential transformations, x'=X(x,a) and x' '=X(x',b), can be always substituted by single transformation x' '=X(x,c), where c is the function of the a's and b's alone, i.e. c = f(a,b). It does not matter, which transformation from sequence was done first, f(a,b) = f(b,a). The function f(a,b) is called the law of composition for parameters a and b.
(2) There is a special parameter a=e, which does nothing, x=X(x,e), and a = f(a, e).
This parameter is called a 'unity' or 'identity' element.
(3) For each parameter a there is a special parameter a^(-1) (it is not the power!), which returns a point to the previous state, x=X(x', a^(-1)), and e =f (a,a^(-1)).
This parameter is called 'inverse' element.
There are many books on Lie groups and transformations. It is hard to recommend you something specific, it's depend on your level of knowledge, but based on you name (QuantumTheory) I would recommend the book by Hamermesh, Group Theory and Its Application to Physical Problems, Dover.
Good luck.