These books come to mind:
Group theory and Its Applications to Quantum Mechanics, Wigner
Group Theory and Quantum Mechanics, Tinkham
Symmetry Principles and Solid State Physics, Lax
Symmetry Principles and Magnetic Symmetry in Solid State Physics, Joshua
Group Theory in Physics, Tung
Wigner is a classic, hard to read without any prior knowledge. Tungs' book is written in terms of continuous groups which is really helpful for high energy physics and relativity theory. Lax is for solid state and very terse, but readable if you have time.
I would read in this order for Joshua, Tinkham then Wigner. If you understand everything in those unless you are a high energy theorist, you'll know more group theory than you will ever need.