Some concepts in abstract algebra were used to derive concepts in quantum mechanics, but that is really the only physics application of it, I could be wrong.
Yeah, that's pretty much wrong.
Abstract algebra can be useful in physics. I'm not an expert on it. But the thing is, group theory, one branch of abstract algebra is all about symmetry. Symmetry is an important concept in physics. For example, you might have a U(1)-symmetry which keeps track of the phase of an electron. And the standard model uses a U(1) cross SU(2) cross SU(3), (maybe modulo Z_6 or something, I forget, since I don't actually understand this stuff).
I'm sure there are lots of other applications that I'm not aware of. For example, things like Von Neumann algebras and C* algebras. Those are examples of "rings", so ring theory can be relevant there. I don't know what the applications are there, but I know some physicists are interested in them. For starters, QM deals with Hilbert spaces. The bounded operators on a Hilbert space are sort of the canonical example of a C* algebra. In QM, a lot of the operators are unbounded, but there are ways to try to approximate them with bounded operators or some such thing. Again, this isn't my specialty.
Another area is quantum groups--these are not actually groups. Again, they are algebras.
Quantum groups have their origins in physics and may be relevant to the physics of anyonic condensed matter systems, exactly solvable models in statistical mechanics, or maybe quantum gravity.
These are just some examples. Depends on what you want to do exactly.