Physical means that if you do a isospin rotation Uiso on a physical system you get a different system: Uiso|proton> = |neutron>; it's the same as if you act with a rotation R on the solar system, i.e. if you rotate the whole solar system in space; you get a different, physically allowed system R|solar system> = |rotated solar system>.
Quantum mechanically the operators U and R are unitary operators, i.e.
U[θ] = exp(iθaIa)
R[α] = exp(iαaLa)
where I and L are the isospin and orbital angular momentum operatorsa and θ, α are the rotation angles.
Unphysical means that if you construct something like U for gauge transformations you can still do a gauge fixing; that means that the Hilbert space becomes something like
H = Hphys + Hunphys
and that for all states in Hphys the unitary operator implementing a gauge transformation g is reduced to the identity:
U[g] |phys> = id |phys>
U[g] |ψ> = |ψ>
This is different from the above described isospin case b/c for isospin (with isospin being an exact symmetry) all measurable quantities are identical, but the states are not! Mathematically there is still an iso-doublet, an iso-triplet etc. But for gauge transformations there is only a gauge-singulet. All states not belonging to the physical (= singulet sector) are unphysical, they can't be created, they do not result from time evolution, they cannot be measured or observed, they do not interact with physical states etc. So mathematically gauge fixing is somethjing like projecting to a huge physical Hilbert space; nothing like that happens for rotations, isospin etc.
Look at rotations: all theories we know are invariant w.r.t (global) rotations; but the states we observe are not! We observe different orientations in space, we can distinguish between different p-orbitals in the hydrogen atom, etc. We have observables in our theories to make this distinction. Nothing like that is possible with gauge transformations; there is no observable which allows us to distinguish between different gauge sectors!
Let's look at the generator of gauge transformations Ga. These generators commute with all observables; better: an operator A is an observable only if it commutes with these Ga.
[A, Ga] = 0
Now look at rotations and isospin; of course you can have an observable Lx which does not commute with Ly.
Regarding abstract space; I think what really matters is that this space is not ordinary 3-space!
Again let's look at ordinary rotations. In field theory you can rotate a position-space vector r using a rotation matrix R; now you can relate this to rotations of fields like scalar fields, vector fields, spinor fields etc. In all cases you get a second "rotation matrix" S now acting on the fields A(r); that means
r → r' = R r
A → S(R) A
I think you are familiar with this concept e.g. when constructin S(R) for the Dirac equation or when translating Lorentz transformations acting on spacetime to transformation laws acting on 4-vectors and 4-tensors.
For an abstract space (like isospin, color,. ... ) there is no such R acting on r. You can't relate isospin rotations or gauge transformations to transformations in position space; so there is no R, only a S.
I hope it's now clear that "physical vs. unphysical" and "abstract" have nothing to do whith each other!