What's the magnetic field of an infinite line charge moving at a speed v? A quick calculation by Ampère's law tell us that it has a 1/s dependence. So what does a solenoid look like out at infinite? Definitely something similar to a moving line charge at infinite.. we could easily guess that at the very least for the solenoid, that it also has a 1/s dependence.
So lets assume then that the magnetic field has a 1/s dependence for the solenoid, well at infinite if it's similar to a moving line charge, it should be zero due to the 1/s dependence. But Ampere's law for a solenoid tells us that the magnetic field at a point a away from the solenoid has to be the same as the magnetic field at a point b away from the solenoid, so the logical progression is 0 everywhere.
"The symmetry is that of an infinite wire and the magnetic field is the same as that of an infinite wire... and different from zero." (i.e. is non zero with a 1/s dependence) is correct, but incomplete.
This is the exact argument used in the textbook.
EDIT: One last thing as a conceptual question. What do you see if you go out a distance infinite from an infinitely long cylinder? The answer is not an infinite line as you suggested.