I commute 45 minutes each way to school; this leads to quit a gas drain if I drive there 5 days a week. In the past I have found people to carpool with, but this is often unreliable. I have gotten to the point where I go to class about as often as I don't. Luckily, I am very familiar with my professors and they don't require attendance whatsoever. I go to each class at least once a week just to be sure I don't miss any huge announcements.
Personally, this has not affected me in any ill manner. I consistently score at the highest end of all tests and my assignments are always turned in ahead of schedule. However, as others have said, this requires a lot of self-discipline. I am taking all senior-level courses along with one graduate course; as long as you are disciplined, it is possible to have mediocre attendance while maintaining a 4.0 GPA as long as your professor doesn't care.
You have to decide for yourself if you are getting what you need from a course by relying on self study for a majority of your learning. For me, it works great. If I wasn't so fanatical about my area of study, I would probably need to go to class more often. In summary: as long as you feel like you are mastering material to the same level as you would going to class, then keep on. Good luck with whatever you end up doing!