I take summer courses every year. I highly recommend it, I've always found it to be a much better experience than regular term courses. The classes are smaller and the students tend to be a bit more motivated (since summer courses are optional), they're offered with more flexibility in scheduling to accommodate part-time enrollment, and the professors have a little bit less in the way of other teaching obligations.
You could also do both, of course. In my experience, the wisdom is that every credit hour taken during the summer is like 2 credit hours during the semester in terms of time obligation. So if your calculus course is 4 or 5 credits, that leaves you with a very part-time schedule that should easily allow you to find time for work.