Assuming you're in the US (details vary in other countries):
For your bachelor's, try to go to a school that does astrophysics research, so you can try it out and make sure that's what you really want to do. Or you can go somewhere else during the summer to get astrophysics research experience. (Look up "REU programs".) Get a good solid physics bachelor's degree. Include some astrophysics electives. Some schools have astrophysics bachelor's degrees, but I don't think they're essential, and an "ordinary" physics bachelor's will give you more flexibility if you change your mind and decide to go into something else.
For graduate school, go somewhere that does astrophysics research, of course. Go straight into the Ph.D. program, as that's the normal practice in the US. You can pick up a master's along the way, when you have a certain number of hours of coursework. In the US, most standalone ("terminal") master's programs are not intended to lead on to a Ph.D. Depending on the school, your degree may be in astrophysics or in physics. What matters is the research that you do and who you do it with.