For multiple years now (since I was roughly 13) I've been teaching myself pure, rigorous mathematics (and the occasional theoretical physics course). I started off with calculus, and then moved into First-Order Logic (Kalish), then Axiomatic Set Theory (Suppes, Jech, Takeuti-Zaring), then Topology (Dugundji), then Analysis (Big Rudin), then Abstract Algebra (including Linear and Multilinear, from Hungerford, MacLane and Birkhoff), then Riemannian Geometry (DoCarmo, Wald), then General Relativity (Wald), and most recently Category Theory (MacLane, Goldblatt).
I've worked perhaps 2 hours a night every day for years to do this, and I could only have done it because I love pure mathematics more than I love anything else (Halmos said the same thing, actually).
Another subject I've been self taught in is philosophy, mainly from reading the more rigorous existentialist works such as Kierkegaard, Nietzche, Camus, Sartre as well as ethicists like Plato.
Next year I'm going to enter the University and most likely work to receive a double major in Pure Math and Philosophy and then move on to work on a PhD. in Mathematics (most likely with a focus in foundational theories such as Category Theory or Topos Theory).
If you have time and a genuine interest in something, you'll be able to find the time to learn it. I've also found that learning on my own is far more enriching than learning at the hands of others, because autodidactism allows me to structure my learning how it best suits me and allows me to focus on what most interests me.