It makes more sense to me that you should start with something that can do everything, and then you find components of it that explain specific things, including sets, lines, circles, algebra, and calculus. Essentially, start with a general case, then make it specific. Unfortunately, schools never function this way. They always teach you special cases, forcing you to memorize numerous concepts that you haven't yet unified, and preventing you from really seeing the big picture.
Disagree. You can't see the point of the general case unless you have lots of examples at your disposal.
"Too many people write papers that are very abstract and at the end they may give some examples. It should be the other way around. You should start with understanding the interesting examples and build up to explain what the general phenomena are. This was you progress from initial understanding to more understanding. This is both for expository purposes and for mathematical purposes. It should always guide you."
--Michael Atiyah
I think my overarching concern here is that presenting things in complete generality is sort of dogmatic. I always think in terms of how I could have come up with the theory myself, in a perfect world. The inspiration usually comes from starting with the special cases. No one would think to do ring theory all of the sudden without anything coming before it. It's only because you have examples, like matrix rings or quaternions that you start caring about rings. Until you have a bunch of examples that you need to unify, so that you don't have to prove things separately for each of them, there's no reason to unify. Doing so would be dogmatic. Besides, proving things separately for different cases is not always bad. Sometimes, it's easier to see how to do it in the special case, so it's more motivated, and it adds extra repetition, if you are aware of it, which can also be a good thing for learning.
In this context, though, it's true that, pedagogically, it doesn't make sense to start with the foundations. You just work a little more informally, rather than trying to build everything from the ground up. But when you're ready, you can learn something about the nature of math by studying the foundations.
The attempts to go from set theory to all math have failed. Bertrand Russell tried about 100 years ago, and then about 50 years ago they altered set theory so they could (using the zemelo-fraenkel set theory).
It wasn't exactly a failure. It is expressible in terms of Zermelo-Fraenkel axioms, at least in principle. What failed, as per Godel, was Hilbert's program of proving consistency and coming up with an effective procedure to find and prove all theorems.
A bit more recently, they created category theory, which was kind of a compromise to the previous rigidity. It seems like they said: 'there's no way to do this, so let's just get rid of our rules or make them so flexible such that we can force this to work'. That thought process, in my opinion, is the prevailing reason behind string theory and quantum theory (not discounting its uses, of course), but that's a whole separate topic.
Actually, category theory is motivated by algebraic topology (fundamental group, homology, etc.) and wasn't originally intended for foundational purposes. I don't really know much about the attempts to use it for that, though.As to the original question, the way math reduces to set theory is roughly something like this. If you want to build the real numbers, you make them out of rational numbers, but you sort of fill in the holes, like pi or e or the square root of 2 that sort of need to be there (for example, the diagonal of a unit square has length equal to the square root of 2). I don't want to get into the details of that construction (actually, there are two equivalent ways: Dedekind cuts or completion), so let's just take for granted that if you give me the rational numbers, I can build the real numbers out of them.
Rational numbers can be reduced to pairs of integers. How? Just think of 4/5 as the ordered pair (4,5). When you do this, you have to keep in mind that there's some reduncy because 1/2 = 2/4. So the pair (1,2) should be identified with (2,4) and so on. And you have to define addition and all that and make sure that it's well-defined, so there are some more details. So, the upshot is that if you give me the integers, I can build you the rational numbers.
Similarly, if you give me the natural numbers, I can build you the integers. This time, again, you can look at pairs of natural numbers like (9,4) and (12, 18). They represent the difference of the pair of numbers, so (9,4) represents 4-9 = -5, and (12,18) represents 18-12 = 6. Again, you have some redundancy like (10, 5) also representing -5, and the details of defining addition and multiplication. But if you give me the natural numbers, I can build you the integers out of them.
And it's natural numbers that are built directly out of sets. Basically, it's just what you would think it would be. The number n is represented by a set with n elements in it. And what the Zermelo-Fraenkel axioms do is give you enough starting assumptions to be able to carry that out. You have to start with something. So, you take the empty set, which has 0 elements, then you have zero, you take a set with 0 in it for a set with 1 thing in it, then you can get {0,1}, which has 2 things in it, and so on. And the axioms just give you the tools to make this actually work. They may be unsatisfactory in some ways. I'm not a logician, so I don't know. But on some level, it does seem to work.