@zhenyong: I am out of the loop for a long time, but if you love math, I think you have a possible chance. That said, you don't really know if you love math until you learn some non trivial math. The foundational knowledge needed for graduate work in pure math, is linear algebra, with theory, and advanced calculus, several variables, including the implicit function theorem, which in my experience is actually quite rare.
But some books that are good background for graduate studies might include say Spivak's Calculus, and his Calculus on Manifolds, and maybe an algebra book like Mike Artin's Algebra. Oh yes, a topology book would help too, like something by Munkres. Have you read the early posts in this huge thread? going back to 2006 or so? This question is surely discussed at length there. Maybe in the very first post #1... Actually I recommend you at least skim the first 50-100 posts in this thread, roughly through page 2 or 4, or maybe a little more.
As to getting admitted to grad school, you mainly have to convince them you are qualified. I myself just walked into the math dept of University of Washington, Seattle, asked to take the PhD prelims, (after preparing of course for some months), passed them, and was offered admission, and a fellowship.
My point is, for a pure math PhD program, I think it matters more what you know, than what grades you have in undergraduate school. E.g. my undergrad GPA was mediocre, but I later learned that material by studying it and teaching it. However, when applying for lower level public school teaching jobs, I was asked for my undergrad GPA, and did not get those jobs. So when looking for positions, I recommend not to aim either too high or too low, if you want to be appreciated for what you have to offer. Maybe the distinction here is that admission to an educational program may hinge mainly on whether you can handle the work, while getting a job often requires holding a specific degree. I.e. even if you lack paper qualifications, you can sometimes convince people in person that you are qualified, assuming that you are, but that may not suffice to get certain jobs.
one possibly relevant comment about being admitted and even supported by a pure math Phd program: these programs often have a certain amount of money to give out to support students, and because pure math is not a particularly lucrative career, they often do not have enough qualified applicants to use up the available money. since they must give out this money every year or else lose it, they will sometimes admit simply the most qualified applicants they have, even if not extremely well qualified. Unfortunately this is sometimes bad for the admitted students at the bottom of the pile, because although they may be admitted, they may not succeed.
But I am just saying that admission to a pure math Phd program is somettimes easier than you might think because it is less competitive since the salary afterwards is lower. E.g. the school, Univ of Washington, where I was offered admission to the pure math PhD program based only on a test performance, nowadays does not even entertain applications to their undergraduate comp sci program from anyone out of state, no matter how well qualified. This is because those graduates can expect good salaries afterwards. It is possible they are still more accepting of pure math applicants for undergrad and grad programs, but I don't actually know that.
Good luck finding the right program for you at this point in your journey. A personal interview with someone sympathetic and knowledgeable, who can assess your background and potential, like a math professor, may be useful in identifying what that should be.
