GR can be understood at many different levels. If all you knew was algebra, you could understand Relativity Simply Explained, by Gardner. If all you know is a semester's worth of calculus, you can understand Exploring Black Holes, by Taylor and Wheeler. With a little more background, you can understand upper-division undergrad books like Hartle.
Sophistication in physics is much more important, and much harder to acquire, then sophistication in mathematics if you want to understand GR.