You might find it very helpful to try this for a concrete example. Just pick a familiar irrational in the unit interval, like x = sqrt(2) - 1 = .4142132...
We want to see if we can approximate x to any desired degree of precision with rationals of the form p2/q2.
So take a random \epsilon ... take \epsilon = 1/100. What do we know? We know that the rationals are dense in the reals, so we can find a rational within \epsilon of x.
What rational is that? Well, one obvious possibility is r = .41 = 41/100. You can see that |x - r| = .0042132 ... < 1/100 = \epsilon. And the denominator's even a square.
41's not a square, but it's surrounded by squares, 36 and 49. Unfortunately, |36/100 - 49/100| is way bigger than our \epsilon; so we have to try to force the distance between consecutive squares to get smaller.
How about if we try r = 41/100 = 4100/10,000. The denominator's still a square, and the squares that bracket 4100 might give us a small enough interval. It actually turns out that they do not. The maximum possible interval between consecutive squares over 104 is still too big.
But we can prove that if we keep multiplying numerator and denominator by 102, the "maximum consecutive squares interval" is eventually small enough so that you can force it to be as close as you like to 41/100; which is within \epsilon = 1/100 of our irrational x.
Why do I keep multiplying numerator and denominator by 100? Because the sequence of denominators is then 102, 104, 106, ... and each of those denominators is a square. So then I just analyze the sequence of square numerators for each denominator 102n; and I see that the maximum possible difference between consecutive squared rationals goes to zero. Of course you have to prove that.
You can make this whole proof go through, but for what it's worth I had to work at it. I had to carefully compute the width of the maximum interval between consecutive squares with denominator 10^2n.
So there's two ideas for you.
One, work the problem with a very specific example.
And two, there's an outline of an idea that I was able to get to work. But it's a long proof, and I had to to a lot of work to convince myself I had it nailed down.
But even if you don't use this particular proof outline (and again, I make no claims that it's elegant or insightful; just that I was able to make it work) I do recommend working this problem with an explicit, concrete example like x = sqrt(2) - 1. I find working with concrete examples helps a lot.