Well, the text says that "A is the set of ALL positive rationals p such that p^2<2". So a) p in A implies p^2-2<0 by definition and b) you can't say A={5} or something similar.
When it comes to the form of (3), I suppose it's just a clever choice, such that q gets the properties we want. However, we know that p^2<2 and p^2>2 give distinct cases, so a factor of p^2-2 somewhere in the expression is to be expected. Similarly, the term linear in p is suggested by the...