In Saying that "i2= -1 is just a basic arithmetic fact", Hurkyl means, basically, that this is how "i" is defined.
But here is a little deeper way of looking at it. If we define the "complex" numbers to be the set of pairs of real numbers, (a, b) with addition and multiplication defined by (a, b)+ (c, d)= (a+ b, c+ d), (a, b)(c, d)= (ac- bd, ad+ bc) then it can be shown that
1) This system forms a "field"
2) The subset of all pairs of the form (a, 0) is a subfield and is isomorphic to the field of real numbers through the isomorphism (a, 0)---> a so we can "label" the pair (a, 0) simply by "a".
3) If we label the pair (0, 1) by "i" then (a, b)= (a, 0)+ (b, 0)(0,1)= a+ bi.
4) so "i2" means (0, 1)(0, 1)= (0(0)- 1(1), 0(1)+ 1(0))= (-1, 0)= -1.
The avoids the problem that saying "i2= -1" doesn't really define i since there are two complex numbers with that property.
And, as Hurkyl said, NO number has any intrinsic "physical meaning". It depends upon how you use them in a specific application.