The reality of the magnetic field is on the same footing as the reality of inertia or angular momentum. In fact, I wonder, do you have the same problem with angular momentum? I suggest to think of the fundamental source of magnetism as a loop of current rather than a straight line of current, analogously to thinking of the source of angular momentum as a spinning mass rather than a mass moving in a straight line. Then, consider the effects of this current loop on other currents. Clearly, the direction of the current in the loop is physically unambiguous, and you can unambiguously talk about "clockwise" and "counterclockwise" directions about a given direction through the loop (in so much as you have established a convention for the sign of the charges). The hard part to understand is that, unlike electric phenomena whose fundamental effect is to push and pull radially, the fundamental effect of magnetic phenomena is to cause rotation. The attraction of an unmagnetised paper clip to a permanent magnet, for example, is a secondary phenomenon due to induced magnetism, analogously to the attraction of a neutral piece of paper to a staticly charged rod due to induced polarization.
Regarding the right-hand-rule, ask yourself, why should the cross-product, which isn't even physical but merely mathematical, be defined so that
[tex]
\hat{x}\times\hat{y}=\hat{z}[/tex]
etc.
What if you mathematically define the cross-product so that
[tex]
\hat{x}\times\hat{y}=-\hat{z}[/tex]
etc.? This will indeed change the sign in the Biot-Savart Law, but it will also change the sign in the magnetic force law, so the physical result will be the same. Can you physically distinguish between this mathematical change of sign in the cross-product vs. the quasi-physical change of sign of the B-field itself? The magnetic field isn't actually a vector field; it is a pseudo-vector field, which basically means that it is a cross-product field. This is a fundamentally different kind of field than, say, the electric field. What this amounts to, physically, is that the direction of the B-field should really be represented as
[tex]
\hat{r}\times\hat{x}[/tex]
etc., rather than simply as
[tex]
\hat{x}[/tex]
etc..