One thing to remember is that the direction of their magnetic moments depend on their spin direction, and both particles can be in either direction. So there is an orientation where they are repulsive, and an orientation where they are attractive (classically speaking). For QM, this means that there is a higher energy state and a lower energy state because of the proton-electron spin-spin coupling. This gives rise to hyperfine structure in the hydrogen spectrum, see for example:
http://hyperphysics.phy-astr.gsu.edu/hbase/quantum/h21.html
So the magnetic repulsion is not responsible for the existence of stationary states in hydrogen, and if you look at the amount of energy involved with the hyperfine splitting, the magnetic interaction causes an extremely small change to the energy levels.