I would like to comment on your claims one by one:
"You're in the rest frame of the proton. As long as you don't consider external forces, this frame will be inertial. "
I do not think the rest frame of the proton can be considered strictly inertial, because the proton is not an isolated particle. From the law of conservation of momentum, it is necessary that the electron acts on the proton.
"You get EM radiation when you consider time-dependent perturbations to this system. You get results such as the Fermi's golden rule in TDPT, I don't see what's wrong with this."
I understand you probably mean that the effect of _external_ electromagnetic radiation on the system can be described by a time-dependent addition to the Hamiltonian. I agree. I see nothing wrong with this.
But the original question was concerned with the behaviour and the radiation of the hydrogen atom itself, not under with the action of external radiation on the atom. In the Hamiltonian model of an isolated hydrogen atom, there is nothing that would correspond to electromagnetic radiation.
"One can consider relativistic effects, all this does is lead to the fine structure and hyperfine structure of hydrogen."
This is only partially true. Standard textbook calculations consider relativistic effects only in a very limited way. They use a slightly modified Hamiltonian with terms containing the speed of light c (Darwin, Breit Hamiltonian). They can describe basic magnetic effects, but as far as I know, none of these models claims to be exact and if I understand them well, none of them takes into account the retardation.
See also
http://en.wikipedia.org/wiki/Darwin_Lagrangian
the author claims that if higher powers of v/c (third and higher) were to be included, the retardation has to be dealt with too.
The retardation is necessary if the radiation due to the electron and the proton is to be described. Hence these standard models are not sufficiently accurate to address the question of the radiation of the constituting particles or stability of the ground state.
In short, the problem is that:
Description of radiation due to the proton and the electron requires that we take into account retardation. But the standard Hamiltonian description (even its relativistic enhancements) is not able to do that by default. It is about conservative systems right from the start.
"If you are talking about quantizing the E&M field itself, then I'm not entirely sure on the specifics of that. I hear that QFT is not trivial to apply to bound states."
I hear exactly the same thing. Perhaps the question of stability of hydrogen atom is still open in the relativistic quantum theory?