Physics has an axiomatic structure, it's just not always laid out plainly to see. In Quantum Mechanics it tends to be set out plainly as a list of 'postulates', which is just another word for axiom. In other physical disciplines the axioms are not so commonly all lined up in one neat location, but they are still there. In additional to the physical axioms, the axioms of set theory are needed, to enable the use of mathematics. Since those axioms are rich enough to allow description of the natural numbers and the operations of addition, subtraction and order comparison, Godel Incompleteness Theorem applies, and it tells us that there are propositions in any physical theory that are undecidable - ie are meaningful but can neither be proven true nor false. What it doesn't tell us is whether those are interesting and significant propositions, or just very artificial ones with no particular physical relevance, like the proposition constructed in Godel's theorem.
My guess is that, if not Godel's theorem itself, something that uses the same sort of self-referential arguments implies that physics can never explain everything interesting about the universe. In very, very hand-wavy terms, it seems to me that something can only be completely explained from the outside. So if we want to explain the universe (or any other system) in its entirety, we need to do so in terms of principles that in some way transcend the system. If there were pan-dimensional beings (perhaps shades of the colour blue like in H2G2) looking at our universe from outside, maybe they could explain it, but then - as with Godel's theorem - they could not explain the higher-dimensional world that they inhabited, and in which our spacetime was embedded.
No matter how many levels up we go, we'd never find a being that could explain everything.
But as I said, this is speculation. It might be that the only propositions that are undecidable are physically uninteresting statements that are loosely analogous to the Godel statement 'this proposition is not provable in this logic'.