A proof in the scientific sense can only be given within mathematics, starting from a system of axioms and then using the rules of logic to prove theorems, lemmas, and however the mathematicians call their results. Mathematics in this sense is a set of rules of our mind. It's not a priori referring to anything in Nature.
With the discovery of the natural sciences in the 17th century (I'd say Kepler, Galileo, and Newton were the first famous figures in the history of modern natural sciences) it has turned out that mathematics is the only adequate language to make precise statements about objective "reality", where I understand "reality" as the objectively observable and quantifiable entities investigated by the natural sciences.
Of course, already much earlier, e.g., in ancient Egypt and Greece, one has used mathematics for practical purposes, most importantly geometry and arithmethics to organize daily life.
That math is such a successful tool in various branches of sciences (reaching from the natural sciences over engineering to economy and sociology) is in my opinion understandable by the fact that many mathematical subjects have been invented using everyday experience and thus are related to "reality" in the above understood sense. E.g., Euclidean geometry has been abstracted from our everyday experience of how bodies around us relate and how distances and directions can be quantified. This holds true even for more abstract subjects like functional analysis which has to a large extent been invented by the mathematicians to make sense of some ideas reaching back to Heaviside in electromagnetic theory ("operator calculus") and Dirac et al in developing modern quantum theory (theory of generalized functions, aka distributions, eigenvalue problems leading to the work by Hilbert, von Neumann, Schwartz et al).
In other cases math was first in inventing very useful theories for application to the natural sciences. E.g., a purely mathematical problem of interest in the 19th century came up when the mathemticians were asking themselves, whether one can prove the axiom of parallels of Euclidean geometry (which itself has undergone a revision by Hilbert et al, making the hidden assumptions on "the continuum" used without being mentioned by the ancient Greeks), leading to the development of non-Euclidean geometry and later on differential geometry and Riemann spaces, later useful for Einstein (after he had been tutored by his old friend Grossmann about this math ;-)) in the devolopment of the General theory of Relativity.
Now the natural sciences are distinct from math already by the fact that the natural sciences aim at describing objective reality or, put more carefully, our observations of objective reality using our senses and technical devices to extent our senses to the utmost small (atomic and subatomic scales) and the utmost large (astrophysical and even cosmological scales) and making everything quantitative and thus precisely analyzable by mathematical means. A physical model thus can never ever be proven in the strict sense a mathematical theorem is proven within a given axiomatic system, but one can test it at higher and higher accuracy by making progress in the development in ever more precise ways to observe and measure phenomena. So, if a physicist is telling you, "one has proven that and that model or theory", it means that so far even the most precise measurements confirmed the predictions of the model. Of course, within a given theory you can have mathematical proofs (like the Heisenberg uncertainty principle from the basic postulates of quantum theory), but this is not a "proof" that the underlying postulates concerning the physics are correct. Instead, the mathematical facts of a physical theory of this kind, may inspire some experimental physicist to new experimental/observational tests of the theory, and the outcome of such new tests always can be that we discover where our contemporary theories fail to explain an observed and measured phenomenon. Then the theoreticians have new work to get even better models, and they have to understand why the old models worked within their range of applicability (e.g., Newtonian mechanics can be understood as an approximately valid limit of relativistic mechanics).
[Sorry ... I just couldn't keep a straight face.]