Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Experimental Results vs Predictions

  1. Nov 3, 2011 #1
    So this has been on my mind:
    When an experiment is performed and results gathered/calculated etc and checked against theoretical predictions:

    * when you take into account random errors and
    * when you take into account systematic errors,

    and the result is within the one predicted, are you confirming the prediction? Or is there some probability to the result and really you're saying what I've measured probably is as predicted?
  2. jcsd
  3. Nov 3, 2011 #2
    yes, predictions should be a value +/- error, and as long as your measurement falls within that range, your prediction is confirmed.
    If your results are outside that range, you should consider if you've overlooked some sources of error. If you don't find any more sources, you should start over and redesign the experiment.
  4. Nov 3, 2011 #3


    User Avatar

    Staff: Mentor

    No, that's not correct. Not only does it not absolutely confirm the theory, being inside your error margin doesn't even absolutely confirm the experimental result. The error bars only represent a certain confidence - 95% iirc.
  5. Nov 3, 2011 #4


    User Avatar

    What you are describing here is 'proof by association/co-incidence'. Unfortunately, it seems that many occasions of what are deeemed (should be!) 'science' get caught up in this kind of thinking and promotes 'experimenter bias' (where the experimenter sees what they were expecting, so further entrenches their ideas).

    The 'strong' scientific way is to use your theory to generate null hypotheses, that is to say; your theory should be able to predict things that won't happen, if it were correct. You then look for where, or try to make, those things happen.

    The data you get back can never prove any theory in itself, but it may serve to either reject a null hypothesis (in which case your theory remains, not disproven), or you fail to reject the null hypothesis, in which case your theory has failed.

    The particular strand of science I worry over in particular is in medical science where the results can be more subjective than one might care for and where clinical protocols themselves tend to work by drawing conclusions by co-incidence, viz. people with more X in their diet have a higher frequency of Y illness. It is almost impossible to construct a meaningful null hypothesis in such a scenario if you have no comprehension of the mechanistic reasons that cause illness Y.
  6. Nov 3, 2011 #5

    Well said and any "scientist" who come forward and annouces they have proven this or that theroy simply was not doing science they were doing statitics and correlation. All a scientist can generally do is step forward and say I preformed these tests with these controls and failed to disprove the theroy of blank thus as of now it is still valid.

    Proof is rare and needs to be concrete hence even evolution is still a theory like relativity.

    All good science is skeptical, beliefs do not belong in the lab.
  7. Nov 3, 2011 #6
    We often show so-called "1-sigma" error bars which are only 68% confidence.

    95% <-> "2-sigma"
    99.7% <-> "3-sigma" -> qualifies for serious publication ("anomaly")
    "5-sigma <-> 0.5 chances in a million for a statistical fluke <- common discovery level in physics

    In general, I would say there is no big deal happening in this thread, there is no "strong vs weak" scientific method. It is very well understood (in the scientific communities) what the difference between "measurement vs prediction" is. When we make a prediction, we use probability. When we infer the parameters of a theory from measurements, or test the agreement of a theory with the measurements, we use statistics. This distinction is understood well enough so that university lectures have different titles.

    When we use statistics to infer the most probable values for the parameters of a theory, or to test the agreement between the predictions and the measurements, we never even think of "proving the theory". There is not any meaning to "the probability that the theory is right given the measurement" because there is not even a well defined space for all possible theories, let alone a metric in such a putative space. Whichever agreement level between predictions and measurements, the theory can always be modified ever so slightly that the agreement is essentially unchanged.

    I would like to share one of the most hilarious moments I have ever witnessed in the interview of a professional physicist for popular account, because it is relevant to this thread. It happens at 45 s in the video below
    Last edited by a moderator: Sep 25, 2014
  8. Nov 3, 2011 #7
    This is your personal opinion, isn't it ?
  9. Nov 3, 2011 #8


    User Avatar
    Gold Member

    See, the difference between theory and practice is that in theory practice and theory are the same but in practice they are not.
  10. Nov 3, 2011 #9
    No...Evolution and Relativity are both theroies unless you are asking if proof is rare and concrete is my opinion?

    To which I would say can you Prove that Proof is common and does not need to be concrete? :tongue:

    One of the common issues with this topic is that global getting hotter thing that people seem to think is "proven" every other week. but we will not talk about that here
  11. Nov 3, 2011 #10
    I guess my question is: how can a theory have failed if your measurement result is just a probablistic statement?
  12. Nov 3, 2011 #11
    If the value predicted by the theory falls within the error bars of the experiment, then the theory is confirmed.
  13. Nov 3, 2011 #12
    Yes you are right. And so is gravity, a theory. And so is the "theory" that you will die if you jump off a 10km altitude plane. And by that I mean, some people are known to have survived. Still a theory good enough for me not to test it.

    There is not such issue. It is getting hotter. The issue is why.
  14. Nov 3, 2011 #13
    I concurr but again that is a topic we are not allowed on here.

    Gravity on the other hand is a Law. If you jump out of that plane you may not die but you will fall period no questions asked.

    The theroy that everyone dies who jumps out af an airplane is easy to disprove with a parachute.

    That theroy would then need to be revised
    the new Hypothesis could be:

    Everyone who jumps out of an airplane from 10km and shoots them self in the head while falling after slitting their wrists with out a parachute before hitting the ground dies?

    That Theroy could one day be promoted to Law of the natural world but really who wants to disprove it?
  15. Nov 3, 2011 #14


    User Avatar
    Science Advisor
    Gold Member

    I thinking this might go into my signature. :rofl:
    Last edited by a moderator: Sep 25, 2014
  16. Nov 3, 2011 #15
    I was not very specific so I will try to improve.

    F = G m_{1} m_{2} / r_{12}^2 is a law. The gravitational part of the lagrangian density being equal to the Ricci scalar is a more precise law. More explicitly, Newton wrote down a full blown theory of gravity. Einstein did too. We know Newton's theory limitations because Einstein's is more precise. Einstein's theory predict its own doom and we have not been able to demonstrate any other of its limitations experimentally.

    There are a few, here and there, historical exceptions to this "law" or "theory".

    That is a loophole to my initial formulation and aside the point. My point was mostly that a "law" is concept weaker than what scientists call a "theory". A law is a mere mathematical relation which is suggested by experimental data but does not have a full blown conceptual basis. A theory is a set of conceptual hypothesis, translated into mathematical laws, numerically developed to experimental predictions which have been tested and validated to some degree (degree of accuracy, of kinematical reach, etc).
  17. Nov 3, 2011 #16


    User Avatar

    No. That is absolutely and totally wrong, and is precisely why it is necessary to do science by null hypotheses. This is very much the subject of the post and the trap that many self-delusionists in science have fallen into.

    OK, a trivial example. All cows are black and white. I go look at a field and count a hundred cows. They are all black and white. My theory is confirmed.

    NO. NO!

    What you do is say 'My theory is "All cows are black and white.". To disprove this I need to find one cow that is not black and white. I look at one field of cows and do not find a non-black and white cow. My theory remains, not disproven.' [You can say no more than this.]

    And it will remain until an observation of a not-B&W cow is made. Once a not-B&W cow is observed, then it is no longer a theory, it is nothing. A theory is something that you can disprove. You can only disprove it, you can never 'prove' it. However, there are levels of confidence so high that we regard them as 'proof' and accept them axiomatically.

    So there is still some pedantery there to overcome, on the borders of scientific philosophy.
  18. Nov 3, 2011 #17


    User Avatar

    Staff: Mentor

    Use of the word "still" is improper here. "Theory" is the highest level of proof with a title in science.
  19. Nov 3, 2011 #18
    Yes. I said confirmed, not proven. I repeat. If the value predicted by the theory falls within the error bars of the experiment, then the theory is confirmed.
  20. Nov 3, 2011 #19
    Again, how can a theory be refuted by experimental results?
    For example:
    Experimental result: 1.5kg +/- 0.2
    Theory predicts: 1.1kg

    But the experimental result has some probability to being right. The true value might be 1.1, but that doesn't show up in the experimental result
  21. Nov 3, 2011 #20
    In this case, the experiment falsifies the theory.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook