Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Occam’s Razor has lost its edge

  1. Nov 11, 2003 #1

    Ivan Seeking

    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    First, we find in the skeptics dictionary that the concept is not well defined.

    http://skepdic.com/occam.html

    Next, the concept predates modern physics and is no more valid than 12th century physics.
    Consider Occam's original premise:
    Clearly the credibility of this premise died with QM.

    ------ RP Feynman

    Not only is this concept ill defined and out of date, it is often used as a scientific principle – like F = ma – when it fact it is a philosophical concept; not a scientific law. The use of this concept to rule out a competing theory is only valid to the extent that it is obvious; and without need of the long dead Occam or his nasty old rusty razor. The application of this principle in any practical matter is pseudoscience.
     
    Last edited: Nov 11, 2003
  2. jcsd
  3. Nov 11, 2003 #2
    There are many versions of it; some of them are well-defined (and correspondingly limited in scope), e.g.:

    http://quasar.as.utexas.edu/papers/ockham.pdf

    Occam's razor, in this view, is a quantitative codification of how people consistently associate beliefs with evidence.

    What?

    Used by who? Scientists know that you can't disprove a theory on the basis of Occam's razor. You can, however, ignore a theory on that basis, if you choose.

    Well, what are you arguing about? Occam was stating the obvious, after all.

    Nonsense. It is not "pseudoscience" to choose which of two theories you want to pursue, on the basis of "simplicity" or any other basis, so long as both theories are scientific and not falsified.

    For that matter, there is a quantitatively rigorous way in which we may compare the relative probabilities of competing hypotheses (under the assumption that the hypotheses in question are the only explanations, of course -- it's impossible to account for hypotheses that nobody has thought of yet); see the above paper.
     
  4. Nov 11, 2003 #3

    selfAdjoint

    User Avatar
    Staff Emeritus
    Gold Member
    Dearly Missed

    I've always thought of OR as a proposed search strategy; don't take up the complex explanations until you have ruled out the simple ones. It surely is not constrained by whatever you happen to be talking about. The notion that it is invalidated by QM strikes me as a nonsequitur, if not a category error.
     
  5. Nov 11, 2003 #4
    OR is intended to cover real choices between totally identical situations, with no other reasons for preference involved. Sure, choose the simpler alternative. But most theories are not identical, either in method or in the choice of territory to be covered. Most new theories propose to move into distinct new territories. Two other criteria are more important: fecundity (richness of results) and plausibility (soundness of deductions).
     
  6. Nov 11, 2003 #5

    Ivan Seeking

    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    In this form it makes perfect sense. This is not the typical language.

    I don't see any simple explanations in modern physics. All of our simple expectations failed. This is why we have to discover the laws of physics; why we can't deduce all by logic as the Greeks tried. I see no evidence that we can anticipate the correct answers with our sense of logic or simplicity. To the point that they even exist, all explanations are complex. In fact, in the end - at the deepest levels - we have no answers.

    Where is the evidence that simple answers prevail?
     
  7. Nov 11, 2003 #6

    Ivan Seeking

    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Re: Re: Occam’s Razor has lost its edge

     
  8. Nov 11, 2003 #7
    Re: Re: Re: Occam’s Razor has lost its edge

    You don't have to assume "all things equal" to apply Occam's razor.

    Regardless of what Occam's intent was, his principle is still applicable.

    Tell me: I have a set of 100 data points that look like a line, except there is a little bit of scatter. I want to fit a curve to it. Do I fit a line, or do I fit a 100th-degree polynomial to it with 100 coefficients? The latter can pass exactly through every data point with no error. Which theory should be preferred: the linear theory or the 100th-degree polynomial theory?

    Sorry, Ivan, but you do not speak for scientists. Yes, scientists do dismiss overly complicated theories when simpler alternatives exist, on the basis of Occam's Razor. But that is not because the simpler theory "must be the correct answer".

    The point is that we should not assume more than we have evidence for. If we have many competing theories, all of which explain the data, then the data cannot decide between them. But unless you have a specific reason to prefer one particular theory over any of the others, one should put the greatest degree of belief in the theory that has the fewest free parameters.

    I can make up an arbitrarily complex theory to account for any data, but that doesn't mean it's a good theory. Like I said, you can come up with a theory that will explain any finite set of data exactly, by making it complicated enough, but it has no predictive power. It's like epicycles, compared to Kepler's laws.
     
  9. Nov 11, 2003 #8

    Ivan Seeking

    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Re: Re: Re: Re: Occam’s Razor has lost its edge

    So we should apply OR when faced with evidence for a more complex hypothesis?


    [quoteRegardless of what Occam's intent was, his principle is still applicable.[/quote]

    Only to the extent that it is not needed.

    Only to the extent that it is not needed.


    I speak from my own experience...so you speak for all scientists?

    Overly complicated; when exactly does the "overly" kick in?

    I'm glad we agree.

    Boloney. They should each be considered on their own merit.

    Like classical physics compared to QM?
     
    Last edited: Nov 11, 2003
  10. Nov 11, 2003 #9
    Re: Re: Re: Re: Re: Occam’s Razor has lost its edge

    We should apply it at all times. For a more complex hypotheses to be more plausible than a simpler one, it has to fit the data considerably better than the simpler hypothesis. If there is enough evidence in favor of the more complex hypothesis, then it will win out.

    On the contrary, everyone uses it every day, including yourself. If you can't find your keys, do you immediately jump to the conclusion that invisible space aliens teleported into your room and stole them, or do you put more belief in the hypothesis that you misplaced them? Both hypothesis are consistent with all of the evidence, after all, but one of them is simpler than the other.

    Better than you do, if you are going to maintain your ridiculous strawman position that scientists think that Occam's Razor is capable of proving or disproving theories.

    That depends on one's judgement, of course, which is why there are arguments among scientists about what theories to prefer. In more restricted cases, it is possible to make "overly" a quantitatively precise concept; that was the point of the article I linked.

    They are considered on their own merit, and the simpler theory has more merit in the absence of evidence favoring the more complex theory. That's the whole point of science, after all. Otherwise we'd be fitting all our data with Nth-degree polynomials instead of coming up with predictive theories.

    That's not a complete sentence. Is what like classical physics compared to QM?

    If you had proposed quantum mechanical laws in the 17th century, you would have been laughed at, and rightly so: there was no evidence that supported such a hypothesis over Newton's laws. You could equally well have proposed many other more complicated alternatives to Newton's laws, which would have been completely unsupported by the data as well. There are always infinitely many laws you can propose to replace existing laws: one of them may be right, but infinitely many others will be wrong. Without evidence, there is no way to pick one over the other, so the simplest theory is the most credible, on the basis of the evidence that exists at the time.
     
  11. Nov 11, 2003 #10

    selfAdjoint

    User Avatar
    Staff Emeritus
    Gold Member
    Dearly Missed

    As a search strategy OR is useful, but in general a lot has been learned in the last few decades about searching, depth first versus breadth first, the use of frustrated systems modeled on spin glasses, etc. What I would like to see is the search strategy google uses!
     
  12. Nov 12, 2003 #11

    Ivan Seeking

    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Re: Re: Re: Re: Re: Re: Occam’s Razor has lost its edge

    How much is considerably? More plausible? Why needlessly rule out potentially viable options? This only increases the chance of following the wrong path ad infintum.


    This is silliness! You assert that it is just as likely that I lost my keys as it is that aliens took them; but the alien theory is just more complicated? This is not an example of OR; it's statistics.


    Ah, you're psychic also? I maintain that beyond the obvious, it has no place in science. I think the history of scientific advances supports this notion.


    It is still arbitrary.


    "in the absence of evidence favoring the more complex theory"
    Not the usage that I see. This also disagrees with your first statement in this post.




    The point is that consistently we see physics increases in complexity; it does not decrease. This is in direct opposition to the implicit logic of OR. We should expect more complex solutions; not simpler ones.

    Edit: If you chose to use only a limited interpretation OR, one that is really the principle of pasimony in the strictest sense, then really we agree. This is not how this principle is interpreted by many educated people; nor is this interpretation the motivation for this thread. If this were the only interpretation we would be done.

    Really this reads like the 2nd ammendment to the constitution: I agree with the first point but not the second. The simplest and the more complex should be considered on their own merit.

    How would you define the word preferred in this context?
     
    Last edited: Nov 12, 2003
  13. Nov 12, 2003 #12

    Ivan Seeking

    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Perhaps we will finally obtain a TOE using random searches in super computers. Hmmmm, that would be disappointing somehow...:frown: Will the next Einstein be a Quantum Computer?
     
  14. Nov 12, 2003 #13
    Re: Re: Re: Re: Re: Re: Re: Occam’s Razor has lost its edge

    Ivan, you obviously are not a practicing scientist. There are infinitely many potentially viable options, and not that many scientists working on any specific problem. Every once in a while someone will bring up a complicated theory that was previously ignored because it was more complicated --- even today, people discussion aether theories and variable-light speed theories as alternatives to relativity, for instance --- but it is simply not possible to spend time on all "potentially viable options", nor is it reasonable to prefer more complicated theories when simpler ones work just as well.

    But Occam's razor is statistics! Read the paper I cited.

    No.

    As I said, Occam's razor is obvious: even you seem to agree with that. So what are you arguing about?

    The history of scientific advances consists of people considering the simplest theory that fits the facts, until the evidence suggests otherwise.

    It is only as arbitrary as you choose to make it. If you want to assign equal prior probabilities to the hypotheses under consideration, then it is completely unarbitrary and determined by the laws of probability, if you're talking about the precise version in the paper. If you're talking about human judgement, so what if it's arbitrary? You are the one who is (incorrectly) claiming that Occam's razor is used to disprove theories.

    Well, I can hardly address some usage that you claim to see, but I have never seen among myself or my colleagues.

    It does not.

    To the contrary, the history of physics has been towards simpler, more unified theories, that predict more with fewer assumptions. The calculations may be harder, because these theories allow us to tackle more complicated

    Occam's razor does not say that the simplest theory is always the right one!

    All right, then, I think we do agree. Even Occam himself agrees: if you read how he phrased the statement (or at least, the phrasing that is attributed to him), he said "ideas should not be multiplied beyond necessity". He didn't say anything like what you are claiming. Whether some people do claim what you say is another matter, but in my experience scientists do use Occam's razor as a principle of parsimony.

    For instance, Lorentz's aether theory is exactly equivalent to Einstein's special relativity in every measurable way. If you ask a relativist why he doesn't consider LET over SR, he won't say that it's because LET is wrong. He will say that Lorentz's aether is unobservable, and SR can do everything that LET can do without an extra untestable aether lying around, so he takes the simpler theory.

    But what does that mean, "should be considered on their own merit"? Even if you assign equal prior belief in the linear theory and the 100th-degree-polynomial theory, in my previous example, after seeing the data are you really going to believe the two are equally plausible explanations?

    Some people disagree with me, but in the spirit of Bayesian probability theory, I define it to mean that one should assign a greater belief in one hypothesis or another, given the evidence at hand.
     
  15. Nov 13, 2003 #14

    Ivan Seeking

    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    I got busy. I'll be back.
     
  16. Nov 14, 2003 #15

    Ivan Seeking

    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Re: Re: Re: Re: Re: Re: Re: Re: Occam’s Razor has lost its edge

    First, I am only a graduate in physics with a 30 year habit [what I do here at PF is mostly not included in that statement]. I do practice science at times as a function of my work and personal interests. Between my degree and my long experience with physics through books, lectures, and formal discussions, I have invested perhaps forty thousand hours into the subject. Clearly I do not qualify as a mainstream research scientist, however considering my degree and my long history with the discipline, I can hardly be considered an outsider.



    Well, you know where I have spent my time lately. I see this concept used improperly by science and engineering students, and graduates [at least] on a fairly regular basis.

    If all my discussion were with you this thread would likely not exist.


    I mean that rather than use words like “belief”, I prefer to consider things strictly in terms of probabilities. I may comment on your paper later. It is most interesting.

    There is still the deeper objection to Occam’s motivation. The original thesis that motivates OR - the religious belief that simplicity is perfection, and since the universe was made by God, it must be simple – appears to be incorrect. You pointed out that physics reduces complexity with simpler models. My objection is that nature keeps forcing more complex models; physics only responds to this requirement. So at its core, the belief in ultimate simplicity appears to be contraindicated by modern physics. For this reason, I argue that in the face of the evidence, the core philosophy of OR is statistically unlikely.
     
  17. Nov 14, 2003 #16
    Re: Re: Re: Re: Re: Re: Re: Re: Re: Occam’s Razor has lost its edge

    To someone who subscribes to the Bayesian interpretation of probability theory, as I do, a probability is a degree of belief. Janyes's book Probability Theory: The Logic of Science has a good discussion of this perspective.

    Occam's motivation and the validity of Occam's razor are independent issues; I'm not too interested in debating the former.

    Well, then we appear to disagree on whether models get simpler or more complex.

    Occam's razor doesn't state that the simplest theory is right. It states that we should place the most belief in the simplest theory consistent with the evidence, in the absence of any evidence favoring the more complicated theories. I would still like to see you address the question of why you wouldn't prefer a 100th-degree polynomial fit instead of a linear fit, since it's "statistically unlikely" that the linear theory is right.
     
  18. Nov 14, 2003 #17

    Ivan Seeking

    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Occam’s Razor has lost its edge

    If you choose this as a definition, fair enough.

    I think the implicit philosophy does matter. This is clearly what motivates many common interpretations of this concept. Also, this is a philosophy forum. I didn't post this in mathematics or physics for a reason.

    You seem to have the wrong idea. If we consider that modern physics began with Newton, clearly we have seen a tremendous increase in the complexity of the physical models required to describe that observed. The latest data point to add complexity to the curve appears to be dark energy.


    As I said, in my first post:
    I have never disputed this point. Why do you keep ignoring this???
     
    Last edited: Nov 14, 2003
  19. Nov 14, 2003 #18
    Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Occam’s Razor has lost its edge

    Many common interpretations of Occam's Razor are based on Occam's theological motivations?? I doubt that!

    We see complexity go up and down. When more phenomena are discovered, more complex models tend to be proposed, but then people find a way of fitting these proposals into an elegant framework and the models get simpler again.

    You seem to be ignoring my response to your point, which is that all uses of Occam's Razor are obvious.
     
  20. Nov 14, 2003 #19

    Ivan Seeking

    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Occam’s Razor has lost its edge

    On a superficial level, no. But at the deepest level, any belief based premise, such as the more general notion taken from OR that we should expect simplicity in nature is ultimately theological in nature. Physicist do practice philosophy daily. It is argued that this is often not unrecognized. Here is a link to an article that discusses this notion. I have posted this in the past but it has been a while.

    http://physicsweb.org/article/world/15/4/2/1#2

    A macroscopic view - the complete history of physics - yields a more and more complex curve...thus far. How many dimensions do we work with now? Up to twelve? I think your Newtonian line in three dimensions has morphed into a hypervolume.

    I don't agree. Please see the link in the first post.

    There is a philosophy implicit in a thousand decisions a day. The article linked above describes the basis for this premise. This however is another discussion altogether.

    Again, this is a philosophical discussion. A rigorous mathematical treatment such as the one posted by you may or may not have anything to do with the premise my argument.
     
  21. Nov 14, 2003 #20

    NateTG

    User Avatar
    Science Advisor
    Homework Helper

    Occam's Razor:
    If there are two theories that make the same prediction, then the simpler theory is better.

    So, for example, according to OR, on classical physics scales - i.e. non-relativistic/non-quantum masses and speeds - classical physicis is the best theory.

    When you are dealing with domains where the theories differ, then you use the theory that is more consistent with your experimental data. (OR does not apply here.)

    So let's take a look at the example of lost keys:

    Theory 1: I lost my keys
    Theory 2: Space aliens stole my keys.

    Theory 1 makes the prediction that my keys will turn up somewhere that I've been.
    Theory 2 makes the prediction that there were aliens near me, so the aliens should have been observable.

    Clearly experience (for most people) is not consistent with theory 2. That eliminates theory 2 before OR really comes into play.

    There are several problems with the application of OR in general -- the notions of theory, the same, and better are not rigorously stated, and it is entirely concievable that a theory can trade conceptual simplicity for computational complexity. An example of this might be the standard model (a catalog of particles and interactions) vs. string theory (conceptually elegant, computationally painfull).

    A good example of Occam's Razor being applied is the history of the motion of the planets: Although Keppler's work is equivalent on the motion of the planets, it's usually not covered in favor of Newton's equations because Newton's equations are simpler, and make the same predictions.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?



Similar Discussions: Occam’s Razor has lost its edge
  1. Occam's Razor (Replies: 27)

  2. Lost (Replies: 7)

  3. The Fool's Razor (Replies: 13)

  4. Occam's razor (Replies: 29)

Loading...