Why Occam's Razor and the Simplest Explanation Matters

[sneez]

Occam's razor states that the explanation of any phenomenon should make as few assumptions as possible, eliminating those that make no difference in the observable predictions of the explanatory hypothesis or theory.

many ppl simplify this into "the simpliest explanation should be the correct one".

why this should hold? what is relationship of simplicity to "truth"/"reality". While its obvious that non-related assumptions to the problem should be eliminated, its not obvious that the simpliest explanation should be truth?

(i was thinking about this in connection to complexity theories, chaos theories. Yes, reductionism is powerfull tool (and only one we have) but why to limit ourselves? Many phenomena may be explored observing systems as whole and their interelated connectedness). (atmospheric dynamics comes to my mind as an example. Is just so well connected to bio,lito, hydro spheres, radiation, etc...)

 

[selfAdjoint]

Occam's Razor is not a test for correctness but a research strategy. Making up "explanations" out of whole cloth (multiplying entities without reason) is not likely to be a productive road to a correct account of things.

 

[vanesch]

I would even add to SA's post: Occam's razor serves as a guideline to favor one amongst empirically equivalent theories: namely the one that needs the least number of concepts. It is important to stress that the theories should be empirically equivalent: a simpler theory that is making WRONG predictions cannot claim superiority through Occam's razor.

Example:
A) Newton's theory of gravity
B) Newton's theory of gravity, except for the planet Mars, which is not held in orbit by the Sun's gravity, but by invisible angels pushing it around on its orbit
C) The theory that says: "things in space don't move".

Clearly, C is the simplest theory. But it is wrong: it doesn't explain the observed motion of the planets.

A and B are empirically equivalent: they make the same predictions and are both on the same level in agreement with observations.
However, in order to do so, B needs extra concepts where A doesn't need them. So Occam's rasor applies here, and prefers A.

 

[-Job-]

The simplest explanation offers the smallest number of possible contradictions, generally speaking. A given explanation of a phenomenon has to be coherent with other theories and observations. The shorter the explanation the less room it leaves for incoherences.
Starting from a large group of unverified explanations of a given phenomenon, of varying complexity, the fittest explanation is the one left standing. It's not guaranteed that the simplest explanation is the fittest as it's perfectly possible that a one-line theory is completely wrong. But suppose that explanation A is the correct one, and that B, C ... Z each include A and a number of other extra statements. If we were to go through a process of verification, to identify the incorrect theories, assuming each of the extra statements is correct with a probability p, then the probability that an explanation is incorrect is dependent on the number of "extra" statements it contains.
From this perspective it holds that the simplest explanation is likely the correct one. Perhaps a better phrasing of Occam's Razor would be: The simplest explanation, out of a number of explanations that include, in their body, the correct explanation, is more likely the least incorrect one.

 

[Hurkyl]

There are lots of theories that will be consistent with experimental evidence.

Some are so ridiculously general that they are of no use! E.G. the theory that "stuff happens" is consistent with every experiment we make, and ought to be consistent with any experiment we ever do, but it tells us nothing useful.


So when selecting among theories that are consistent with the data, you should prefer theories that are less "pliable", because in some heuristic sense, a rigid theory that fits data is more likely to be "right", whereas a pliable theory that fits data is more likely to simply be "flexible".

 

[sneez]

thanx for clarification to all.

From this perspective it holds that the simplest explanation is likely the correct one. Perhaps a better phrasing of Occam's Razor would be: The simplest explanation, out of a number of explanations that include, in their body, the correct explanation, is more likely the least incorrect one.

what has simplicity to do with correctness or incorectness? (an example of this not being true is medical research. I have in mind a cancer therory developments show that "simplification" through rigid reductionism was wrong. (im not in any sense proposing not reductionism). How to determine simplicity in general sense?

Occam's Razor is not a test for correctness but a research strategy.

i like this view. However, this is not the meaning the occam theory is used by general public and scientific comunity (from several book i have read).
It always mean (by general acceptance) correctness of theory. (religious ppl claim since God is by this theory simpliest explanation of certain events/"world", it proves god theory correct.)

So when selecting among theories that are consistent with the data, you should prefer theories that are less "pliable", because in some heuristic sense, a rigid theory that fits data is more likely to be "right", whereas a pliable theory that fits data is more likely to simply be "flexible".

makes sense to me..., how to determine correct "amount" of rigidness?

 

[Hurkyl]

makes sense to me..., how to determine correct "amount" of rigidness?

Empirically.


Actually, I find this extremely similar to artificial intelligence / data mining / et cetera. The buzzword to look for is "overfitting".

 

[russ_watters]

what has simplicity to do with correctness or incorectness?

I think you may be looking at it backwards. Occam's Razor is about simplicity only in how it is affected by assumptions. Basically, if one theory has two assumptions and another has four, the one with four has twice as many opportunities to be wrong and twice as many holes to plug. Therefore, it could be said to be statistically twice as likely to be wrong.

 

[sneez]

I think you may be looking at it backwards. Occam's Razor is about simplicity only in how it is affected by assumptions. Basically, if one theory has two assumptions and another has four, the one with four has twice as many opportunities to be wrong and twice as many holes to plug. Therefore, it could be said to be statistically twice as likely to be wrong.

this makes sense, however, two less assumptions in a theory make it statistically less prone to error, but not realistically less prone to error. Ex: Ether theory was in my understanding simplier and flowing from what we already knew about waves (assuming that light was considered wave). GR or SR was very much fought against on the basis that it was not simple enough for back in the time mode of thought/thinking. The two less assumptions just might be what makes the theory wrong.

While i would agree that irrelevant assumptions are bad, i don't see a strong point in rejecting a theory based on occam razor assertion.

thanx for the discussion.

 

[Lars Laborious]

I think you may be looking at it backwards. Occam's Razor is about simplicity only in how it is affected by assumptions. Basically, if one theory has two assumptions and another has four, the one with four has twice as many opportunities to be wrong and twice as many holes to plug. Therefore, it could be said to be statistically twice as likely to be wrong.

... and twice as likely to be right?

 

[3trQN]

Divide and conquer and Go forth an multiply, growth in both cases but one is the inverse of the other, as truth is to falisy. One creates complexity, the other reduces it to understandable chunks.

Why the principle of parcinomy (Occam's Razor) is often correct i believe is due to entropy and the laws of reason and energy conservation.

 

[russ_watters]

... and twice as likely to be right?

No, just twice as likely to be wrong. That's the point: assumptions, particularly the utterly baseless ones like the aether below, allow a theory to be made consistent, but often add nothing in the way of likelyhood of being correct.

this makes sense, however, two less assumptions in a theory make it statistically less prone to error, but not realistically less prone to error.

Not just statistically less prone to error, but actually less likely to be conceived incorrectly. It is almost psychological: In an effort to make things work, people often start making assumptions to fit a model. Often these assumptions are simply products of the mind of the scientist - they don't just add error, but they are often a sign of flawed logic itself.

Ex: Ether theory was in my understanding simplier and flowing from what we already knew about waves (assuming that light was considered wave). GR or SR was very much fought against on the basis that it was not simple enough for back in the time mode of thought/thinking. The two less assumptions just might be what makes the theory wrong.

That may depend on historical context, but when first proposed by Einstein, SR was the theoretically simpler theory. It made only two assumptions, both of them basic and both of them have evidence to support them: that the laws of the universe are the same everywhere and that the speed of light is constant in a vacuum. I consider those "soft" assumptions, since we have evidence to support them but for logical reasons, we must consider them assumptions in order to develop the theory.

Constrast that with the aether, which is an assumption for which there is zero evidence - even worse, some aether theories actually say that it is utterly undetectable. It is completely superfluous.

 

[HallsofIvy]

Another point: science is about disproving theories. We can never prove that a theory is true, we can only prove that a theory is not true and thus reduce the number of competing theories. By starting with "simple" theories, you start with those that, if they are untrue, can be disproved quickly.

 

[arildno]

A VERY good post, HallsofIvy!
But FOUR copies of it?

 

[vanesch]

While i would agree that irrelevant assumptions are bad, i don't see a strong point in rejecting a theory based on occam razor assertion.

I'd say there are cases where Occam's razor's application is clear, and there are cases where it is just a heuristic which is dependent on the way one wants to interpret it.

An example comes to mind. When there was the cold fusion hype, in order to bring into agreement the "observations" and "theory", grossly following steps were taken:

1) some small thermal inbalance was observed (in a somewhat dubious setup)

--> horray, fusion !

(this was by far not the only possibility for the thermal flux in the apparatus)

2) but how could this happen in a metallic crystal ?

--> hypothesis of special transport of deuterium nucleae in the crystal, a previously unknown phenomenon

3) how come we don't have any neutrons ?

--> hypothesis of new fusion reaction in which no neutrons are produced

4) how come there are no gammas ?

--> hypothesis of coherent absorption of gammarays into the xtal

...

Clearly, in order to keep the explanation "horray, fusion" a lot of extra assumptions were needed which only served the purpose of saying that there could still be fusion...


Here, I think it was wise to use Occam's razor.


But in other cases, it is less evident whether Occam's razor is to be applied. For instance, look at Newtonian mechanics. Its mathematics is simpler if we can assume the existence of an unobservable absolute space, but it goes against the relativity principle of Galileo.
(this is in fact the same situation as the earlier notion of ether theory in SR).

What is Occam to cut ? The absolute space (unneeded concept) or the more involved mathematical structure when one needs to formulate things without this concept ?

It is probably a matter of taste and intuition, what is the "simpler" situation.

 

[loseyourname]

The classic example of a clear cut application of Occam's razor comes from planetary mechanics. Before we had the ability to go into space and observe for ourselves that planetary orbits were indeed elliptical and that the sun was at the center of the whole business, Occam's razor was the only reason to favor a Copernican model over a Ptolemaic model, since they both had equal explanatory power.

The best example of a modern-day application of this comes from cladistics, in which a slightly altered version of the razor called the "parsimony principle" is employed. In grouping organisms taxonomically according to the order in which each diverged from a common evolutionary ancestor, the cladogram that posits the least number of evolutionary changes is always the one chosen, even though others might have equal explanatory power.

We can see that the value of the razor has changed somewhat between these two examples. In the former case, it has epistemological value. When the Ptolemaic models were continually shown to not have predictive power, they were simply added onto ad hoc, which is not an acceptable way to conduct scientific inquiry. The hypothesis or model with better explanatory and predictive power is the one that is to be preferred, not simply for aesthetic reasons, but for reasons of intellectual integrity. We must be prepared to reject those hypotheses whose predictions are not born out by replicable empirical observation.

In the latter case, however, the razor has taken on more of a pragmatic role than an epistemological one. While probability does dictate that the simpler cladograms will be correct more often than the very complicated cladograms, it will not always be the case that the model chosen for the sake of parsimony is the correct model. In fact, it is accepted that this will not always be the case. However, in this usage of the razor the simpler model, given that it has equal explanatory power to the more complex model, is to be favored in part for purely practical reasons, that is, because it is easier to work with and also because it gives a clear cut reason for choosing one model over another when hundreds of possible cladograms might fit a given taxonomic family equally well and there is no other reason to choose one over the other.

The other classic examples of entities pushed out of scientific theorizing by Occam's razor include caloric, phlogiston, the ether, and elan vital, each of which provided no additional explanatory or predictive power over their mechanical counterparts. In certain cases, these theories might actually be simpler than the mechanical ones, but Occam's dictate did not state that the simpler model is always to be preferred (as the modern parsimony principle employed in cladistics does state). It stated rather that we are not to multiply entities beyond necessity. Ergo, if something like elan vital is not required to explain biological functioning, then do not posit its existence, even if it would simplify the theory.