Why Won’t You Look at My New Theory?

[Total: 18    Average: 4.7/5]

In any forum where science is discussed, there will always be people who have a great new personal theory and can’t understand why no one else is interested in it. Here at PF we have rules about this, but I want to look at the more general question of why there is apparently so little interest in such personal theories, independently of whatever rules a particular forum might have. Is it just because people are closed-minded, unwilling to consider new ideas? Or is there some more cogent reason?

Of course, personal theories cover a very wide range; but here I want to focus on a particular kind of personal theory, one which arises from the following scenario: A new observation or experimental result is reported that appears to be inconsistent with what we think we already know. Rather than pick on recent examples (of which there are plenty), I’ll give two examples from the history of Solar System astronomy in the 19th century, since the outcomes of these cases are both well established by now so they can serve as good test cases without raising anyone’s hackles. Here they are:

(1) In the 19th century the motion of the Moon appeared to be inconsistent with the predictions of Newtonian gravity–i.e., the Moon was observed in the sky at locations which were different from those predicted by Newtonian calculations from previous observations. The differences were small, but the calculations and observations were believed to be accurate enough to make them significant.

(2) In the 19th century the motion of Mercury also appeared to be inconsistent with the predictions of Newtonian gravity. Here, again, the differences were small, but it was believed that the calculations and observations were accurate enough that the discrepancy was significant.

The question then arises, what is the reason for the apparent inconsistency? There are basically two possibilities:

(A) The inconsistency is only apparent; it is because we haven’t worked out carefully enough the implications of what we already know. This was the case for the apparent anomaly in the motion of the Moon: it turned out that there were small perturbations due to the other planets that hadn’t been correctly calculated, and when the calculations were corrected, the discrepancy between the theory and observation went away. This means, of course, that people’s belief prior to this discovery, that the calculations of the Newtonian prediction were correct, was in error.

(B) The inconsistency is real; it is because there is some fundamentally new effect going on that our current theories don’t comprehend. This was the case for the anomaly in the motion of Mercury. It turned out that the current theory of gravity (Newton’s theory) was not correct. When Einstein replaced that theory with the general theory of relativity, one of the first predictions to be re-calculated based on the new theory was the motion of Mercury, and the correction to the Newtonian prediction due to general relativity brought the prediction into line with observation.

It is worth noting, by the way, that before GR was developed, scientists considered a more mundane explanation of the discrepancies in Mercury’s orbit: that there might be a small planet inside the orbit of Mercury that was perturbing its motion just enough to account for the discrepancy. But such a planet was never observed despite increasingly sensitive attempts to do so, and this possibility had been rejected by the time Einstein began working on GR.

I don’t think any reasonable person would disagree that, in principle, (A) and (B) above are both valid possibilities in any situation of the general type we are discussing. However, I think there is a vast disagreement between scientists and non-scientists about the relative frequency of occurrence of (A) and (B). Many nonscientists seem to believe that situations of type (B), where a fundamentally new effect is there and the theory has to be modified to account for it, are common in science; whereas all good scientists know that in fact, almost all situations turn out to be of type (A), where the theory is fundamentally correct but its implications haven’t been calculated accurately enough. This is not because scientists are lazy or incompetent: it’s because calculating the predictions of a known theory is not a cookie cutter mechanical process but a separate intellectual effort in its own right, and it is subject to the same kinds of errors as any other theoretical efforts.

I don’t know exactly why so many nonscientists seem to believe that type (B) situations are vastly more common than they actually are, but I can think of several possible reasons:

(1) Type (B) situations are far more exciting, so historians of science tend to focus on them, while the vastly more common type (A) situations are left out of popular accounts. So the nonscientist’s erroneous belief about the frequency of type (B) situations is due to a straightforward sampling bias.

(2) Type (B) situations, because they intrinsically involve the overthrow of some part of an accepted theory and its replacement with a new theory, always involve a dynamic of resistance by the scientific community to the new theory. Scientists understand that this resistance, even to theories that ultimately win out, is rational, and a necessary part of science; but nonscientists just focus on the underdog fighting against the establishment because it feeds their pet beliefs about such situations. So nonscientists’ erroneous belief that type (B) situations are common is just a special case of the general belief (which is also erroneous) that underdogs fighting establishments are usually right.

(3) Type (B) situations hold out the hope that, in principle, anybody can overthrow an accepted scientific theory. Newton was a lowly college student when he came up with his laws of motion and his theory of gravity. Einstein was a patent office clerk who had failed to obtain an academic job when he published his famous papers on special relativity and quantum theory. Nonscientists look at these examples and draw the (erroneous) conclusion that you don’t need to actually know anything about the established theories in order to overthrow them; you don’t need to go through all the bothersome stuff that members of the scientific establishment do, like taking classes, getting degrees, doing research, publishing papers, going through peer review, etc. Just come up with a great new idea and you’re set.

Scientists, though, understand that Newton, Einstein, and the other scientists who found themselves in real type (B) situations did do all that stuff–they did learn the established theories inside and out before they tried to overthrow them. They did their “homework” in an unconventional way, but they still did it. So nonscientists’ erroneous belief that type (B) situations are common is due to their erroneous belief that you can come up with a new scientific theory that works, without actually having to do the work involved in understanding what is currently known.

Of these possibilities, the third would appear to be the one most likely to spawn personal theories of the kind I referred to at the top of this article. And, conveniently, it also offers an explanation of why others are so seldom interested: because the obvious counterpoint to the view that anybody can overthrow an accepted scientific theory is to go too far in the other direction and believe that only professional scientists–those with degrees or other credentials, etc.–can come up with a valid scientific theory. So of course any random person posting on an internet forum can’t possibly have a valid theory.

But the fact that this heuristic works 99.9999% of the time still does not make it right. Unfortunately, I think a large part of the reason it is so often adopted is that professional scientists themselves promote it–wittingly or unwittingly. For there is a flip side to the observation that nonscientists often come up with personal theories that nobody listens to: the observation that professional scientists, when talking to nonscientists, often fail to distinguish the varying levels of confidence we have in different parts of science, and often present science in a way that encourages people to say “Oh, wow!” and accept whatever they are told on the authority of the scientist, rather than to think critically and try to build an understanding of their own. This is why PF also has rules about acceptable sources: because even scientists can’t always be trusted to fairly represent science. At least in a peer-reviewed paper there are other experts looking who can call them on it if they go too far afield (though admittedly that doesn’t always work either).

I’ve painted a fairly gloomy picture in this article, but please bear in mind that I’m focusing here on something that only makes up a small fraction of all the posts on PF. Most discussions here don’t raise either of the issues I describe above. But if you’re tempted to post about your personal theory, or if you’re tempted to ask a question based on a pop science source, it might be worth taking some time out to consider.



228 replies
Newer Comments »
  1. Alex Klotz
    Alex Klotz says:

    One thing I've wondered: suppose somebody is not crazy, but is also not in the physics community, but has read a lot and think they have come up with something new. How do they get people to read it, to help them figure out whether their idea is right or wrong? Pretty much every serious online physics community has rules against this. They can write up a paper and submit it to a journal, but the role of peer review isn't really to be a first-pass vet of peoples' ideas, and a person not in the field will likely have papers rejected pretty quickly. So where should non-crazy people turn to, to get help?

  2. John Baez
    John Baez says:

    ogg wrote:"Could you confirm that Newtonian Gravitation (with instantaneous interactions) is consistent (with classical physics ca 1890-1900)."Yes.  Maxwell's equations of electromagnetism are not consistent with pre-special-relativity ideas about how things should look in a moving frame of reference, but that's a separate matter. Newtonian gravitation is perfectly consistent with these ideas.  In fact it's the best theory that uses these ideas."My impression is that "neo"Newtonian Gravity (finite speed of c & force) is quite consistent with the Solar System's orbital mechanics – is this right or wrong?"If you say the gravitational force moves at the speed of light and obeys a "delayed" force law, conservation of angular momentum breaks down.  In other words, suppose each a particle is attracted to where it would _see_ each other particle _was_, feeling an inverse square force.  Then the particles are not attracted toward their current center of mass!  This means angular momentum is not conserved.  Orbits would spiral down.  This effect is big enough that we can be sure by now that's not how things work.For details see:

  3. John Baez
    John Baez says:

    My comment here is poorly formatted.  On this particular page,  unlike the rest of Physics Forums, I don't see any way to edit my own comments.  I feel I'm missing a trick.

  4. Chestermiller
    Chestermiller says:

    I think the article is wonderful!!!  Very perceptive!  So, if I travel at 4.2 times the speed of light, will I fall through a wormhole and end up at alpha centauri, or will I pass through the event horizon of the Milky Way super-massive black hole?

  5. eltodesukane
    eltodesukane says:

    "resistance, even to theories that ultimately win out, is rational"–As many have said:"An extraordinary claim requires extraordinary proof." (Marcello Truzzi)"Extraordinary claims require extraordinary evidence." (Carl Sagan)"The weight of evidence for an extraordinary claim must be proportioned to its strangeness." (Pierre-Simon Laplace)

  6. eltodesukane
    eltodesukane says:

    interesting reference:Aberration and the Speed of Gravity (1999 dec) (S. Carlip)(Apparent instant action at a distance in GR. The observed absence of gravitational aberration requires that "Newtonian" gravity propagate at a speed >2×10^10 c. Aberration in general relativity is almost exactly canceled by velocity-dependent interactions. This cancellation is dictated by conservation laws and the quadrupole nature of gravitational radiation.)

  7. Jeff Rosenbury
    Jeff Rosenbury says:

    There is a strong economic motive for professors to discount work from outside academia.  Their time is valuable, and much of its value comes from the respect for current science. So checking some crackpot's idea is a two fold hit to their income. First, it's time that could be more profitably spent on learning more science, grading papers, doing their own research, watching a movie, bowling, etc. Second, if the crackpot turns out to be right, (Rare, but it happens. Ex. Tesla.) he undermines the value of the professor's work. (Ex. "War of the currents.") One of the big reasons for universities to be open and publicly supported is to limit this economic disincentive to explore new ideas. Most professors I've known love finding smart students and hunt them out. I don't think it's a coincidence the War of the Currents was promoted by the first commercial research lab, and not by a public university. Business and science don't always play well together.

  8. Giovanniontheweb
    Giovanniontheweb says:

    Human is indeed a motivation-driven being, if I’m spending my time typing on this small keyboard is not because I care about transmitting to you any useful information but because I do care about my inner integrity.I did appreciate few last posts about pop science that I’ve found well done, with care and relevant background, as you can share when you are around 18 with friends of yours. I’ve been surprised lately about how many questions coming from kids I could’t answer when my clock says 50 turns, neither I feel confortable to say that courious pop science people are less knowledgeble than myself . I do understand bad engineering as it comes around and, by axtrapolation I might understand scientist’s mood. As well, I do acknowledge most of the relevant humans’ discoveries happen by mistake and the following points look clear to mea)    Detecting scientific mistakes happen in the labs where common people spend very limited timeb)    The so called scientists are very much format in their own thinking path that makes them at the same time very clever and very limitedWell, nothing is lost, common to all beings, including the so called animals, looks to be curiosity, I hope I’ll be able to deal with my neibour’s kid next question, maybe we will have less pop science in the future and more real scientists..

  9. ebos
    ebos says:

    I like to think that most people have a teeny bit of the "B" type inside. Just like the Rock Star scenario… or the potential Cult Hero. Yeah, I think that's me – the Cult Hero. Although shredding that Fender Strat in front of 100,000 screaming fans does run a close second. Anyways, now I can also accept the "B" Side scientist ; )

  10. Giovanniontheweb
    Giovanniontheweb says:

    can you fly as an eagle in the skyes, I guess you can't unless sitting in 1st on a boeing. The eagle can feel what you cannot and it can use it. Being able to intuitevely understand differs from being able to use the lernt books to explain it, we can keep mixing it up if you like. I can be very good in repeating books yet it took me a while to feel the first page.

  11. rjbeery
    rjbeery says:

    I think there's a romantic element in rooting for the underdog; it certainly empowers anyone who identifies with being one.  Similarly there is a feeling of superiority if you choose to advocate something that the majority does not, which I believe is the motivation behind conspiracy theories, for example.

  12. fundamentally
    fundamentally says:

    John Baez, one of the contributors to this discussion wrote an article "Division algebras and quantum theory” . It is very fundamental and important to those that want to understand the foundations of physical reality or want to deliberate about that. Not many people will understand the importance of this paper because most scientist are applying their capabilities for supporting applied science that brings their regular earnings. This happened to me also. Since I am retired I have more time free for thinking deep about the foundations of physical reality and that free time appears to be an important resource. Already in the sixties during my studies I became suspicious about the direction that current physics has taken. Some leading physicists guided the direction that quantum physics took and that guidance was accepted without much criticism. Still some notions such as elementary particles and photons are full of contradictions. Thus something must be wrong with our knowledge of the deep foundations of physics. My conviction is that the discovery of Garret Birkhoff and John von Neumann indeed introduced a suitable foundation of physical reality and that this foundation is built into physical reality including the concept of the Hilbert space. This however sins strongly against the so called scientific method that most physicists still use as a defense that covers the actual reason of their attitude.  If the full consequence of the discovery of the orthomodular lattice is accepted, then it becomes possible to dive much deeper into the foundations of physics than experiments such as the LHC can bring us. The main problem for foundation researchers is that applied physics flourishes well without the knowledge of the deep foundation of physical reality and that most scientist want to spend their resources in a more profitable way. My career was in high tech industry. I had no time to think about the orthomodular lattice and its consequence until I retired. Now I have that time and in 2011 my personal research project that investigates the foundation of physical reality got its current name "The Hilbert Book Model". It uses a pure mathematical test model in order to see whether it can raise sufficient similarities with what we know from physical realty via experiments and more direct observation. The results of the project are so unorthodox and so controversial that I have little hope that my reports will ever pass peer-review. Also arXiv's endorsement requirement will pose a significant barrier. That is why I publish on vixra. The current state of the project fulfills my own selfish needs. I think that I understand a lot more about the foundations of physical reality than most others do. My papers are free accessible and do not reserve copyrights. May be you will find the reverse bracket method, which I discovered, a useful tool. It helped me understand Hilbert spaces much better.

  13. thefurlong
    thefurlong says:

    The problem is that these days, going into scientific research is prohibitively expensive for many people (like me), especially if you're already paying off debts from your first time in school.  I would love to spend all of my time studying physics rather than doing these intellectually non-stimulating computer programming tasks that I am paid to do every day.  However, I would also prefer not to add $80k more debt to the debt I already have, and I certainly don't want to find myself living in poverty as a graduate student.The fact is that the troglodytes running my country have decided that encouraging people to go into scientific research is not a priority.  Additionally, the prospect of doing university research is less than enticing at this point, given that university boards think that investing in frequent, safe, results is way more important than investing in attempts to truly innovate.  These aspects of research create a fairly imposing barrier to entry for many people.How many innovators are there in this world who have fallen by the wayside because their true interest lies in making fundamental progress rather than incremental progress?  How many others never reached their full potential because, in order to actually obtain knowledge, they have to sacrifice their financial well-being by doing the financial equivalent of purchasing a car every year for 4 years, or more.And heaven forbid they have a learning disability, which is just mild enough not to count as debilitating according to some arbitrary benchmark, but just severe enough to negatively affect their academic performance, which, in turn, affects the prestige of the institution that accepts them for research?

  14. indimingo
    indimingo says:

    One thing that you've not incorporated into your insight is this, quite important and very valid point.  Incorrect/unlikely theories that people come-up with and post in a place like this forum, regardless of how many flaws they might be seemingly obviously be riddled-with, inspire the wise scientist to find a whole host of previously never-considered perspectives that may point them in a wonderful new direction!   Sometime, the wilder the concocted theory (actually 'hypothesis' would be more appropriate) is, the more the 'imaginative-juices' start to flow!  For this reason alone, I can very much appreciate some of the weird things people come up with, relative to mainstream science, or even SOMETHING that reasonably approaches it!.Another way that it can be "productive", is when people look past the pseudoscientific ideas as something worth just their face-value, and take the time to explain/teach the person the 'whys' and 'whats' of their flaw(s).  Hopefully some people can accept what the problems are once they see them and/or understand them – although I realize very few truly do so!  That's just a bonus, but, the truly productive energy is gained by the one who tries to present sound/valid arguments to explain the errors in such a way that they, theoretically, could understand it!  To go over it several times and from several different approaches, while remaining cool and not getting frustrated and just throwing their hands up in the air and walking away!  That improves one's teaching skills, true, but even most critically important, it reinforces the scientific concepts they use to 'teach' someone far more deeply into the wiring of their brain!  Using everything you can to try to talk someone out of their idea is a profoundly difficult challenge to accept – but the end-result is marginally as important than realizing that it's through teaching that we learn the most.  So I say they are simply free lunch for the mind!The downside is obviously 'clutter' and a need for extensive moderation to quickly separate them if/when possible…But it's a small price to pay if you take what I've said at all seriously.

  15. Borg
    Borg says:

    Nice article PeterDonis. I liked the reasons why nonscientists choose B over A. From the crackpot threads that I’ve seen, it often seems that the basic motivation for them is wanting to be famous for coming up with a new theory – regardless of how little sense their theory makes. Some of them do go to great lengths to ‘prove’ themselves but they really don’t know even the basics.

  16. Dale
    Dale says:

    Excellent Insight!

    I don’t know if it would fit in with this Insight, but I have always found it interesting how scientists and nonscientists view the status of a theory in the light of contradictory evidence. The example of Mercury is relevant.

    Nonscientists tend to categorize theories as “right” or “wrong”. So regardless of how many experiments confirm a theory, a single counterexample (of type B) makes it “wrong”. In this sense confirmatory evidence is considered much weaker than contradictory evidence.

    Scientists tend to think in terms of domains of applicability. A counterexample (of type B) does not destroy all of the supporting evidence, it simply places a limit on the domain where we believe that the theory applies. Newtonian gravity is not “wrong” but it only applies in a certain region of experimental conditions and can be inaccurate outside that domain.

    A couple of nice features of the scientific “domain of applicability” view are that it places value on both confirmatory and contradictory evidence. It also applies well to the type A contradictions. Since we never know the entire state of the whole universe, whenever we attempt to apply a theory to a scenario we always use some simplifying assumptions. A type A contradiction tells us that our simplifications have some limited domain of applicability. Even if the theory still applies, some previously neglected term is no longer negligible in this new domain.

Newer Comments »

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply