What Unsolved Mysteries in Physics Lack a Theoretical Explanation?

[TeethWhitener]

why the shift?

I think you’re jumping to unwarranted conclusions again. I gave a few examples of the pre-Einstein work scientists did, but I’m not familiar with every piece of science from that period. It could be that there was no shift: tons of theories may have been published and we simply don’t hear about them because they were wrong. I think asserting there was a shift (or for that matter, asserting that scientists weren’t bothering to do anything) requires a lot of supporting evidence that you haven’t provided.

 

[Vanadium 50]

(except the fact that there is another planet between Mercury and Sun which is not really a theory

Why is that not really a theory?

 

[DaveC426913]

Why is that not really a theory?

I guess it depends on whether someone actually took the time to develop it. Otherwise, would it not more suitably be called a conjecture?

 

[Vanadium 50]

I'd like @kelly0303 to be the one to say what he or she means, rather that start the PF guessing game.

 

[kelly0303]

I'd like @kelly0303 to be the one to say what he or she means, rather that start the PF guessing game.

What I meant is that, as far as I know, that idea was not heavily developed into a solid mathematical formulation (if it was and I am not aware of it, I would like to read some papers about it). For example, when the light curves of the galaxies were not what we expected, we didn't say: "oh it's just some matter we can't see, let's move on". The scientific community came up with thousands of theoretical models trying to explain it, and hundreds of experiments to test it. I am not aware of this with Mercury precession. Again, if I am wrong please show me attempts to solve that problem (I am actually really interested in that, I just didn't find anything). Thank you!

 

[TeethWhitener]

What I meant is that, as far as I know, that idea was not heavily developed into a solid mathematical formulation (if it was and I am not aware of it, I would like to read some papers about it). For example, when the light curves of the galaxies were not what we expected, we didn't say: "oh it's just some matter we can't see, let's move on". The scientific community came up with thousands of theoretical models trying to explain it, and hundreds of experiments to test it. I am not aware of this with Mercury precession. Again, if I am wrong please show me attempts to solve that problem (I am actually really interested in that, I just didn't find anything). Thank you!

Start here:
https://en.wikipedia.org/wiki/Vulcan_(hypothetical_planet)
plus links therein. Clemence's Rev. Mod. Phys. article gives some pertinent observations:

In justice it should be said that the questions involved are not simple ones, but are complicated by three causes: (1) Observations of Mercury are among the most difficult in positional astronomy. They have to be made in the daytime, near noon, under unfavorable conditions of the atmosphere; and they are subject to large systematic and accidental errors arising both from this cause and from the shape of the visible disk of the planet. (2) The planet's path in Newtonian space is not an ellipse but an exceedingly complicated space-curve due to the disturbing effects of all of the other planets. The calculation of this curve is a difficult and laborious task, and significantly different results have been obtained by different computers. (3) The observations cannot be made in the Newtonian frame of reference. They are referred to the moving equinox, that is, they are affected by the precession of the equinoxes, and the determination of the precessional motion is one of the most difficult problems of positional astronomy, if not the most difficult. In the light of all these hazards it is not surprising that a difference of opinion could exist regarding the closeness of agreement between the observed and theoretical motions.

So it looks like there was a solid 70 years of experimental and theoretical work before Einstein's general relativity took care of the problem of Mercury's precession. A propos of nothing, Vera Rubin's first paper on dark matter came out almost 50 years ago.

 

[sophiecentaur]

I see what you mean. So, why the shift? From lots of experiments before theory to lots of theories from scarce data.

The simple answer to that one is that the experiments to investigate new phenomena get more and more complicated and call for higher energies and much more money.
Experiments in the old days were easy to carry out and many people used to carry them out in parallel. One experiment to observe the Sun close up (the Parker Probe) is all we can afford for a generation or two.

 

[ohwilleke]

It is also worth recognizing that the relative amounts of experimental v. theoretical work differs profoundly by subdiscipline in physics.

Low energy quantum physics experiments are small budget, single university laboratory scale efforts with a couple to half a dozen scientists involved. Usually, they don't discover things that aren't already strongly suspected by existing Standard Model theory but they can probe important corners of untested assumptions or demonstrate something that is known theoretically but still very counterintuitive in very compelling ways. There are thousand of experiments like this going on, most of which don't make the headlines.

We have hundreds of relatively small to medium budget astronomy observation devices ("telescopes" really doesn't do justice to devices that cover the entire EM frequency range, gravitational waves, neutrinos, and other non-photon particles from space deceptively called "cosmic rays", some of which are space based and some of which coordinate global data sets and new methods, as well as devices designed to detect dark matter and axions) and research projects, from a lot of independent groups that produce a torrent of new experimental results on a daily basis. So, while there are some serious theoretical issues being explored, there is a lot of incoming new and improved data to distinguish between and evaluate the theories. Individual astronomy collaborations have scientists who number in the dozens or less (although there are coalitions of collaborations that are pretty ad hoc that aggregate multiple devices and observations that are collectively quite large).

As a practical matter, this means that some of the big unsolved problems, particularly those related to dark matter and dark energy, have an immensely growing amount of experimental data that can be use to test, constrain and devise theories with.

Neutrino physics is intermediate. There are about a dozen major neutrino oriented experiments out there from reactor source neutrinos, to Earth and space natural source neutrinos, to neutrino-less double beta decay experiments, etc. and these are also "medium budget" enterprises. These typically involve a dozens to hundreds of scientists in their collaborations. This is generating enough new data that theoretical work doesn't have to go too far afield from the data.

We've gone in a relatively short time frame as science history goes, from discovering the neutrinos have mass and oscillate, to measuring the parameters related to the oscillation and bounding potential absolute neutrino masses with workable precisions that are usually as good as or better than QCD after six decades of active experiments.

In contrast, in high energy physics, several of the active experiments are all different groups using the Large Hadron Collider, at unprecedentedly high costs, and there are maybe two or three other experiments that are currently collecting data in much less ambitious experiments that are "medium budget" efforts. The LHC involves several thousand scientists. It is in HEP where the new data flow is such a trickle compared to theoretical output.

The LHC has discovered the Higgs boson and is devoting immense effort to confirming that it behaves like the SM Higgs boson which it does across the board within margins of error which are rapidly shrinking. The LHC has ruled out myriad other kinds of new physics, the search for which was the main justification for building it. It has found some anomalies (e.g. the possible lepton universality violations in B meson decays), but for a variety of reasons most of the anomalies it has found so far have evaporated as more data was collected, and there is every reason to expect the same result from the current anomalies.

One dirty little secret of HEP is that QCD experiments routinely have theoretical predictions by different means that are all over map and not infrequently QCD experiments produce results seriously in tension with the theoretical predictions. But, for the most part, this doesn't make headlines because both the predictions and the measurements are routinely so imprecise (accuracy to the 1% level or worse is pretty much par for the course) compared to EM and weak force processes, because many past discrepancies have not persisted, and because nobody really thinks that the very elegant, but very difficult to operationalize, version of QCD that the Standard Model enshrines is wrong. Almost everybody blames poor approximation methods when experiments and theory don't match properly, rather than a fundamental new physics flaw in the underlying theory.

 

[meopemuk]

A very simple experiment that can have serious consequences:

R. de Sangro, G. Finocchiaro, P. Patteri, M. Piccolo, G. Pizzella, "Measuring propagation speed of Coulomb fields", Eur. Phys. J. C, 75 (2015), 137. arXiv:gr-qc/1211.2913v2.

Eugene.

 

[ohwilleke]

A very simple experiment that can have serious consequences:

R. de Sangro, G. Finocchiaro, P. Patteri, M. Piccolo, G. Pizzella, "Measuring propagation speed of Coulomb fields", Eur. Phys. J. C, 75 (2015), 137. arXiv:gr-qc/1211.2913v2.

Eugene.

Subsequent commentary on it somewhat undermines that paper. https://arxiv.org/pdf/1605.02545.pdf which is:

A note on "Measuring propagation speed of Coulomb fields" by R. de Sangro, G. Finocchiaro, P. Patteri, M. Piccolo, G. Pizzella
D. M. Gitman, A. E. Shabad, A. A. Shishmarev
(Submitted on 26 Apr 2016)

 

[meopemuk]

Subsequent commentary on the commentary restored the balance.

R. de Sangro, G. Finocchiaro, P. Patteri, M. Piccolo, G. Pizzella, "Why the interpretation of ``Measuring propagation speed of Coulomb fields'' stands", Eur. Phys. J. C, 77 (2017), 75. arXiv:gr-qc/1611.06935v1

Eugene.