Wiktorius I said:
TL;DR: Does physics establish that mass, particles, and fields are fundamental, or could they be emergent properties of a deeper physical structure?
Are mass, particles, and fields fundamental physical entities, or could they be observable consequences of a deeper physical structure? I am particularly interested in what experimental evidence allows us to distinguish between these possibilities.
They aren't necessarily fundamental, but we have no real idea what kind of deeper physical structure could cause what we see and there are no experiments or observations that are likely to take place in the next few decades that can tell us.
Generally, going as deep as we possibly can, we think of particles in the context of an effective field theory as local excitations of fields.
But, most popular accounts, because it is more intuitive to understand, talk in terms of particles, with fermions corresponding to what we think of as matter in a colloquial sense, and bosons corresponding to "force fields" for the fundamental forces of the Standard Model.
Generally speaking, the mass of particles (both fundamental and composite) is not understood as fundamental. Massive Standard Model fundamental particles like electrons, muons, tau leptons, and quarks and W bosons and Z bosons and the Higgs boson (with the probable exception of the neutrinos whose mechanism for acquiring mass isn't well understood at this point) get their mass from their interactions with the Higgs field. Massive composite particles (like protons and neutrons) get their mass from a combination of the masses of the fundamental Standard Model particles that they are made up of (to a small extent) and from the energy of the gluon fields that bind them together (to a great extent). But the sometimes elusive concept of mass-energy is something that is fundamental and conserved.
There is no real consensus over whether gravity is something that arises from the exchange of massless spin-2 gravitons (which are bosons) in a way analogous to the way that electromagnetism arises from the exchange of massless spin-1 photons, or whether gravity is a property of space-time itself which is either classical and smooth, or as in the quantum gravity approach of loop quantum gravity, in which case space-time itself is discrete at some ultrafine scale.
There is no consensus on this because we don't have any observational tools sufficiently powerful to distinguish between the hypotheses, and because while the math behind classical general relativity is hard, we haven't cracked the math behind quantum gravity theories of various kinds outside some very specific isolated cases in theories that don't hold true for all circumstances and have internal mathematical issues with thm that make quantum gravity theories in the general case intractable.
We don't have any well accepted theories that explain why the couple dozen plus experimentally measured physical constants of the Standard Model plus general relativity have the values that they do, although electroweak theory does establish that a few of the Standard Model physical constants are not actually mathematically independent of each other, so the number of independent degrees of freedom in the experimentally measured physical constants of the Standard Model is a few less than the number of experimentally measured physical constants (e.g. the electromagnetic coupling constant, weak force coupling constant, W boson mass, and Z boson mass are related to each other by a formula which has fewer than four independent degrees of freedom).
There are some theories out there that go part of the way, providing a road map for possible ways to explain more of the experimentally measured physical constants of the Standard Model and general relativity. But while these are bit more than speculative, none of them are established in a settled way that can secure a scientific consensus or anything close to that. Moreover, as impressive as our scientific methods are, imprecision in measuring some of these experimentally measured physical constants limits the degree to which we can distinguish one explanation for their values from another. Many key physical constants are measured only to percent or part per thousand levels.
But the failure of scientists so far to find any meaningful deviations from the Standard Model in particle accelerator experiments and other terrestrial tests of it (like ultra-precise measurements of the anomalous magnetic moment of the muon) profoundly constrains any alternatives that would predict anything experimentally different from what the Standard Model predicts.
I can't recall ever meeting a physicist who thinks that there isn't some deeper theory that could explain why those physical constants have the values that they do. But most physicists, having seen the parade of physicists before them who have failed to find that deeper theory, are content to not go looking for it very seriously.
As a practical matter, even if you found the deeper theory that explained why most of those physical constants have the values that they do, it wouldn't necessary change much for most physicists, because we've already measured those physical constants to high precision in most of the cases where precise values are important to precise calculations. Having the deeper theory might allow you to assign more precise values to the physical constants whose values are hardest to measure, which would be a helpful thing on the margins as a practical matter and very personally satisfying, but that might be just about it.
Of course, if we knew why those physical constants had the values that they do, we would probably also rule out all sorts of beyond the Standard Model theories and models in a way that would be more satisfying than the way that we do now. But it wouldn't have many very practical consequences.
There are a variety of reasons why the most naively obvious alternative to the list of Standard Model particles being fundamental, which would be that they are composite particles of a smaller number of more fundamental particles (generically called preons), is unlikely to be an explanation. The reasons are a bit technical (e.g. the binding energy necessary for this to be the case is too great to be consistent with the hypothesis), but the consensus is that this is pretty much a dead end.
String theory, at first, seemed to be another path to a more fundamental explanation of the laws of nature than the Standard Model plus general relativity (which aren't themselves compatible with each other which implies that one or the other or both aren't truly fundamental) but this research agenda in the many decades since it was first introduced has turned out to be a quagmire that even many of its leading theorists don't claim is an accurate description of the world we actually live in, even though the things we learned along the way investigating it may have value.
So, while our core theories, the Standard Model and general relativity, probably aren't the most fundamental and complete explanations of the laws of nature that are possible, we really have no solid leads on alternatives.
Perhaps the most promising area where we might gain a better understanding is from astronomy where ill-understood concepts summarized as dark matter and dark energy might be better understood with more observational astronomy evidence. But we are far from reaching a consensus understanding of them, and are less confident about the baseline explanation of them called Lambda CDM (for cold dark matter) theory now than we have been since Lambda CDM was formulated.
The prospects for something more fundamental didn't always seem so bleak. In the late 1980s and early 1990s, a group of theories called "supersymmetry" and their gravitational extension, "supergravity" looked very promising. It would have explained many of the fundamental constants in an overall framework (although not all of them), it would have provided a path to unifying the Standard Model coupling constants at higher energies into a single force, it would have provided a candidate for dark matter particles that was sufficient to fit the astronomy and particle physics data available at the time, there was no evidence that the cosmological constant wasn't constant, it would have provided a bridge between the Standard Model and string theory, and a path to a workable theory of quantum gravity.
But bit by bit, that fell apart. Supersymmetry predicted particles with well defined properties, but when particle collider scientists looked for those predicted particles they weren't were they were assumed to be. Better data showed that the Standard Model coupling constants weren't on a path to converging at high energies. Better astronomy observations found that dark matter phenomena behaved differently than it would have if the supersymmetry candidates were what explained it. Better astronomy observations found that the cosmological constant might not be constant after all. The discovery that neutrinos had mass complicated the situation and the failure to detect neutrinoless beta decay complicated one of the most elegant explanations of neutrino mass that might fit better into the supersymmetry/string theory framework. The math necessary for string theory and quantum gravity turned out to be much more problematic than early researchers in these fields had hoped or expected at a very fundamental level. Strong experimentally confirmed anomalies that would deviate from the Standard Model to prove that supersymmetry was correct, instead failed to materialize. The experimental anomalies that seemed to potentially support supersymmetry went away as theoretical predictions and experimental measurements grew more precise.
Supersymmetry isn't definitively ruled out yet, but any version of it that is discovered won't solve the problems that a large share of the physics community hoped and believed that it would in the late 1980s and early 1990s, before they had enough experimental data and mathematical understanding to realize that their beautiful solution to all of the unsolved problems in fundamental physics with a deeper theory wasn't going to hold together.
A generation or two later, we have bits and pieces of ideas - MOND, entropic gravity, axion-like ultralight dark matter particles, Koide's rule and extrapolations of it, multiple Higgs doublet theories, see saw theories of neutrino mass, and more. There are many more partial theories to explain one little thing that surfaced as an anomaly or possibility for a while that were later ruled out. But we have no real solid agenda for developing a deeper theory.
Who knows?
Maybe we're just a genius or three away from ending a period of non-consensus and confusion about a way forward and finding the deeper theories we need to explain the outstanding unsolved problems of fundamental physics. But I'm not that genius and there aren't a lot of good candidates among living physicists today for people who could be that genius.