Thoughts on neural networks "discovering" physical concepts

[sphyrch]

I came across an interesting paper from which I'll quote parts of the intro:

[...] the physical theories we know may not necessarily be the simplest ones to explain the experimental data, but rather the ones that most naturally followed from a previous theory at the time. The formalism of quantum theory, for instance, is built upon classical mechanics; it has been impressively successful, but leads to conceptually challenging consequences [...]

This raises an interesting question: are the laws of quantum physics, and other physical theories more generally, the most natural ones to explain data from experiments if we assume no prior knowledge of physics? [...] we investigate whether neural networks can be used to discover physical concepts in classical and quantum mechanics from experimental data, without imposing prior assumptions and restrictions on the space of possible concepts.

and the conclusion:

[...] we have shown that neural networks can be used to recover physical variables from experimental data. To do so, we have introduced a new network structure, SciNet, and employed techniques from unsupervised representation learning to encourage the network to and a minimal uncorrelated representation of experimental data. [...] Furthermore, the analogy between the process of reasoning of a physicist and representation learning provides insight about ways to formalize physical reasoning without adding prior knowledge about the system.

obviously the authors are more competent in Physics than I am - so are most PF member. what are your opinions on this? can the work of these authors "discover" physical laws in any way or is the paper title misleading?

would stuff like special relativity or quantum mechanics be discovered purely from experimental data without using prior knowledge? even if physical laws were encoded by these networks as "black box" models, would they be a good replacement of closed-form expressions or differential equation solutions?

 

[anuttarasammyak]

Thanks for interesting information.

I am not sure the networks find really new physical laws but expect they propose simpler interpretation or presentation of physical law. For examples I would expect these AI networks handle experimental results including those of industrial application without prejudice in history and propose
- The most simple, friendly and widely usable unit system including electromagnetic unit system,
- The most simple operational meaning of quantum mechanics with no prejudice of the historical interpretations.

 

[A.T.]

... we have introduced a new network structure, SciNet, ...

I laughed, but I'm afraid the general public will not enjoy this naming.
https://en.wikipedia.org/wiki/Skynet_(Terminator)

 

[anorlunda]

In science, sometimes theory comes before observational data, and sometimes the data comes first and the theory second. Obvious examples were Einstein's theories of special relativity and general relativity.

In special relativity, the speed of light and Maxwell's equations were know before, and Einstein crafted a theory to explain.

In general relativity, Einstein made the theory first. Many consequences of general relativity were unexpected but were shown to be true by later experiments.

You could certainly use statistics and neural nets to sift data looking for anomalous events. An anomalous event might provide evidence to support an existing theory, or it might indicate that new theory is needed. But to create the new theory, you would need something very different than the data sifter, and there would be very little data to use for training that second stage device.

 

[sphyrch]

In science, sometimes theory comes before observational data, and sometimes the data comes first and the theory second. Obvious examples were Einstein's theories of special relativity and general relativity.

In special relativity, the speed of light and Maxwell's equations were know before, and Einstein crafted a theory to explain.

In general relativity, Einstein made the theory first. Many consequences of general relativity were unexpected but were shown to be true by later experiments.

You could certainly use statistics and neural nets to sift data looking for anomalous events. An anomalous event might provide evidence to support an existing theory, or it might indicate that new theory is needed. But to create the new theory, you would need something very different than the data sifter, and there would be very little data to use for training that second stage device.

in case of special relativity where the data was known first, assuming we had such a neural network back then, it would still not be possible for it to figure out lorentz transformations on its own, right?

even in case of plenty of data available from QM experiments, I'm not sure how a neural network would figure out the mathematical formalism required to make sense of observations (e.g. square of wavefunction as probability, where wavefunction is hilbert space element, etc.)

that's why I'm confused on why the paper makes the assertion of "discovering" physical laws. last time i checked, there's more to discovering a physical law than number crunching from experiments

 

[anorlunda]

it would still not be possible for it to figure out lorentz transformations on its own, right?

Not impossible perhaps, but not likely to happen in real life.

All forms of "curve fitting" involve guessing a mathematical function to "fit" a set of data. Nothing magic there.

But neural nets are "trained" with huge collections of examples, data+solution. The net eventually learns how to infer the solution from the data. So with 10 million sets of data and the functions that have been shown to fit them, we could train a neural net to do curve fitting.

It might be possible to do the same with cases like special relativity, but first you need millions of theories like relativity or QM, and the matching data sets that inspired those theories. That is the training data needed to develop the neural net. In the remaining lifetime of Planet Earth, you are not likely to collect such a set of training data. So "possible" yes, "practical" no.

I should point out that I do not believe that neural nets and deep learning are the ultimate paradigm of AI for all time. Eventually, other approaches may emerge.

 

[Shirish]

obviously the authors are more competent in Physics than I am - so are most PF member. what are your opinions on this? can the work of these authors "discover" physical laws in any way or is the paper title misleading?

I went through the paper. They've built a network that finds an efficient representation of a physical problem.

For example, say you have a system of just one particle that travels along $x$ axis with constant velocity. Experimental data looks like $\{t_i,x(t_i)\}$. How do you "describe" this system such that you can answer specific questions?

If the questions are of the form "What is $x(t)$?" for some $t$, then a way to "describe" this physical system is to find a minimal representation using which such questions can be answered - in this case, $x(t_0),v$ would be an example of such a representation.

Their claim is that the representation can be manually analyzed by varying the input to see how the representation changes, or to vary the representation to see how the output changes. It can then be tallied against what we know and it may turn out to be something novel (something humans didn't think of). That can give insights - and I don't dispute that.

I don't see how that's a way to "discover physical concepts". It doesn't tell you the explicit expression through which outputs are linked to representations, nor does it give any mathematical formalism, as you pointed out.

I wonder if people use clickbait titles in their papers to increase the odds of publication or media attention.