Deep Learning the new key to nonlinear PDEs?

In summary, researchers at Caltech have developed a new deep-learning technique for solving PDEs that is more accurate and generalizable than previous methods. It is also 1,000 times faster than traditional mathematical formulas, making it a promising tool for solving complex problems such as the Navier-Stokes equation without the need for retraining or supercomputers. However, there are some concerns about the reported results and lack of validation procedures.
  • #1
BWV
1,477
1,798
This paper getting some press, with promises that NNs can crack Navier-Stokes solutions more efficiently than traditional numerical methods.

Now researchers at Caltech have introduced a new deep-learning technique for solving PDEs that is dramatically more accurate than deep-learning methods developed previously. It’s also much more generalizable, capable of solving entire families of PDEs—such as the Navier-Stokes equation for any type of fluid—without needing retraining. Finally, it is 1,000 times faster than traditional mathematical formulas, which would ease our reliance on supercomputers and increase our computational capacity to model even bigger problems
... In the gif below, you can see an impressive demonstration. The first column shows two snapshots of a fluid’s motion; the second shows how the fluid continued to move in real life; and the third shows how the neural network predicted the fluid would move. It basically looks identical to the second.

ns_sr_v1e-4_labelled.gif


https://www.technologyreview.com/20...er-stokes-and-partial-differential-equations/

https://arxiv.org/abs/2010.08895
 
  • Like
  • Informative
  • Wow
Likes aheight, Delta2, fog37 and 1 other person
Physics news on Phys.org
  • #2
There are some caveats and problems with the evaluation in the paper. I have to say I am pretty skeptical.

1) Performance testing compared with traditional solvers is not described in enough detail to understand the significance of the reported results. In fact there is only one sentence reporting a comparison, and it gives only a single data point.

In a sharp contrast, FNO takes 0.005s to evaluate a single instances while the traditional solver, after being optimized to use the largest possible internal time-step which does not lead to blow-up, takes 2.2s.

It is reported that both methods use the GPU, but it isn't reported which variation of a traditional method is used (there are a lot of strategies and variations), and there is no indication, or argument that the traditional method they chose is state of the art. Furthermore, they gave no details about how the computation was performed on the GPU, how the program was compiled, etc. They could easily have compiled their method with optimizations and the other without, or ran the them on different machines, or with carefully selected work sizes, or whatever to get the results they wanted, and they would not even be lying in the paper. They just omit all of these details.

2) The neural network must be trained before it can be applied. They report that training takes about 12 hours, which is still faster overall.

This amounts to 2.5 minutes for the MCMC using FNO and over 18 hours for the traditional solver. Even if we account for data generationand training time (offline steps) which take 12 hours, using FNO is still faster! Once trained, FNO canbe used to quickly perform multiple MCMC runs for different initial conditions and observations, while thetraditional solver will take 18 hours for every instance.

Supposing that this is true, and a good representation, the difference between 18 hours and 12 hours could be easily due to how well the compared methods were implemented, and how the GPU kernels are parameterized. They give no details, and no code, so we have to assume they chose results that look favorable.

3) They perform or report no serious validation procedure that will give them an estimation of the generalization error, and therefor no evidence to support the suggestion that the trained network can then be used for multiple runs with different initial conditions and observations. At least we do not know what the error will be. In order to know this, they would have had to perform cross validation or some other resampling method to rigorously test the model on conditions that it didn't see in training (note that running the traditional solver with those conditions is required first in order for the model to see those conditions in training). And we are left without a good idea if the training was sufficient enough that the model would generalize, and to what degree it was just memorizing partial results that are only valid for their training data.

Anyways, an honest assessment is that they were able to find a case where an implementation of their method (including training time) was about 1.4 times faster than an unnamed traditional method+implementation. And the authors speculate that their model could generalize, which would allow applications to new simulation runs without complete re-training. And there is no way to tell if any of it is true. But it's still interesting.
 
Last edited:
  • Like
  • Informative
Likes pbuk, Ssnow, Hugo S and 3 others
  • #3
"It’s also much more generalizable, capable of solving entire families of PDEs—such as the Navier-Stokes equation for any type of fluid—without needing retraining. Finally, it is 1,000 times faster than traditional mathematical formulas, which would ease our reliance on supercomputers and increase our computational capacity to model even bigger problems"

as for me that is enough to stop reading
 
  • Haha
Likes S.G. Janssens

Related to Deep Learning the new key to nonlinear PDEs?

1. What is Deep Learning?

Deep Learning is a subset of machine learning that uses artificial neural networks to learn and make predictions from large datasets. It is inspired by the structure and function of the human brain and is able to handle complex and nonlinear relationships between variables.

2. How is Deep Learning used in solving nonlinear PDEs?

Deep Learning is used in solving nonlinear PDEs by approximating the solution using a neural network. The network is trained on a dataset of known solutions to the PDE and then used to make predictions for new inputs. This approach can handle complex and highly nonlinear PDEs that are difficult to solve using traditional methods.

3. What are the advantages of using Deep Learning for solving nonlinear PDEs?

One of the main advantages of using Deep Learning for solving nonlinear PDEs is its ability to handle complex and highly nonlinear relationships between variables. It also does not require explicit knowledge of the underlying physics, making it a more general approach. Additionally, Deep Learning can potentially reduce the computational cost and time required for solving PDEs.

4. What are the limitations of Deep Learning in solving nonlinear PDEs?

One of the limitations of Deep Learning in solving nonlinear PDEs is the need for large and diverse datasets for training. Without a sufficient amount of data, the neural network may not be able to accurately approximate the solution. Additionally, the interpretability of the results may be limited, making it challenging to understand the underlying physics of the problem.

5. Can Deep Learning completely replace traditional methods for solving nonlinear PDEs?

No, Deep Learning cannot completely replace traditional methods for solving nonlinear PDEs. While it may be able to handle certain types of PDEs more efficiently, traditional methods are still necessary for validating the results and understanding the underlying physics. Deep Learning should be seen as a complementary tool in the field of PDEs rather than a replacement for traditional methods.

Similar threads

  • Differential Equations
Replies
4
Views
1K
  • Differential Equations
Replies
1
Views
1K
  • STEM Academic Advising
Replies
4
Views
2K
Replies
6
Views
5K
  • Earth Sciences
Replies
10
Views
8K
  • MATLAB, Maple, Mathematica, LaTeX
Replies
5
Views
2K
  • MATLAB, Maple, Mathematica, LaTeX
Replies
5
Views
2K
Back
Top