Neural networks - uses in Physics and Astrophysics?

[Vrbic]

Hello,
I'm not very familiar with neural networks, but it seems very interesting. I would like to ask how may I use it in physics, especially in astrophysics? I guess for some data mining? But could anyone show some concrete example of what it can do with data? Or if there is anything else how to use it.
Thank you.

 

[anorlunda]

That's an interesting question, but very broad. What research have you done before posting this?

https://en.wikipedia.org/wiki/Neural_network
https://en.wikipedia.org/wiki/Deep_learning

 

[jedishrfu]

Here's one example in multi-messenger astrophysics:

https://journals.aps.org/prd/abstract/10.1103/PhysRevD.97.044039

Here's another about gravitational lensing:

https://www.businessinsider.com/astrophysicists-neural-net-ai-analyse-gravitational-lensing-2017-8

Basically neural nets are great at analyzing images and signals looking for interesting patterns that its been trained to detect.

I would suggest that you do some google searches for many more article and video examples.

 

[Vrbic]

Thank you both of you,
I know a bit about using NN for detecting a pattern in the signal, but I wondered if there is some totally different use in physics. I notice there is some predicting potential of such technic, but it is probably used mostly in the financial sector, I read.

 

[anorlunda]

Your question is very open-ended. So here's a tongue in cheek open ended answer. Use NN to learn why some grant proposals are accepted and others rejected. Even astronomers need money.

 

[newjerseyrunner]

Neural networks are really good at thinking outside of the box because they are free to explore thousands of variations of how to do things. Humans are not very good at fine tuning things because it requires iteration. Humans do an experiment, analyze the results, run it again, check the results, and keep iterating. A NN can do that thousands of times a second.

Probably the most famous neural network achievement was using lasers to create a Bose-Einstein condensate. They used lasers to cool it down by both moving the lasers in 3D space and changing their outputs. Humans managed to do this and it won a nobel prize. The neural network was able to recreate the experiment from scratch in about an hour and did so in a completely different way than the nobel prize winning scientists. https://www.forbes.com/sites/bridai...-nobel-prize-physics-experiment/#4ef7cda76678

So NNs are really good at iterating quickly and making rapid and complex reactions. Considering that mirrors of modern telescopes reshape themselves in reaction to the atmosphere, I would't be surprised if a neural network was doing it.

 

[Borg]

I have just started getting involved in this fascinating field. The first big step that I had to figure out was the coding environment. When I first started Googling about neural networks and AI processing, I found a lot of confusing information. Just figuring out where to start left me with dozens of links on how to get started but, it seemed that none of them agreed on what was needed. It looked like I would be stuck wading through hundreds of useless videos and websites. Fortunately, I recently discovered some awesome opportunities at work to be mentored by and study with actual data scientists. I am very early in my studies but these are some of the things that I've learned so far.

What programming languages do you know? You're going to need to know Python at a minimum.

If you're going to be programming with neural networks, https://conda.io/docs/user-guide/install/index.html is a good starting point for your development environment. I just downloaded it today and have started working through the tutorials. Most of my training so far has used the Jupyter package in Anaconda.

Farther out, you may want to work with something called TensorFlow. I originally thought that neural network processing was all about how powerful your CPU was but, as you get deeper into it, that isn't always the case. Since neural network processing is all about vector manipulation, the GPU (graphics card) on your computer increases your TensorFlow processing because graphics cards are designed to handle vectors natively (see the NVIDIA Cuda GPUs page for more information). Note that while you may eventually end up using your GPU, it is recommended for beginners to initially install the GPU-only version of TensorFlow. Adding the GPU capabilities isn't needed at the beginning and just adds to your startup complexity.

I haven't looked up many tutorials yet on learning how to program neural networks but at least I know at this point what my environment should be. When I need to suppliment my training at work with online tutorials, it will be a lot easier to separate out the garbage.

 

[anorlunda]

It depends on your interest. If your interest is AI software, then programming languages are of interest. If your interest is in application of NN, then something like https://www.ibm.com/cloud/deep-learning?S_PKG=AW&cm_mmc=Search_Bing-_-Watson+Core_Watson+Core+-+Discovery-_-WW_NA-_-+ibm++deep++learning_Broad_&cm_mmca1=000019OO&cm_mmca2=10008718&cm_mmca7=52003&cm_mmca8=kwd-81913675448838:loc-190&cm_mmca9=cdb80ce8-8789-47ff-b319-0327995afee7&cm_mmca10=81913645621082&cm_mmca11=p&mkwid=cdb80ce8-8789-47ff-b319-0327995afee7|1081|126529&cvosrc=ppc.bing.%2Bibm%20%2Bdeep%20%2Blearning&cvo_campaign=000019OO&cvo_crid=81913645621082&Matchtype=p&msclkid=efd1b52ce43114eb493deea5b0f4d6ed (or the equivalent from other sources) is more productive. It seems to be the trend that specialized hardware implements the NN, not general-purpose computers and languages. The same can be said of deep learning. You can focus on doing deep learning, or on using deep learning. It's your choice.

 

[FactChecker]

What programming languages do you know? You're going to need to know Python at a minimum.

This recommendation is too narrow. I am sure that many languages have NN utility packages. I have seen a some NN applications done using a MATLAB package which would than automatically generate C code.

 

[Borg]

It depends on your interest. If your interest is AI software, then programming languages are of interest. If your interest is in application of NN, then something like https://www.ibm.com/cloud/deep-learning?S_PKG=AW&cm_mmc=Search_Bing-_-Watson+Core_Watson+Core+-+Discovery-_-WW_NA-_-+ibm++deep++learning_Broad_&cm_mmca1=000019OO&cm_mmca2=10008718&cm_mmca7=52003&cm_mmca8=kwd-81913675448838:loc-190&cm_mmca9=cdb80ce8-8789-47ff-b319-0327995afee7&cm_mmca10=81913645621082&cm_mmca11=p&mkwid=cdb80ce8-8789-47ff-b319-0327995afee7|1081|126529&cvosrc=ppc.bing.%2Bibm%20%2Bdeep%20%2Blearning&cvo_campaign=000019OO&cvo_crid=81913645621082&Matchtype=p&msclkid=efd1b52ce43114eb493deea5b0f4d6ed (or the equivalent from other sources) is more productive. It seems to be the trend that specialized hardware implements the NN, not general-purpose computers and languages. The same can be said of deep learning. You can focus on doing deep learning, or on using deep learning. It's your choice.

This recommendation is too narrow. I am sure that many languages have NN utility packages. I have seen a some NN applications done using a MATLAB package which would than automatically generate C code.

This is why it's so difficult for someone who's getting into it for the first time. There are so many options available. How is a person to know which is the best path forward for them when they're getting into this from ground zero? Or, even know which paths are available? Myself, I would probably rather just use pre-built packages eventually but, what good does that do if I don't have a clue for how they work? And, from the little that I've glimpsed, there are hundreds of prebuilt packages available dealing with everything from language processing to jedishrfu's gravitational lensing link. How are you supposed to know which one is better than another for your needs? What level of training is best for your software - and what does that even mean? For people who have been working with NN for a while, the choices may be obvious but, for someone who's brand new, it's a confusing nightmare.

 

[anorlunda]

Myself, I would probably rather just use pre-built packages eventually but, what good does that do if I don't have a clue for how they work?

I presume that you did not write your own web browser, or your own Internet, or your own computer, or generate your own electric power to access Physics Forums, nor do you need to understand how those things are done. The world of science and engineering is much to broad to not specialize on something. Which part interests you most? I'm confident that you can make up your own mind without help if you take the time to give is some thought.

 

[FactChecker]

This is why it's so difficult for someone who's getting into it for the first time. There are so many options available. How is a person to know which is the best path forward for them when they're getting into this from ground zero?

That's a good point. With that in mind, I guess my advice to a beginner would depend on what type of guidance they have available. If there is an established expert, follow his lead and use the language / utility package that he recommends. Otherwise, Python is a good language to learn these days and might be the best one to find NN help in.

PS. The NN work that I have witnessed was done by a person whose PhD research was in that area.

 

[Borg]

I presume that you did not write your own web browser, or your own Internet, or your own computer, or generate your own electric power to access Physics Forums, nor do you need to understand how those things are done. The world of science and engineering is much to broad to not specialize on something. Which part interests you most? I'm confident that you can make up your own mind without help if you take the time to give is some thought.

I actually did write the code for a web browser when I was first learning to code. It displayed two separate pages on a split screen.

With respect your suggestion to make up my mind, I have things at work that will tend toward language processing but that isn't necessarily what I want to work with on my own time. So, I do want to keep my options open while learning as much as I can on the topic. I also checked out your IBM toolkit link and, halfway down the page, they have a tutorial using the same products that I originally mentioned (Jupyter and TensorFlow). I haven't tried to run the tutorial yet but I remember checking out the Watson stuff several years ago. I gave up on it because of the learning curve and the fact that I had no idea whether it would get me somewhere useful. I will keep the toolkit site in mind during my studies. Thanks.

Anyway, the main point that I was trying to make is that there is a lot of wildly different information out there on NNs. Even if you know what you want to apply it to, it's very difficult to find a starting point just by searching on the internet. To use part of your analogy, for me, it has been like trying to get electricity installed in my home and the companies that I found offer power as we know it, solar panels, emergency generators and, kits for building your own hydroelectric dam - with each of them claiming that their solution is the best. And, while I'm trying to decide, a door-to-door salesman shows up trying to sell me $5 pictures of power lines. While the simpliest choice is obvious for this example, that hasn't been my experience with searches on the internet on how to get started with NNs.

As I've stated, I'm starting from scratch but I've thrown myself in the deep end. My current goals are to get a better depth of understanding on NNs and Deep Learning, learn what others are doing in the field and to get myself up to speed on the working environments that I will need. To that end, I've reached out to colleagues to get guidance and have directed my efforts toward their recommendations. I am attending weekly sessions with others where they are working their way through this book on Deep Learning (unfortunately, I didn't hear about them until they were working on chapter 10). I'm also scheduled to attend local Meetups on the topic and will also be attending a conference at the end of Sept. I expect to get a wide range of information during this stage.

 

[newjerseyrunner]

This recommendation is too narrow. I am sure that many languages have NN utility packages. I have seen a some NN applications done using a MATLAB package which would than automatically generate C code.

Agreed. I’ve written multiple neural networks in C++. I don’t think language is important as just generic programming paradigms: specifically OOP and threading.

 

[Fred Wright]

I suggest that if you are interested in the mathematical underpinnings of NN, then read "Neural Networks for Pattern Recognition" by Christopher Bishop. I down loaded it off the web (free) about a year ago. It was published in 1995 but is very enlightening for the beginner (like me).

 

[Borg]

Do you have a link?

 

[jedishrfu]

Here's a reference to the 10 free books on ML:

https://www.analyticsvidhya.com/blog/2018/02/10-free-must-read-machine-learning-e-books/

and here's Bishop's book on Amazon circa 2011:

https://www.amazon.com/dp/0387310738/?tag=pfamazon01-20

 

[WWGD]

I have just started getting involved in this fascinating field. The first big step that I had to figure out was the coding environment. When I first started Googling about neural networks and AI processing, I found a lot of confusing information. Just figuring out where to start left me with dozens of links on how to get started but, it seemed that none of them agreed on what was needed. It looked like I would be stuck wading through hundreds of useless videos and websites. Fortunately, I recently discovered some awesome opportunities at work to be mentored by and study with actual data scientists. I am very early in my studies but these are some of the things that I've learned so far.

What programming languages do you know? You're going to need to know Python at a minimum.

If you're going to be programming with neural networks, https://conda.io/docs/user-guide/install/index.html is a good starting point for your development environment. I just downloaded it today and have started working through the tutorials. Most of my training so far has used the Jupyter package in Anaconda.

Farther out, you may want to work with something called TensorFlow. I originally thought that neural network processing was all about how powerful your CPU was but, as you get deeper into it, that isn't always the case. Since neural network processing is all about vector manipulation, the GPU (graphics card) on your computer increases your TensorFlow processing because graphics cards are designed to handle vectors natively (see the NVIDIA Cuda GPUs page for more information). Note that while you may eventually end up using your GPU, it is recommended for beginners to initially install the GPU-only version of TensorFlow. Adding the GPU capabilities isn't needed at the beginning and just adds to your startup complexity.

I haven't looked up many tutorials yet on learning how to program neural networks but at least I know at this point what my environment should be. When I need to suppliment my training at work with online tutorials, it will be a lot easier to separate out the garbage.

Do you use Graphlab? I have had trouble installing it.

 

[Borg]

Do you use Graphlab? I have had trouble installing it.

No, I haven't seen that until now. Is this the one that is also referred to as Turi?

 

[gleem]

Do you have a link?

http://www.cs.ukzn.ac.za/~sviriri/Books/Machine-Learning-Pattern-Recognition/book5.pdf

 

[jedishrfu]

Here's a recent article in Nature that might have long lasting effects in ML:

https://www.nature.com/articles/d41586-019-00012-4

During the twentieth century, discoveries in mathematical logic revolutionized our understanding of the very foundations of mathematics. In 1931, the logician Kurt Gödel showed that, in any system of axioms that is expressive enough to model arithmetic, some true statements will be unprovable1. And in the following decades, it was demonstrated that the continuum hypothesis — which states that no set of distinct objects has a size larger than that of the integers but smaller than that of the real numbers — can be neither proved nor refuted using the standard axioms of mathematics2–4. Writing in Nature Machine Intelligence, Ben-David et al.5 show that the field of machine learning, although seemingly distant from mathematical logic, shares this limitation. They identify a machine-learning problem whose fate depends on the continuum hypothesis, leaving its resolution forever beyond reach.

With respect to language choices, many books presume that Python is winning the race in ML and so it makes sense to be familiar with it and perhaps consider it for projects. The Jupyter Notebooks + Anaconda is a good combo for Python development as is the more traditional PyCharm + Anaconda environment. In the ML for Dummies book circa 2015-2016 they suggest R and Python as two languages to know. However more recently, it seems Python has won this race for the hearts and minds of data scientists.

For my own projects, I am looking to Julia (fast execution + MATLAB like syntax) as the language of choice with its Flux.jl library. Unfortunately there are few to no books yet so I have to rely on the Python books and convert as needed.

 

[fluidistic]

For my own projects, I am looking to Julia (fast execution + MATLAB like syntax) as the language of choice with its Flux.jl library. Unfortunately there are few to no books yet so I have to rely on the Python books and convert as needed.

I think there's a nice julia community on IRC, mailing lists and stack overflow.

 

[jedishrfu]

Thanks for the reference @fluidistic I hadn't considered IRC. I know some folks post on the julialang.org forums and I usually look there when I have questions on things.