Maths of Neural Networks -- books and papers?

In summary, there are a lot of books on the mathematics of neural networks, but some recommended ones are "Hands-on Machine Learning with Scikit-Learn, Keras and Tensorflow" and "Algorithms for Optimization". These books cover both theory and practice, with the latter also including code written in Julia. Additionally, the user is looking for research papers on optimizing and minimizing the placement and number of neurons in a network, particularly in relation to calculating weights, as well as papers on geometry and differential equations.
  • #1
s00mb
33
10
Does anyone know any good up to date books on the mathematics of neural networks? If you know any good research papers that would be cool too. I've read through "Intro to the theory of neural computation" by John Hertz but it's a 1991 book. I'm looking for generalized theorems on optimizing/minimizing the placement and number of neurons in the network assuming some given way of calculating the weights. Also, if you know any cool research papers I'd like to know them. I've already seen the one on changing one pixel input to confuse a NN though. Geometry and Differential Equations (especially fractional applications) a plus. I've been browsing the internet for it and figured I'd try to pick your brains for it while I am at it. Thanks in advance.
 
Physics news on Phys.org
  • #2
There's a lot of books out there. The one I like is:

Hands-on Machine Learning with Scikit-Learn, Keras and Tensorflow

It likely has some interesting math in it although most books skip past the details and go for the practice more. People want to use the algorithms to do cool stuff and leave the math to the developers of these packages.

Another book of interest would be:

Algorithms for Optimization

which covers many of the algorithms used in machine learning. All the code is written in Julia.
 
  • #3
Yeah, I've read through most of the "practical" books, I was looking for rigorous books with proofs. Ohhh bonus points for using Julia I like that I'll have to check it out just for that :)
 
Last edited:

1. What is the significance of studying the maths of neural networks?

The maths of neural networks is essential for understanding the inner workings of these complex systems. It allows scientists to develop and improve upon neural network models, and also provides insight into how the brain processes information.

2. What are some recommended books for learning about the maths of neural networks?

Some popular books on the maths of neural networks include "Neural Networks: A Systematic Introduction" by Raul Rojas, "Deep Learning" by Yoshua Bengio, Ian Goodfellow, and Aaron Courville, and "The Elements of Statistical Learning" by Trevor Hastie, Robert Tibshirani, and Jerome Friedman.

3. Are there any specific papers that are considered must-reads for understanding the maths of neural networks?

Some widely cited papers on the maths of neural networks include "A Mathematical Theory of Communication" by Claude Shannon, "Gradient-Based Learning Applied to Document Recognition" by Yann LeCun, Yoshua Bengio, and others, and "Backpropagation Through Time: What It Does and How to Do It" by Paul Werbos.

4. How does linear algebra play a role in the maths of neural networks?

Linear algebra is a fundamental tool for understanding the maths of neural networks. It is used to represent and manipulate the weights and biases in a neural network, and also plays a crucial role in the backpropagation algorithm used for training these networks.

5. What are some common challenges in understanding the maths of neural networks?

Some common challenges in understanding the maths of neural networks include the complexity of the models and the use of advanced mathematical concepts such as calculus, linear algebra, and probability theory. It can also be difficult to visualize and interpret the results of these models, making it challenging to understand their inner workings.

Similar threads

  • Science and Math Textbooks
Replies
3
Views
1K
Replies
6
Views
744
  • Programming and Computer Science
Replies
1
Views
919
  • Programming and Computer Science
Replies
1
Views
2K
  • Programming and Computer Science
Replies
5
Views
938
  • Science and Math Textbooks
Replies
14
Views
3K
  • Science and Math Textbooks
Replies
1
Views
2K
  • Sticky
  • Science and Math Textbooks
Replies
10
Views
5K
Replies
15
Views
6K
  • Nuclear Engineering
Replies
8
Views
2K
Back
Top