Prob/Stats Any good math-theory-focused books on neural networks and data science?

[s00mb]

Hi. I'm looking for books on data science, preferably leaning towards neural networks, that focus on mathematical rigor. For example, theorems on optimization, minimum number of layers to accomplish a task efficiently, etc. Most books I've seen seem to hand wave this stuff. Anyone know any juicy books on the topic?

 

[jedishrfu]

There's a rather recent book by Kochenderfer called Algorithms for Optimization with many examples written in Julia, a hot programming language from MIT that folks are using for numerical work in diverse fields including ML and Data Science.

https://www.amazon.com/dp/0262039427/?tag=pfamazon01-20

There's also the 100 page ML book by Burkov:

https://www.amazon.com/dp/199957950X/?tag=pfamazon01-20

which is available online as a try and buy book.

Lastly, Geron's book Hands-on ML with Scikit-Learn, Keras and Tensorflow:

https://www.amazon.com/dp/1492032646/?tag=pfamazon01-20

All are good books that discuss the math behind the ML although not at the rigor you're looking for.

 

[BvU]

I looked through "Data mining" by Witten, Frank, Hall and, Pal which covers most of these, but it isn't rigorous like the good old math analysis books. No wonder for such a rapidly developing (practical) area.

 

[atyy]

I think these are still mainly in papers.

Function approximation:
https://arxiv.org/abs/2006.13915
https://arxiv.org/abs/2010.08895

Effectiveness of gradient descent:
https://arxiv.org/abs/1406.2572
https://arxiv.org/abs/1902.04811
https://arxiv.org/abs/1904.04480

Error estimation
https://arxiv.org/abs/1506.02142