Finding the Perfect Machine Learning Textbook for a Physics BSc

[vancouver_water]

Hello, recently I've become interested in learning more about machine learning, and am looking for a good textbook to study from. My background: Finished most of the required courses for my physics BSc and the related math courses. Some relevant courses I have taken:

Linear Algebra
Calculus I-IV
Complex Analysis
Differential Equations (ODEs, 2 courses in PDEs)
Greens Functions
Calculus of Variations
Probability

I haven't taken a Stats course but our probability course covered some basic stats. I also have a lot of experience in numerical computing and numerical linear algebra.

Any recommendations? Thank you!

 

[The Bill]

I've seen the free online book http://neuralnetworksanddeeplearning.com recommended.

 

[Daverz]

I believe the standard text is

Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning

https://www.amazon.com/dp/B00475AS2E/?tag=pfamazon01-20

You can find a (legitimately) free PDF online.

https://web.stanford.edu/~hastie/Papers/ESLII.pdf

 

[StoneTemplePython]

My favorite on the subject is

Learning From Data

The book is only $30 in the US, though in other locations it costs more due to bizarre licensing issues (e.g. in Canada I think it is $100).

The associated course run by one of the authors at Caltech is going on right now on edx. (I took it in a prior session.) You are a bit late but may want to jump in -- the prof personally answers like $\frac{3}{4}$ of the questions in the forum himself which is remarkable. The book is complementary to the course and the course is programming language agnostic.

I would not start with a book on neural nets and deep learning. That is way too specialized to start, and there is already an issue in the industry where a sizable number of people think machine learning is only deep learning. The situation is reminiscent of the blind men touching an elephant, and one of them only touches the tail and thinks and elephant is like a snake.

Hastie and Tibshirani is fine and a lot of people like their book. They seem to be a bit too obsessed with classical stats and R in my view, but they are very highly regarded and cover a lot of ground.

 

[vancouver_water]

Thanks for the recommendations guys, I'll look into these books. Would it also be recommended to work through a stats book at the same time, or do these all cover the necessarystats?

 

[Daverz]

For stats background, I've been reading Practical Statistics for Data Scientists:

https://www.amazon.com/dp/1491952962/?tag=pfamazon01-20

 

[atyy]

https://www.cs.ubc.ca/~murphyk/MLbook/
Machine Learning: a Probabilistic Perspective
by Kevin Patrick Murphy

http://www.inference.org.uk/mackay/itila/
Information Theory, Inference, and Learning Algorithms
by David J. C. Mackay
http://www.inference.org.uk/mackay/itprnn/
Information Theory, Pattern Recognition and Neural Networks
University of Cambridge, Part III Physics Course, Minor option
by David J. C. Mackay

 

[dkotschessaa]

I believe the standard text is

Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning

https://www.amazon.com/dp/B00475AS2E/?tag=pfamazon01-20

You can find a (legitimately) free PDF online.

https://web.stanford.edu/~hastie/Papers/ESLII.pdf

Yes, I've hear that the "ESL" is the kind of machine learning bible.

I couldn't *remember* it though, so thank you for posting and providing the link.

 

[vancouver_water]

https://www.cs.ubc.ca/~murphyk/MLbook/
Machine Learning: a Probabilistic Perspective
by Kevin Patrick Murphy

http://www.inference.org.uk/mackay/itila/
Information Theory, Inference, and Learning Algorithms
by David J. C. Mackay
http://www.inference.org.uk/mackay/itprnn/
Information Theory, Pattern Recognition and Neural Networks
University of Cambridge, Part III Physics Course, Minor option
by David J. C. Mackay

Interesting, Kevin Murphy used to be a prof at my school (UBC) but now works for Google.

 

[onoturtle]

Without any stats background I found the earlier edition of ESL to be a tough read. Worth a shot since you can get the PDF legit for free on their website nevertheless.

If that doesn't work out, you can check out Introduction to Statistical Learning, which shares an author or two with ESL and assumes less stats knowledge. http://www-bcf.usc.edu/~gareth/ISL/ Again, they got a legit PDF free on their site.

The above are more of the statistician's perspective on the subject, which I think is valuable. I read bits and pieces from both and I thought it well written. From a more engineering/cs viewpoint, I like Bishop's Machine Learning and Pattern Recognition book. Unfortunately there is no legit free PDF on that one. Check out the free samples out there and see what you think. It is outdated, but if you supplement with say Goodfellow et al.'s Deep Learning (free electronic version http://www.deeplearningbook.org/), you should have a good foundation.

I personally don't like Murphy's ML book. Seemed encyclopedic and way too brief at some areas, just throwing equations without much motivation, explanation, etc. I think the introductory material on probability and stats is more expanded and probably better than Bishop's though.

Another suggestion, if you're into theory, then Understanding Machine Learning is great http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/ with free PDF.

If you want to learn some probably & stats separately, I don't really have good suggestions.

- Wasserman, All of Statistics. I used this in a course. I liked it in the context of a course with a lecture, but it is too concise to learn from alone, I think.

- Casella & Berger, Statistical Inference. I bought a cheap Chinese copy of this to supplement the above. Probably overkill, as it covers a lot of analytical methods to evaluate hypothesis tests and confidence intervals. I guess you could skim that like I did :) Lacking on material on Bayesian stats though, which is useful in some ML methods.

EDIT: I am reminded of the reading list Michael Jordan wrote. This is for a deep dive into the subject as prep for a PhD under him http://www.statsblogs.com/2014/12/30/machine-learning-books-suggested-by-michael-i-jordan-from-berkeley/