Need calculus and several books as a prerequisite for Machine Learning

[njama]

Hey!

Happy New Year 2013 to all of you!

I am in good mood for learning something new so I need advice.

I'm currently watching the videos for Machine learning from Stanford University, but I'm stuck at Lagrangian multipliers and duality.

I got solid background in Calculus I and II (I read and learned from the book by Irl Bivens and I really like it) but I've never learned about Lagrange multipliers nor Matrix Calculus, and now it's time to step my Calculus up

Also I got solid basic background in Linear Algebra but never learned about Eigenvalues and Eigenvectors or Semidefinite Matrices.

So I need some book for self-studying:

Here are the topics that are new to me:

Matrix Calculus 20
4.1 The Gradient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.2 The Hessian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.3 Gradients and Hessians of Quadratic and Linear Functions . . . . . . . . . . 23
4.4 Least Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.5 Gradients of the Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.6 Eigenvalues as Optimization

3.10 The Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.11 Quadratic Forms and Positive Semidefinite Matrices . . . . . . . . . . . . . . 17
3.12 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.13 Eigenvalues and Eigenvectors of Symmetric Matrices . . . . . . . . . . . . . 19

Convex Optimization
Convex Optimization Part 2

Here are the materials from the course (there are documents in Section Notes).

Thanks a lot.

Regards.

 

[jasonRF]

I would first look at free books online.

FOr calculus, including Hessians and basic optimization, one nice option is the book by Kenneth Kuttler to be found at:
http://www.e-booksdirectory.com/listing.php?category=4

Other books in this category may fit the bill for Hessians as well, so browse around and see what works for you. This book also covers eigenvectors and such.

Other linear algebra references can be found:
http://www.e-booksdirectory.com/listing.php?category=538
I like the books by Heffron listed there (you can buy a hardcopy via amazon for <$20 US), as well as the book "linear algebra done wrong" by Treil (which is harder than Heffron).

Again, browse around. Good luck!

jason

 

[njama]

Thanks a lot for the help, I have never heard about e-booksdirectory.com it's pretty cool site for free e-books.

I've found courses from Stanford on Youtube about Convex Optimization and some applied linear algebra.

Regards.

 

[dcassell]

This is a good free online resource too for Calc III and linear algebra http://tutorial.math.lamar.edu/.

 

[rezeew]

Hi there,

Happy New Year to you too! It's great to hear that you're in the mood for learning something new. Machine learning is a fascinating and constantly evolving field, and it's wonderful that you're taking the initiative to self-study.

Based on the topics you listed, it seems like you have a solid foundation in calculus and linear algebra. However, to fully understand and apply machine learning algorithms, it is important to have a strong understanding of these concepts as well as matrix calculus, eigenvalues, and convex optimization. These topics are essential for understanding the underlying mathematical principles behind machine learning.

I would recommend looking into textbooks such as "Calculus" by James Stewart and "Linear Algebra and Its Applications" by David C. Lay for a comprehensive review of these topics. Additionally, for matrix calculus specifically, "The Matrix Cookbook" by Kaare Brandt Petersen and Michael Syskind Pedersen is a great resource.

For convex optimization, "Convex Optimization" by Stephen Boyd and Lieven Vandenberghe is a highly regarded textbook in the field. It covers both theory and applications, and will provide you with a solid understanding of the mathematical foundations of convex optimization.

Lastly, I would highly recommend going through the materials from the course you are currently watching, as well as any additional resources provided by the instructor. This will give you a more comprehensive understanding of the topics and help you apply them in a practical setting.

Best of luck with your studies! Keep up the great work and happy learning.

Best regards,