How do we know the principle of least action is true?

It gives the right answers. What more do you expect?

 

In some sense, it's a mathematical abstraction of things we already knew [like "shortest distance between two points is a line", geometric optics, newton's laws of motion]... and we take pleasure to find that newer theories [developed after (say) Lagrange] seem to fit into this scheme... for example, quantum mechanics.

Implicit in this discussion is how one finds action that yield the equations of motion.

If you are interested in Lagrange's motivation, that's more of a historical question... which might be answered in (say) https://www.amazon.com/Variational-Principles-Mechanics-Physics-Chemistry/dp/0486650677".

You might find some starting points here in this wikipedia article
http://en.wikipedia.org/wiki/History_of_variational_principles_in_physics (standard caveats apply).

You might find this section from The Structure and Intrepretation of Classical Mechanics interesting
http://mitpress.mit.edu/SICM/book-Z-H-8.html#%_sec_1.1





If you want something deeper, then you might have to look at (say) Arnold or Abraham-Marsden.

 

Back when I was an undergrad, my analytical mechanics TA briefly mentioned that the principle of least action arose from some sort of theological argument. Unfortunately I don't have any further details on that.

 

another good book to learn these stuff is the "classical" Classical Mechanics book by Goldstein.

to understand these, need to get your mind around Hamilton's or D'Alembert principle