Why are positive definite matrices useful?

[golmschenk]

I've recently been learning about how to tell if a matrix is positive definite and how to create a positive definite matrix, but I haven't been given a reason why they're useful yet. I'm sure there are plenty of reasons, I just haven't seen them yet. In what ways do the properties of a positive definite matrix make them advantageous to have? Thanks for your time!

 

[monea83]

One reason is that if a matrix A is positive definite, the quadratic form

$$f(x) = \frac{1}{2} x A^T x + b^Tx + c$$

has a unique minimum (expressions like these crop up in a number of places). A positive definite matrix A can be visualized as a paraboloid (look at the graph of f) that is stretched in the directions of A's eigenvectors. If A is indefinite, the graph will have a saddle point instead of a nice minimum (or be degenerated further).

An article that explains this (and some other linear algebra key ideas) nicely is "Introduction to the Conjugate Gradient Method Without the Agonizing Pain" by JR Shewchuck.

 

[AlephZero]

Numerical algorithms on positive definite matrices are usually well behaved. The underlying reason is that all the eigenvalues are positive, so the sort of operations that occur in numerical methods don't lose precision when positive and negative quantities are added and cancel out. (Of course individual elements of a positive definite matrix can be negative, but in a sense they can't be "negative enough" to cause numerical problems.)

This means there are usually faster and simpler numerical algorithms for positive definite matrices than for general matrices.

In physics, matrices are often Hermitian (which includes real symmetric matrices) as well as positive definite, and the product x^t A x represents some kind of work or energy.