Derivative of Log Determinant of a Matrix w.r.t a parameter

Click For Summary
SUMMARY

The theorem regarding the derivative of the log of the determinant of a matrix states that for a symmetric matrix A, the derivative is given by the formula: d/dx ln |A| = Tr[A^{-1} dA/dx]. The discussion highlights the derivation process, emphasizing the use of eigenvalues and eigenvectors in the expression for A and its inverse. The theorem holds true even when A is not symmetric, as confirmed by the participants.

PREREQUISITES
  • Understanding of matrix calculus, specifically derivatives of matrix functions.
  • Familiarity with symmetric matrices and their properties.
  • Knowledge of eigenvalues and eigenvectors in linear algebra.
  • Proficiency in using the trace operator in matrix operations.
NEXT STEPS
  • Study the derivation of the log determinant for non-symmetric matrices.
  • Explore applications of the theorem in optimization problems involving matrices.
  • Learn about the implications of matrix derivatives in machine learning algorithms.
  • Investigate the relationship between matrix determinants and eigenvalue distributions.
USEFUL FOR

Mathematicians, data scientists, and engineers involved in optimization, linear algebra, and matrix analysis will benefit from this discussion.

CuppoJava
Messages
23
Reaction score
0
Hi,
I'm trying to see why the following theorem is true. It concerns the derivative of the log of the determinant of a symmetric matrix.

Here's the theorem as stated:

For a symmetric matrix A:
\frac{d}{dx} ln |A| = Tr[A^{-1} \frac{dA}{dx}]

Here's what I have so far, I'm almost at the answer, except I can't get rid of the second term at the end:

A = \sum_{i} \lambda_{i} u_{i} u_{i}^{T}
A^{-1} = \sum_{i} \frac{1}{\lambda_{i}} u_{i} u_{i}^{T}

So
A^{-1} \frac{dA}{dx} = \sum_{i} \frac{1}{\lambda_{i}} u_{i} u_{i}^{T} \frac{d}{dx}(\sum_{j}\lambda_{j} u_{j} u_{j}^{T})<br /> =\sum_{i}\sum_{j}\frac{1}{\lambda_{i}}\frac{d\lambda_{j}}{dx}u_{i} u_{i}^{T}u_{j} u_{j}^{T} + \sum_{i}\sum_{j}\frac{\lambda_{j}}{\lambda_{i}}u_{i} u_{i}^{T}\frac{d}{dx}u_{j} u_{j}^{T}<br /> =\sum_{i}\frac{1}{\lambda_{i}}\frac{d\lambda_{j}}{dx}u_{i} u_{i}^{T} + \sum_{i}\sum_{j}\frac{\lambda_{j}}{\lambda_{i}}u_{i} u_{i}^{T}\frac{d}{dx}u_{j} u_{j}^{T}

And this would be just perfect if the second term was equal to zero. But I can't see how that could be made to happen.

Thanks a lot for your help
-Patrick
 
Physics news on Phys.org
This theorem is true indeed, and doesn't even need A to be symmetric.

Using :
\frac{\partial}{\partial x} ln det A = \sum_{i,j} \frac{\partial a_{ij}}{x} \frac{\partial}{\partial a_{i,j}}[\tex]<br /> <br /> with : <br /> \frac{\partial }{\partial c_{ij}} ln det A = (A^{-1})_{ji}[\tex]&lt;br /&gt; &lt;br /&gt; you get : &lt;br /&gt; \frac{\partial}{\partial x} ln det A = Tr(A^{-1}\frac{\partial A}{\partial x}) = Tr(\frac{\partial A}{\partial x}A^{-1})[\tex]&amp;lt;br /&amp;gt; &amp;lt;br /&amp;gt; I hope that will help...&amp;lt;br /&amp;gt; &amp;lt;br /&amp;gt; Canag
 

Similar threads

Replies
0
Views
892
  • · Replies 5 ·
Replies
5
Views
3K
  • · Replies 12 ·
Replies
12
Views
2K
Replies
4
Views
1K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 6 ·
Replies
6
Views
2K
  • · Replies 16 ·
Replies
16
Views
2K
  • · Replies 8 ·
Replies
8
Views
5K
  • · Replies 2 ·
Replies
2
Views
1K