Finding Stationary Points of a Matrix Function: Derivatives and Eigenvectors

[libragirl79]

Hi,

I am trying to find stationary points of the function f(x)=(xtAx)/(xtx) (the division of x transpose times A times x divided by x transpose x) where A is a px1 symmetric matrix. I need to take the derivative of this to show that when i set it to zero i get the eigenvectors of A. I know how to take the derivative of xtx in respect to x, which is xtAt + xtA and since A=At, then this would be 2Axt, but I don't know what to do when it's a division, is there an equivalent of a quotient rule for matrix derivatives?

Thanks very much!

 

[I like Serena]

Hi,

I am trying to find stationary points of the function f(x)=(xtAx)/(xtx) (the division of x transpose times A times x divided by x transpose x) where A is a px1 symmetric matrix. I need to take the derivative of this to show that when i set it to zero i get the eigenvectors of A. I know how to take the derivative of xtx in respect to x, which is xtAt + xtA and since A=At, then this would be 2Axt, but I don't know what to do when it's a division, is there an equivalent of a quotient rule for matrix derivatives?

Thanks very much!

Welcome to PF, libragirl79!

Suppose you diffentiate $f(\mathbf x) = f((^x_y)) = {(x\ y)A(^x_y) \over (x\ y)(^x_y)}$.

First you would use the quotient rule to differentiate with respect to x to find:
$${\partial f \over \partial x} = {((1\ 0)A(^x_y) + (x\ y)A(^1_0))\cdot (x\ y)(^x_y) - (x\ y)A(^x_y) \cdot 2x \over ((x\ y)(^x_y))^2}$$

Now let's rewrite, using $A=A^T$, with $\mathbf x$:
$${\partial f \over \partial x} = {2\mathbf x^T A (^1_0) \cdot \mathbf x^T\mathbf x - \mathbf x^T A \mathbf x \cdot 2\mathbf x^T(^1_0) \over (\mathbf x^T\mathbf x)^2}$$
$${\partial f \over \partial x} = {2\mathbf x^T A \cdot \mathbf x^T\mathbf x - \mathbf x^T A \mathbf x \cdot 2\mathbf x^T \over (\mathbf x^T\mathbf x)^2}(^1_0)$$

In other words, if $A=A^T$:
$$Df(\mathbf x) = {2\mathbf x^T A \cdot \mathbf x^T\mathbf x - \mathbf x^T A \mathbf x \cdot 2\mathbf x^T \over (\mathbf x^T\mathbf x)^2}$$

 

[libragirl79]

Thanks for your reply!

Since I forgot a lot of my matrix algebra, when you say A (x y) it looks like x choose y, but i am assuming that's not what you meant...

 

[I like Serena]

Thanks for your reply!

Since I forgot a lot of my matrix algebra, when you say A (x y) it looks like x choose y, but i am assuming that's not what you meant...

Since you are talking about x transpose, that means that x is a vector.
A vector is commonly written as:
$$\mathbf x = \begin{pmatrix}x_1 \\ x_2 \\ \vdots \\ x_n\end{pmatrix}$$

I have simplified it to:
$$\mathbf x = \begin{pmatrix}x \\ y\end{pmatrix}$$

So indeed, this does not mean x choose y. ;)


Furthermore:
$$\mathbf x^T = \begin{pmatrix}x_1 & x_2 & \dots & x_n\end{pmatrix}$$
and
$$\mathbf x^T \mathbf x = \begin{pmatrix}x_1 & x_2 & \dots & x_n\end{pmatrix}\begin{pmatrix}x_1 \\ x_2 \\ \vdots \\ x_n\end{pmatrix} = x_1^2 + x_2^2 + \dots + x_n^2$$

 

[libragirl79]

Ok, I see! Thanks! :)