Derivative (mimization) of matrix trace

[onako]

Given a function
f(X)= Tr(X'AX) - 2Tr(X'BC), with X' denoting matrix transpose, I'm supposed to find the expression used to miminize the function with respect to X. The derivatives should be used, but I'm not sure how to proceed.
Any help is appreciated.

 

[Simon_Tyler]

Using linearity of the trace you get
<br /> f(X) = Tr(X^t (A-2B) X)<br />

Then you can take the matrix derivative using the chain rule
<br /> \frac{\partial f(X)}{\partial X} = (A-2B)X + (A-2B)^t X = (A+A^t-2B-2B^t)X<br />
This can be checked using index notation if you're not sure of how the derivatives work.

Finally, you have just a linear equation to solve in order to find the stationary points of f(X).
The columns of X must be null vectors of the symmetric matrix (A+A'-2B-2B').

Here's some Mathematica code that let's you check it for any dimension n.

In[1]:= (n=2;X=Array[x,{n,n}];A=Array[a,{n,n}];B=Array[b,{n,n}]);
In[2]:= Tr[X\[Transpose].A.X]-2Tr[X\[Transpose].B.X]==Tr[X\[Transpose].(A-2B).X]//Expand
Out[2]= True
In[3]:= D[X,{X}]==Array[Boole[#1==#3 && #2==#4]&,{n,n,n,n}]
Out[3]= True
In[4]:= D[Tr[X\[Transpose].A.X],{X}]==Array[D[Tr[X\[Transpose].A.X],x[##]]&,{n,n}]
Out[4]= True
In[5]:= D[Tr[X\[Transpose].A.X],{X}]== (A+A\[Transpose]).X//Expand
Out[5]= True

--------------------------------------
The LaTeX image generation wasn't working, so here's the first two equations:
f(X) = Tr(X' (A-2B) X)
f'(X) = (A-2B) X + (A-2B)' X = (A+A'-2B-2B') X

 

[onako]

Thanks.
I'm not sure I understand this derivation.
Given f(X)= Tr(X'AX) - 2Tr(X'BC), X' denoting transpose of X,
I'm supposed to find the first derivative of f(x).
Could you follow step by step procedure; first derivative of Tr(X'AX), and then derivative of 2Tr(X'BC)?
The final derivative should have A, B, C, and X.

 

[Simon_Tyler]

Sorry about that, I misread your question as
f(X)= Tr(X'AX) - 2Tr(X'BX)

------

So, taking the derivative of your actual expression gives

f'(X)= (A+A')X - 2BC

Which is a linear equation you can solve,

------

The best (I imagine that's subjective) way to be sure that you got all of the transposes correct is to use index notation. E.g. https://www.physicsforums.com/showthread.php?t=198712. Then you basically just take derivatives wrt the components, instead of the whole matrix.

I've written up a detailed derivation, but since I have a suspicion that this is a homework question, and you haven't shown any working so far, I might hold off posting it.
Read the notes I linked to and have a go at proving the result. Post what you get here for either congratulations or some more help!

 

[onako]

Thanks a lot. I need a rationale for the procedure I'm supposed to code. So, in a sense, this (as everything) is a 'homework' question. But, for real homeworks, I'm a bit too old. :)

 

[Simon_Tyler]

Fair enough!

I've attached a pdf that contains a derivation using index notation.
If you have any questions, just ask.