Derivate of this function of matrix

[abhisheksingh]

Hi ,

I am stuck with the problem of finding derivative wrt. A of following function:

Def:
y - column vector of size mx1
A - matrix of size m x n
b - column vector of size n x 1
K - matrix of size n x n
y' - denotes the transpose
K^-1 - denotes inverse

f(A) = (y - Ab)' (AKA')^-1 (y-Ab)


Can anyone please help me?

thanks,
abhishek.

 

[Hurkyl]

It looks like a straightforward application of differentiation rules -- where are you having trouble?

 

[abhisheksingh]

I am not able to differentiate it. If I expand the terms and apply chain rule, there are terms which involve differentiation of matrix wrt. a matrix, which will result in some kronecker product terms, while the size of final derivative has to be m x n.

Probably I am missing something.

 

[abhisheksingh]

hi Hurkyl,

can you help me in this?

thanks ,
abhishek.

 

[Hurkyl]

Where specifically do you run into the problem? Some expressions seem like there can't possibly be any difficulty. For example, the derivative of (y-Ab) is surely -(dA)b, where I've used dA to denote whatever the heck the derivative of A with respect to A is supposed to be. These sorts of derivative rules tend to be correct no matter what you're differentiating with respect to. (In fact, if they weren't, we usually wouldn't call the operation a derivative!)


Incidentally, out of curiousity, just what is the definition you're using for 'derivative with respect to A'?

 

[abhisheksingh]

Here it goes:

d{(y-Ab)' (AKA')^-1 (y-Ab)}
= d{(y-Ab)'(AKA')^-1} (y-Ab) + (y-Ab)' (AKA')^-1 d{(y-Ab)}
= [d{(y-Ab)'} (AKA')^-1 + (y-Ab)' d{(AKA')^-1}] (y-Ab) - (y-Ab)' (AKA')^-1 (dA) b
= -[b' (dA)' (AKA')^-1 + (y-Ab)' d{(AKA')^-1}] (y-Ab) - (y-Ab)' (AKA')^-1 (dA) b

After this, how to evaluate d{(AKA')^-1} ? Further, how to get the closed form expression for d{f(A)}/dA ?abhishek

 

[Hurkyl]

d{(AKA')^-1}

Ah, so you don't have a rule for differentiating an inverse?

In that case, you can try implicit differentiation to work it out -- let B be a matrix function of A. Can you think of any good equations involving B^{-1} in which you know how to differentiate everything else?


Given the symmetry under transpose of the original expression, you might be able to simplify some things. e.g. that final term is the transpose of one of the terms you get if you distribute that product. I expect those terms to actually be equal, but that does depend upon precisely what dA means.