# Matrix Calculus

1. Feb 27, 2008

### hotvette

Referring to:

Is there an easy way to illustrate why the following is true?

$$d(x^TCx) = x^T(C+C^T)dx$$

My attempt at using the product rule doesn't seem to work:

$$A = x^TC$$

$$B = x$$

$$d(AB) = (dA)B + A(dB) = d(x^TC)x + x^TC$$

Last edited: Feb 27, 2008
2. Feb 27, 2008

### John Creighto

$$d(X^TCX): = (X^TCdX): + (d(X^T) CX): = (I ¤ X^TC) dX: + (X^TC^T ¤ I) dX^T:$$

Anyway, it looks like they did exactly what you did, except that they didn't set dx to the identity matrix. You are assuming differentiation by dx. This might not be the case. Also notice that (d(X^T) CX) is scaler and thus invariant to the transpose operator. They multiplied this scaler by the identity matrix so adding the two matrices make sense. Anyway, nice link. It is a good reference.

Last edited: Feb 27, 2008
3. Feb 28, 2008

### hotvette

I was just trying to understand why the 2nd item under the heading "Differential of Quadratic Products":

$$d(x^TCx) = x^T(C+C^T)dx = [C=C^T]2x^TCdx$$

is true. Is there any way to illustrate?

Assuming the standard product rule is valid, it means:

$$d/dx(x^TCx) = x^TC + (d/dx(x^T))Cx = x^TC + x^TC^T=x^T(C+C^T)$$

What I don't understand is why:

$$(d/dx(x^T))Cx = x^TC^T$$

4. Feb 29, 2008

### trambolin

$$d(x^TCx)=d(x^TC^Tx)$$

$$x^TCdx + (dx)^TCx = x^TC^Tdx + (dx)^TC^Tx$$

Since they are scalars

$$x^TCdx + \left((dx)^TCx\right)^T = x^TC^Tdx + \left((dx)^TC^Tx\right)^T$$

$$x^TCdx + \left(x^TC^Tdx\right) = x^TC^Tdx + \left(x^TCdx\right)$$

$$x^T\left(C+C^T\right)dx = x^T\left(C^T+C\right)dx$$

If $$C$$ is symmetric then that equals $$2x^TCdx$$

5. Mar 2, 2008

### hotvette

Thanks, I get it now. The key is the fact that the expressions evaluate to scalars. John Creighto mentioned that also. I didn't see that before. Thanks to both of you.