Why is d(x^TCx) equal to x^T(C+C^T)dx?

[hotvette]

Referring to:

http://www.ee.ic.ac.uk/hp/staff/dmb/matrix/calculus.html#deriv_quad

Is there an easy way to illustrate why the following is true?

$$d(x^TCx) = x^T(C+C^T)dx$$

My attempt at using the product rule doesn't seem to work:

$$A = x^TC$$

$$B = x$$

$$d(AB) = (dA)B + A(dB) = d(x^TC)x + x^TC$$

 

[John Creighto]

Referring to:

http://www.ee.ic.ac.uk/hp/staff/dmb/matrix/calculus.html#deriv_quad

Is there an easy way to illustrate why the following is true?

$$d(x^TCx) = x^T(C+C^T)dx$$

My attempt at using the product rule doesn't seem to work:

$$A = x^TC$$

$$B = x$$

$$d(AB) = (dA)B + A(dB) = d(x^TC)x + x^TC$$

From your link:

$$d(X^TCX): = (X^TCdX): + (d(X^T) CX): = (I ¤ X^TC) dX: + (X^TC^T ¤ I) dX^T:$$

Anyway, it looks like they did exactly what you did, except that they didn't set dx to the identity matrix. You are assuming differentiation by dx. This might not be the case. Also notice that (d(X^T) CX) is scaler and thus invariant to the transpose operator. They multiplied this scaler by the identity matrix so adding the two matrices make sense. Anyway, nice link. It is a good reference.

 

[hotvette]

I was just trying to understand why the 2nd item under the heading "Differential of Quadratic Products":

$$d(x^TCx) = x^T(C+C^T)dx = [C=C^T]2x^TCdx$$

is true. Is there any way to illustrate?

Assuming the standard product rule is valid, it means:

$$d/dx(x^TCx) = x^TC + (d/dx(x^T))Cx = x^TC + x^TC^T=x^T(C+C^T)$$

What I don't understand is why:

$$(d/dx(x^T))Cx = x^TC^T$$

 

[trambolin]

$$d(x^TCx)=d(x^TC^Tx)$$

$$x^TCdx + (dx)^TCx = x^TC^Tdx + (dx)^TC^Tx$$

Since they are scalars

$$x^TCdx + \left((dx)^TCx\right)^T = x^TC^Tdx + \left((dx)^TC^Tx\right)^T$$

$$x^TCdx + \left(x^TC^Tdx\right) = x^TC^Tdx + \left(x^TCdx\right)$$

$$x^T\left(C+C^T\right)dx = x^T\left(C^T+C\right)dx$$

If $$C$$ is symmetric then that equals $$2x^TCdx$$

 

[hotvette]

Thanks, I get it now. The key is the fact that the expressions evaluate to scalars. John Creighto mentioned that also. I didn't see that before. Thanks to both of you.