Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Derivative of a vector function

  1. Feb 8, 2012 #1

    I am reading through a book called "Matrix Differential Calculus" by Magnus and Neudecker. They go through taking the derivative of a vector in quadratic form that I need help with.

    For [itex] \vec{x} [/itex] being a vector and A being a constant square matrix
    [tex] \frac {d(\vec{x}^TA\vec{x})} {d\vec{x}} [/tex]
    [tex]= \frac {d\vec{x}^T}{d\vec{x}}A\vec{x} + \vec{x}^TA\frac {d\vec{x}}{d\vec{x}} [/tex]
    [tex]= (\frac {d\vec{x}^T}{d\vec{x}}A\vec{x})^T + \vec{x}^TA\frac {d\vec{x}}{d\vec{x}} {?}[/tex]
    [tex]= \vec{x}^TA^T\frac {d\vec{x}}{d\vec{x}} + \vec{x}^TA\frac {d\vec{x}}{d\vec{x}} [/tex]
    [tex]= \vec{x}^T(A+A^T)\frac {d\vec{x}}{d\vec{x}} [/tex]

    [tex] \frac {d(\vec{x}^TA\vec{x})} {d\vec{x}}=\vec{x}^T(A+A^T) [/tex]

    I don't understand how one can go from the second line to the third line
  2. jcsd
  3. Feb 8, 2012 #2
    It's because [itex] \frac {d\vec{x}^T}{d\vec{x}}A\vec{x} [/itex] is a scalar quantity.
  4. Feb 8, 2012 #3
    I don't see how it is a scalar quantity. We proved that for [itex] \vec{y}=A\vec{x}, \frac {d\vec{y}}{d\vec{x}}=A [/itex] using the definition [itex] \frac {d\vec{y}}{d\vec{x}}=[(\frac {d}{d\vec{x}})^T (\vec{y}^T)]^T [/itex]. This yields a matrix for [itex] \frac {d\vec{y}}{d\vec{x}} [/itex] and does as well for [itex] \frac {d\vec{x}^T}{d\vec{x}}[/itex].

    This is the site that I am looking at
  5. Feb 8, 2012 #4
    Oh, yeah sorry you're right. I don't know what I was thinking.
  6. Feb 8, 2012 #5
    Well you could write it out in index notation

    [tex] \frac{\partial}{\partial x_i} \left( x_j x_k A_{jk}\right) = \frac{\partial x_j}{\partial x_i} x_k A_{jk} + x_j\frac{\partial x_k}{x_i} A_{jk}[/tex]
    [tex] = \delta_{ji} x_k A_{jk} + x_j \delta_{ki} A_{jk}[/tex]
    [tex] = x_k A_{ik} + x_j A_{ji} [/tex]
    [tex] = x_k A_{ik} + x_k A_{ki} [/tex]
    [tex] = \left( \bf{A} + \bf{A}^T\right)x[/tex]
  7. Feb 8, 2012 #6
    Yeah. that does make sense, thanks. But does anybody know why my original line of thought is wrong?
  8. Feb 8, 2012 #7
    The dimensional inconsistency is really in the second line, where one term is nx1 and the other 1xn. It almost seems to me like you'd have to write
    [tex] \frac {d(\vec{x}^TA\vec{x})} {d\vec{x}} [/tex]
    [tex]= \frac {d\vec{x}^T}{d\vec{x}}A\vec{x} + \left(\vec{x}^TA\frac {d\vec{x}}{d\vec{x}}\right)^T [/tex]
    [tex]= \left((\frac {d\vec{x}^T}{d\vec{x}}A\vec{x})^T + \vec{x}^TA\frac {d\vec{x}}{d\vec{x}}\right)^T[/tex]
    [tex]= \left(\vec{x}^TA^T\frac {d\vec{x}}{d\vec{x}} + \vec{x}^TA\frac {d\vec{x}}{d\vec{x}}\right)^T [/tex]
    [tex]= \left(\vec{x}^T(A+A^T)\frac {d\vec{x}}{d\vec{x}}\right)^T [/tex]
    [tex] = (A + A^T)\vec{x}[/tex]
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook