1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Gradient of a function containing a matrix?

  1. Sep 28, 2014 #1
    1. The problem statement, all variables and given/known data
    http://i.imgur.com/TlDOllQ.png

    2. Relevant equations
    As stated.

    3. The attempt at a solution

    I'm not sure how to slay this beast. I know the gradient is just a partial derivative and that the solution likely involves multiple partial derivatives, one for each element in the vector x. But how would the partial derivatives be computed, given the matrices?
     
  2. jcsd
  3. Sep 29, 2014 #2

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    Just type out the problem here; do not use thumbnails. Read the pinned post 'Guidelines for students and helpers', by Vela, to see why. I cannot read your thumbnail on some media, so I will make no attempt to help.
     
  4. Sep 29, 2014 #3
    I'll help you out with the formatting, at least. This might help you if you want to post future questions.

    Let ##A \in\mathbb R^{m\times n}## and ##B \in\mathbb R^m##. Compute the gradient of$$f:\mathbb R^n\rightarrow\mathbb R,f(x)=\frac{1}{2}x^Tx+log\left(e^TE(Ax+b)\right),$$ where ## e=\left(1,1,1,\ldots,1\right)\in\mathbb R^m ## and ##E:\mathbb R^m\rightarrow\mathbb R^m## is a component wise exponential function, i.e., ##(E(x))_i=exp(x_i)## for ##i=1,2,\ldots m.##
    Use ##diag(v)## for a ##m\times m## diagonal matrix with diagonal elements given by ##v\in\mathbb R^m##
     
  5. Sep 30, 2014 #4
    Thanks for the formatting help.

    I attempted a solution by differentiating with respect to ##x_n##.

    $$\frac{\partial f}{\partial x_n} = 2x_n + \frac{A^T e^T exp(Ax+b)}{e^T exp(Ax+b)}$$

    But this isn't correct, I don't think. Shouldn't ##A^T## cancel out somewhere? Can the gradient contain a matrix in the final solution?
     
  6. Oct 1, 2014 #5

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    Just write out the original function ##f = f(x_1,x_2, \ldots, x_n)## in detail so you can see what is happening; then, after finding the derivative you can re-write the answer using matrices again, if you want. So
    [tex] f = \frac{1}{2} \sum_{i=1}^n x_i^2 + \log \left(\sum_{i=1}^n \exp (b_i + \sum_{j=1}^n a_{ij} x_j ) \right) [/tex]
    You could also use the fact that ##\log(\sum_i (\exp(g_i)) = \prod_i g_i##, but I don't know if that makes things better or worse---you would need to try it for yourself. Also, you don't want just ##\partial f / \partial x_n##, you want all the ##\partial f / \partial x_k, k = 1,
    \dots,n##.

    BTW: in TeX/LaTeX you should use "\exp" instead of "exp" and "\log" instead of "log"; this applies also to the other standard functions (the trig functions, the hyperbolic functions plus things like "max", "min", "mod", etc.) The results really do look better: you get ##\exp## instead of ##exp##, ##\log## instead of ##log##, etc.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted



Similar Discussions: Gradient of a function containing a matrix?
Loading...