Undergrad Solving Gradient of ||f(x)||^2 - Chain Rule

  • Thread starter Thread starter perplexabot
  • Start date Start date
  • Tags Tags
    Gradient
Click For Summary
The discussion focuses on solving the gradient of the squared norm of a function, specifically \triangledown_x ||f(x)||^2_2. The initial attempt using the chain rule led to confusion regarding the output being a scalar instead of a vector. After further analysis, it was clarified that the correct expression involves the Jacobian of f(x) and results in 2(\triangledown_xf(x))^Tf(x). The final formula accurately reflects the dimensional consistency needed for the gradient of a scalar-valued function. This correction highlights the importance of ensuring proper matrix multiplication in gradient calculations.
perplexabot
Gold Member
Messages
328
Reaction score
5
Hey,
I have been trying to figure out how to solve \triangledown_x ||f(x)||^2_2.
I have used the chain rule (hopefully correctly) to get the following:
\triangledown_x ||f(x)||^2_2=2\triangledown_xf(x)^T \frac{f(x)^T}{||f(x)||_2}
Is this correct?

The reason I doubt my answer is because I know the gradient of a scalar valued function should be a vector. My answer seems to give a scalar. Can anyone please shed some light...

Note: x \in \Re^n and I am using the convection that the gradient, \triangledown_x, of a function is a row vector. Also assume f: \Re^n\rightarrow \Re^m .
 
Last edited:
Physics news on Phys.org
So, I was washing the dishes when I realized that 2\triangledown_xf(x)^T \frac{f(x)^T}{||f(x)||_2} is not a scalar. This is because \triangledown_xf(x)^T is a matrix (this is actually the jacobian!). So now I have a feeling the above may be close, but still wrong. I would appreciate a confirmation tho.

Thank you :)

EDIT: Just realized this is my 300th post! YAY me! Love you physics forums!
 
Last edited:
So I have made a couple of fixes, specifically making sure the matrix multiplaction agrees "dimensionally" \triangledown_x ||f(x)||^2_2=\frac{2f(x)^T}{||f(x)||_2}(\triangledown_xf(x))
I wonder if this is correct now. Anyone?
 
My work is wrong! Here is the correct method for the sake of completeness.
<br /> \begin{equation*}<br /> \begin{split}<br /> \triangledown_x ||f(x)||^2_2 &amp;=\triangledown_x (f(x)^Tf(x)) \\<br /> &amp;=(\triangledown_xf(x)^T)^Tf(x)+(\triangledown_xf(x))^Tf(x) \\<br /> &amp;=(\triangledown_xf(x))^Tf(x)+(\triangledown_xf(x))^Tf(x) \\<br /> &amp;=2(\triangledown_xf(x))^Tf(x)<br /> \end{split}<br /> \end{equation*}<br />

Thank you all for reading.
EDIT: Also here and here are some references I used.
 

Similar threads

  • · Replies 2 ·
Replies
2
Views
1K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K
Replies
3
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 4 ·
Replies
4
Views
1K
  • · Replies 8 ·
Replies
8
Views
2K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 9 ·
Replies
9
Views
2K