Solving Gradient of ||f(x)||^2 - Chain Rule

  • Context: Undergrad 
  • Thread starter Thread starter perplexabot
  • Start date Start date
  • Tags Tags
    Gradient
Click For Summary

Discussion Overview

The discussion centers around the calculation of the gradient of the squared norm of a function, specifically \(\nabla_x ||f(x)||^2_2\). Participants explore the application of the chain rule in this context, addressing concerns about dimensionality and the nature of the resulting expressions. The scope includes mathematical reasoning and technical explanation.

Discussion Character

  • Mathematical reasoning
  • Technical explanation
  • Debate/contested

Main Points Raised

  • One participant proposes using the chain rule to derive \(\nabla_x ||f(x)||^2_2 = 2\nabla_x f(x)^T \frac{f(x)^T}{||f(x)||_2}\) but expresses doubt about the correctness due to the expectation that the gradient should yield a vector.
  • Another participant realizes that \(\nabla_x f(x)^T\) is a matrix (the Jacobian), suggesting that the initial expression may not be a scalar, indicating a potential error in the formulation.
  • A subsequent post attempts to correct earlier claims by ensuring dimensional consistency in the expression, proposing \(\nabla_x ||f(x)||^2_2 = \frac{2f(x)^T}{||f(x)||_2}(\nabla_x f(x))\) and seeks confirmation on its correctness.
  • Finally, a participant presents what they claim to be the correct method for calculating the gradient, providing a detailed derivation that results in \(2(\nabla_x f(x))^T f(x)\), while also acknowledging their previous mistakes.

Areas of Agreement / Disagreement

Participants do not reach a consensus on the correctness of the initial expressions. There are multiple competing views regarding the proper application of the chain rule and the dimensionality of the resulting expressions.

Contextual Notes

Some participants express uncertainty about the correctness of their calculations and the dimensionality of the terms involved. The discussion reflects a progression of ideas and corrections without resolving the underlying mathematical questions definitively.

perplexabot
Gold Member
Messages
328
Reaction score
5
Hey,
I have been trying to figure out how to solve \triangledown_x ||f(x)||^2_2.
I have used the chain rule (hopefully correctly) to get the following:
\triangledown_x ||f(x)||^2_2=2\triangledown_xf(x)^T \frac{f(x)^T}{||f(x)||_2}
Is this correct?

The reason I doubt my answer is because I know the gradient of a scalar valued function should be a vector. My answer seems to give a scalar. Can anyone please shed some light...

Note: x \in \Re^n and I am using the convection that the gradient, \triangledown_x, of a function is a row vector. Also assume f: \Re^n\rightarrow \Re^m .
 
Last edited:
Physics news on Phys.org
So, I was washing the dishes when I realized that 2\triangledown_xf(x)^T \frac{f(x)^T}{||f(x)||_2} is not a scalar. This is because \triangledown_xf(x)^T is a matrix (this is actually the jacobian!). So now I have a feeling the above may be close, but still wrong. I would appreciate a confirmation tho.

Thank you :)

EDIT: Just realized this is my 300th post! YAY me! Love you physics forums!
 
Last edited:
So I have made a couple of fixes, specifically making sure the matrix multiplaction agrees "dimensionally" \triangledown_x ||f(x)||^2_2=\frac{2f(x)^T}{||f(x)||_2}(\triangledown_xf(x))
I wonder if this is correct now. Anyone?
 
My work is wrong! Here is the correct method for the sake of completeness.
<br /> \begin{equation*}<br /> \begin{split}<br /> \triangledown_x ||f(x)||^2_2 &amp;=\triangledown_x (f(x)^Tf(x)) \\<br /> &amp;=(\triangledown_xf(x)^T)^Tf(x)+(\triangledown_xf(x))^Tf(x) \\<br /> &amp;=(\triangledown_xf(x))^Tf(x)+(\triangledown_xf(x))^Tf(x) \\<br /> &amp;=2(\triangledown_xf(x))^Tf(x)<br /> \end{split}<br /> \end{equation*}<br />

Thank you all for reading.
EDIT: Also here and here are some references I used.
 

Similar threads

  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 4 ·
Replies
4
Views
3K
  • · Replies 8 ·
Replies
8
Views
1K
Replies
3
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 9 ·
Replies
9
Views
3K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K