Lagrangian Gradient Simplification

Click For Summary
SUMMARY

The discussion focuses on the derivation of the gradient of the Lagrangian term \(\lambda_{2}(w^{T} \Sigma w - \sigma^{2}_{\rho})\) with respect to the vector \(w\). It clarifies that \(w^{T} \Sigma w\) should be expanded fully to avoid errors, resulting in the expression \(\sum_{i=1}^n \sum_{j=1}^n \sigma_{ij} w_i w_j\). This expansion includes both diagonal and off-diagonal terms, emphasizing the importance of precise notation in mathematical derivations. Additionally, the discussion advises on proper formatting for mathematical expressions in forum posts.

PREREQUISITES
  • Understanding of Lagrangian mechanics
  • Familiarity with gradient calculations
  • Knowledge of covariance matrices, specifically symmetric matrices
  • Proficiency in mathematical notation and formatting for online discussions
NEXT STEPS
  • Study the properties of symmetric covariance matrices in optimization problems
  • Learn about the derivation of gradients in Lagrangian formulations
  • Explore advanced mathematical notation techniques for online forums
  • Investigate the implications of gradient simplification in machine learning models
USEFUL FOR

Mathematicians, data scientists, and researchers involved in optimization and machine learning who require a deeper understanding of Lagrangian methods and gradient calculations.

zmalone
Messages
10
Reaction score
0
From the attached image problem:
When deriving the third term in the Lagrangian:
[itex]\lambda[/itex][itex]_{2}[/itex](w[itex]^{T}[/itex]∑w - [itex]\sigma[/itex][itex]^{2}_{\rho}[/itex]) with respect to w, are w[itex]^{T}[/itex] and w used like a w[itex]^{2}[/itex] to arrive at the gradient or am I oversimplifying and it just happens to work out on certain problems like this?

(∑ is an n x n symmetric covariance matrix and w is n x 1 vector)
 

Attachments

  • LinAlgQuestion1.jpg
    LinAlgQuestion1.jpg
    20.2 KB · Views: 422
Physics news on Phys.org
zmalone said:
From the attached image problem:
When deriving the third term in the Lagrangian:
[itex]\lambda[/itex][itex]_{2}[/itex](w[itex]^{T}[/itex]∑w - [itex]\sigma[/itex][itex]^{2}_{\rho}[/itex]) with respect to w, are w[itex]^{T}[/itex] and w used like a w[itex]^{2}[/itex] to arrive at the gradient or am I oversimplifying and it just happens to work out on certain problems like this?

(∑ is an n x n symmetric covariance matrix and w is n x 1 vector)

The best way of avoiding errors, at least when you are starting, is to write it out in full:
[tex]w^T \Sigma w = \sum_{i=1}^n \sum_{j=1}^n \sigma_{ij} w_i w_j<br /> = \sum_{i=1}^n \sigma_{ii} w_i^2 + 2 \sum_{i < j} \sigma_{ij} w_i w_j[/tex]

BTW: you don't need all those '[ itex]' tags: just enclose the whole expression between '[ itex]' and ' [/itex]' (although I prefer an opening '# #' (no space) followed by a closing '# #' (again, no space). Compare this: ##\lambda_2 (w^T \Sigma w - \sigma_{\rho}^2)## with what you wrote. (Press the quote button, as though you were composing a reply; that will show you the code. You can then just abandon the reply and not post it.)
 
Last edited:

Similar threads

Replies
8
Views
2K
  • · Replies 8 ·
Replies
8
Views
2K
Replies
4
Views
2K
  • · Replies 8 ·
Replies
8
Views
2K
Replies
3
Views
2K
  • · Replies 6 ·
Replies
6
Views
3K
  • · Replies 11 ·
Replies
11
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
Replies
7
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K