Steepest Descent Method with Matrices

  • Thread starter Thread starter ver_mathstats
  • Start date Start date
  • Tags Tags
    Matrices Method
Click For Summary
SUMMARY

The discussion focuses on the Steepest Descent Method applied to the function f(x)=(1/2)(xT)Qx+qTx-B, where the search direction is defined as sk=-∇f(xk). The gradient ∇f(xk) is clarified as ∇f(xk)=Qxk+q when Q is symmetric. The confusion arises from whether the gradient should include a negative sign or not, with the consensus being that it does not when Q is symmetric.

PREREQUISITES
  • Understanding of gradient descent algorithms
  • Familiarity with matrix calculus
  • Knowledge of quadratic forms in optimization
  • Experience with symmetric matrices
NEXT STEPS
  • Study the properties of symmetric matrices in optimization
  • Learn about the Conjugate Gradient method and its relationship to the Steepest Descent Method
  • Explore the derivation of gradients for quadratic functions
  • Investigate numerical methods for solving optimization problems
USEFUL FOR

Mathematicians, optimization specialists, and computer scientists working with numerical methods and matrix operations will benefit from this discussion.

ver_mathstats
Messages
258
Reaction score
21
Homework Statement
Perform the steepest descent method with exact line search for the function f(x)=(1/2)(x^T)Qx+(q^T)x-B.
Relevant Equations
f(x)=(1/2)(x^T)Qx+(q^T)x-B
We are given f(x)=(1/2)(xT)Qx+qTx-B where xk+1=xkksk, the search direction is sk=-∇f(xk). Q is a 2x2 matrix and q is 2x1 matrix and B=6. My issue is I'm confused what -∇f(xk) is, is ∇f(xk)=Q(xk)-q? Just like how it is in Conjugate Gradient/Fletcher Reeve's method? Or is it Q(xk)+q?

Thank you
 
Last edited:
Physics news on Phys.org
\nabla f = \frac12(Q^T + Q)x + q, which is Qx + q if Q is symmetric.
 

Similar threads

  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 13 ·
Replies
13
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 4 ·
Replies
4
Views
4K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 1 ·
Replies
1
Views
5K
  • · Replies 5 ·
Replies
5
Views
4K
Replies
4
Views
2K
  • · Replies 80 ·
3
Replies
80
Views
10K