Why steepest descent gives a wrong direction search?

  • Thread starter Thread starter ymhiq
  • Start date Start date
  • Tags Tags
    Direction Search
Click For Summary
SUMMARY

The discussion centers on minimizing the function ƒ(x1,x2)=(x1-1)²+x2³-x1x2 using the steepest descent method, starting from the initial point [1,1]T. The gradient was incorrectly computed as ∇ƒ(x1,x2)=[2(x1-1)-x2, 3x2²-x1], leading to erroneous values for α. After clarification, the correct formulation of the function and its gradient revealed that the initial steepest descent direction was flawed, resulting in no stationary points for the derived function φ(α) = f(x0 - α∇f(x0)). The participant acknowledged their mistake in the original problem statement and gradient calculation.

PREREQUISITES
  • Understanding of gradient descent optimization techniques
  • Familiarity with partial derivatives and gradient computation
  • Knowledge of stationary points and their significance in optimization
  • Basic proficiency in mathematical functions and their properties
NEXT STEPS
  • Review the steepest descent method and its applications in optimization
  • Learn about gradient computation for multivariable functions
  • Study the implications of stationary points in optimization problems
  • Explore alternative optimization methods such as Newton's method
USEFUL FOR

Students and professionals in mathematics, engineering, and computer science who are involved in optimization problems, particularly those using gradient-based methods.

ymhiq
Messages
11
Reaction score
0
1. Homework Statement
I have to minimize the function (x1-1)2+x23+x1x2 by the steepest descent method. The initial point is [1,1]T

Homework Equations

The Attempt at a Solution


The gradient of this function is ∇ƒ(x1,x2)=[2(x1-1)-x2 3x22-x1]. This gradient evaluated in the initial point is ∇ƒ(1,1)=[-1 2]. Following the steepest descent method it is mandatory to minimize the function ƒ(x0-α∇ƒ(x0)) in order to find the value of α. So ƒ(x0-α∇ƒ(x0))=-5α+15α2-8α3 and ƒ'(x0-α∇ƒ(x0))=-5+30α-24α2. This function has extreme points in α1=0.95061 and α2=5.094. In order to be a minimum of this curve ƒ''(x0-α∇ƒ(x0))=30-48α has to be positive. This is my problem ƒ''(x0-α∇ƒ(x0) evaluated at both α values is negative so they don´t minimize the direction. So what I am doing wrong?
 
Physics news on Phys.org
ymhiq said:
1. Homework Statement
I have to minimize the function (x1-1)2+x23+x1x2 by the steepest descent method. The initial point is [1,1]T

Homework Equations

The Attempt at a Solution


The gradient of this function is ∇ƒ(x1,x2)=[2(x1-1)-x2 3x22-x1]. This gradient evaluated in the initial point is ∇ƒ(1,1)=[-1 2]. Following the steepest descent method it is mandatory to minimize the function ƒ(x0-α∇ƒ(x0)) in order to find the value of α. So ƒ(x0-α∇ƒ(x0))=-5α+15α2-8α3 and ƒ'(x0-α∇ƒ(x0))=-5+30α-24α2. This function has extreme points in α1=0.95061 and α2=5.094. In order to be a minimum of this curve ƒ''(x0-α∇ƒ(x0))=30-48α has to be positive. This is my problem ƒ''(x0-α∇ƒ(x0) evaluated at both α values is negative so they don´t minimize the direction. So what I am doing wrong?

Just inspecting the gradient of the original function f(x1, x2), something doesn't look right.

If you take ∂f / ∂x1, how did you obtain [2(x1-1)-x2], specifically, the ' - x2' part? I'm confused, because there were no negative signs between terms in the original definition of f(x1, x2). A similar question arises in what you show to be ∂f / ∂x2.
 
  • Like
Likes   Reactions: ymhiq
ymhiq said:
1. Homework Statement
I have to minimize the function (x1-1)2+x23+x1x2 by the steepest descent method. The initial point is [1,1]T

Homework Equations

The Attempt at a Solution


The gradient of this function is ∇ƒ(x1,x2)=[2(x1-1)-x2 3x22-x1]. This gradient evaluated in the initial point is ∇ƒ(1,1)=[-1 2]. Following the steepest descent method it is mandatory to minimize the function ƒ(x0-α∇ƒ(x0)) in order to find the value of α. So ƒ(x0-α∇ƒ(x0))=-5α+15α2-8α3 and ƒ'(x0-α∇ƒ(x0))=-5+30α-24α2. This function has extreme points in α1=0.95061 and α2=5.094. In order to be a minimum of this curve ƒ''(x0-α∇ƒ(x0))=30-48α has to be positive. This is my problem ƒ''(x0-α∇ƒ(x0) evaluated at both α values is negative so they don´t minimize the direction. So what I am doing wrong?

As SteamKing has pointed out, your gradient formula is incorrect, and your initial steepest-descent direction is wrong. However, when you correct these errors, you will obtain a function ##\phi(\alpha) = f(x_0 - \alpha \nabla f(x_0))## that has no stationary points at all. What does that tell you?
 
  • Like
Likes   Reactions: ymhiq
Oh! Excuse me! You are right! However I made a mistake when I wrote the original problem. Let me write it again. I have to minimize the function ƒ(x1,x2)=(x1-1)2+x23-x1x2. The initial point is [1,1]T.
 
Excuse me all of you. Finally I got the mistake I made solved. It was an incorrect solutions of ƒ'(x0-α∇ƒ(x0))=-5+30α-24α2 .
 

Similar threads

Steepest Descent Method with Matrices
  • · Replies 1 ·
Replies
1
Views
1K
Python Finding a local max/min of a function in Python
  • · Replies 7 ·
Replies
7
Views
5K
Optimizing Functions: Strategies and Considerations
  • · Replies 8 ·
Replies
8
Views
2K
MHB What is the Gradient of a Function at a Given Point?
  • · Replies 2 ·
Replies
2
Views
2K
Graduate Steepest Descent with constraints
  • · Replies 1 ·
Replies
1
Views
4K
Finding the Inverse Function and Its Derivative - Analyzing f(x) at Point (1,1)
  • · Replies 7 ·
Replies
7
Views
3K
Is my math okay for gradient descent
  • · Replies 5 ·
Replies
5
Views
1K
Graduate Is the Magnitude of Descent Directions on Manifolds Meaningful?
  • · Replies 2 ·
Replies
2
Views
2K
Graduate Optimization Solver - BFGS method with bound constraints
  • · Replies 12 ·
Replies
12
Views
13K
Graduate Help with paper on gradient descent evolution of surfaces
  • · Replies 2 ·
Replies
2
Views
3K