1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Unconstrained optimziation: Local minimum

  1. Jan 24, 2012 #1
    1. The problem statement, all variables and given/known data
    Let f: M->R
    where M is a open subset of Rn
    Suppose f is C2(M)
    Let x E M such that
    "gradient of f at x" = 0 and the Hessian of f at x is positive definite
    Then x is a strict local minimum point of f.
    The above theorem is given in my textbook.

    If instead we have "gradient of f at x" = 0 and the Hessian of f at x is positive SEMI-definite, can we conclue that x is a local minimum point of f? Why or why not?
    This puzzles me and I can't find this in the textbook.

    2. Relevant equations
    Unconstrained optimziation

    3. The attempt at a solution

    Any help is appreciated!
  2. jcsd
  3. Jan 24, 2012 #2

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    Look at the function f(x,y) = (y - x^2)*(y - 2x^2) near (0,0). We have that (0,0) is a stationary point and the Hessian of f at (0,0) is positive semi-definite. However, (0,0) is neither a local max nor min. To see this, note that for y > 2x^2, y is above both parabolas y = x^2 and y = 2x^2, so both factors of f are > 0 and f > 0. For y < x^2, y is below both parabolas, so each factor of f is < 0 but f > 0. However, for x^2 < y < 2x^2, y is between the two parabolas, so one factor of f is > 0 and the other factor is < 0, giving f < 0. Therefore, in any neighbourhood of (0,0) there are points (x,y) where f(x,y) > f(0,0) = 0, and other points (x,y) where f(x,y) < f(0,0).

    In general, having a positive semi-definite Hessian is a second-order NECESSARY condition for a minimum, but the sufficient condition for a (strict, local) minimum is stronger. Actually, this is even true in one dimension; just look at the two functions f1(x) = x^2 and f2(x) = x^3 at x = 0. In this case, the analog of positive-semidefinite is ">=0", while the analog of positive-definite is ">0".

    Last edited: Jan 24, 2012
  4. Jan 25, 2012 #3
    I see.

    I have a few more related questions...

    1) Does positive definite always imply postive semi-definite?

    2) For a one-variable function f:R->R, is the Hessian being positive definite at x exactly equivalent to saying that f ''(x)>0?

    Thanks for answering!
  5. Jan 26, 2012 #4

    Ray Vickson

    User Avatar
    Science Advisor
    Homework Helper

    Yes to both.

Share this great discussion with others via Reddit, Google+, Twitter, or Facebook