Is Function $f$ Convex if its Hessian Matrix is Positive Semidefinite?

  • Context: Undergrad 
  • Thread starter Thread starter Euge
  • Start date Start date
Click For Summary
SUMMARY

The discussion centers on the mathematical property of convexity in relation to the Hessian matrix of a twice differentiable function. It is established that if the Hessian matrix of a function $f : \Bbb R^n \to \Bbb R$ is positive semidefinite at all points, then the function $f$ is convex. This conclusion is based on the definition of convex functions and the implications of the Hessian matrix's properties in multivariable calculus.

PREREQUISITES
  • Understanding of convex functions in multivariable calculus
  • Knowledge of Hessian matrices and their properties
  • Familiarity with the concept of positive semidefiniteness
  • Basic proficiency in differential calculus
NEXT STEPS
  • Study the implications of positive semidefinite matrices in optimization problems
  • Learn about the relationship between convexity and optimization techniques
  • Explore examples of convex functions and their Hessians
  • Investigate the role of convexity in machine learning algorithms
USEFUL FOR

Mathematicians, students of calculus, and professionals in optimization and machine learning who seek to understand the properties of convex functions and their applications.

Euge
Gold Member
MHB
POTW Director
Messages
2,072
Reaction score
245
Here is this week's POTW:

-----
Let $f : \Bbb R^n \to \Bbb R$ be twice differentiable function whose Hessian matrix is everywhere positive semidefinite. Show that $f$ is convex.
-----

 
Physics news on Phys.org
No one answered this week’s problem. You can read my solution below.

For fixed $\mathbf{x}, \mathbf{y}\in \Bbb R^n$, define $g(t;\mathbf{x}, \mathbf{y}) = f(\mathbf{x}+ t\mathbf{y})$, for all $t\in\Bbb R$. If $H_{\mathbf{z}}(f)$ denotes the Hessian matrix evaluated at $\mathbf{z}$, then $g’’(t;\mathbf{x}, \mathbf{y}) = \frac{1}{2}\mathbf{y}^TH_{\mathbf{x} + t\mathbf{y}}(f)\mathbf{y}$, which is nonnegative by the positive semidefiniteness of the Hessian matrix. Therefore $g(t;\mathbf{x},\mathbf{y})$ is convex. Since $\mathbf{x}$ and $\mathbf{y}$ are arbitrary, it follows that $f$ is convex. Indeed, for all $\mathbf{x},\mathbf{y}\in \Bbb R^n$ and $t\in [0,1]$, $$f((1-t)\mathbf{x} + t\mathbf{y}) = f(\mathbf{x} + t(\mathbf{y}-\mathbf{x})) = g(t; \mathbf{x},
\mathbf{y}-\mathbf{x}) \le (1-t) g(0;\mathbf{x}, \mathbf{y}-\mathbf{x}) + t g(1; \mathbf{y} - \mathbf{x}) = (1-t) f(\mathbf{x}) + t f(\mathbf{y})$$ as desired.
 

Similar threads

  • · Replies 2 ·
Replies
2
Views
5K
Replies
1
Views
3K
  • · Replies 1 ·
Replies
1
Views
2K
Replies
1
Views
3K
  • · Replies 24 ·
Replies
24
Views
4K
Replies
1
Views
3K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 1 ·
Replies
1
Views
3K
  • · Replies 1 ·
Replies
1
Views
3K
Replies
1
Views
3K