- #1
Kreizhn
- 743
- 1
Hi,
I'm trying to do a constrained optimization problem. I shall omit the details as I don't think they're important to my issue. Let [itex] f:\mathbb R^n \to \mathbb R [/itex] and [itex] c:\mathbb R^n \to \mathbb R^+\cup\{0\}[/itex] be differentiable functions, where [itex] \mathbb R^+ = \left\{ x \in \mathbb R : x> 0 \right\} [/itex]. The problem I want to solve is
[tex] \min_{\vec x\in\mathbb R^n} f(\vec x), \quad \text{ subject to } c(\vec x) = 0 [/tex]
Now all constrained optimization procedures that I know of use Lagrange multipliers. In this case, the Lagrangian would be
[tex] L(\vec x,\lambda) = f(\vec x) - \lambda c(\vec x) [/tex]
and the key to this theory is that at the optimal point [itex] \vec x^* [/itex] the gradients are parallel, so that
[tex] \nabla f(\vec x^*) = \lambda \nabla c(\vec x^*) [/itex]
However, every x that satisfies [itex] c(\vec x)=0[/itex] is a minima of [itex]c(\vec x)[/itex], we have that [itex] \nabla c(\vec x) =\vec 0 [/itex] for all feasible points [itex] \vec x[/itex]. Hence the gradient of the constraint is orthogonal to every feasible point and Lagrange multipliers breaks down.
Is there a way to fix this? To change the constraint so that the gradient is non-zero? Should I switch to a non-Lagrange multiplier method? If such a method exists, what are they called? Any help would be appreciated.
I'm trying to do a constrained optimization problem. I shall omit the details as I don't think they're important to my issue. Let [itex] f:\mathbb R^n \to \mathbb R [/itex] and [itex] c:\mathbb R^n \to \mathbb R^+\cup\{0\}[/itex] be differentiable functions, where [itex] \mathbb R^+ = \left\{ x \in \mathbb R : x> 0 \right\} [/itex]. The problem I want to solve is
[tex] \min_{\vec x\in\mathbb R^n} f(\vec x), \quad \text{ subject to } c(\vec x) = 0 [/tex]
Now all constrained optimization procedures that I know of use Lagrange multipliers. In this case, the Lagrangian would be
[tex] L(\vec x,\lambda) = f(\vec x) - \lambda c(\vec x) [/tex]
and the key to this theory is that at the optimal point [itex] \vec x^* [/itex] the gradients are parallel, so that
[tex] \nabla f(\vec x^*) = \lambda \nabla c(\vec x^*) [/itex]
However, every x that satisfies [itex] c(\vec x)=0[/itex] is a minima of [itex]c(\vec x)[/itex], we have that [itex] \nabla c(\vec x) =\vec 0 [/itex] for all feasible points [itex] \vec x[/itex]. Hence the gradient of the constraint is orthogonal to every feasible point and Lagrange multipliers breaks down.
Is there a way to fix this? To change the constraint so that the gradient is non-zero? Should I switch to a non-Lagrange multiplier method? If such a method exists, what are they called? Any help would be appreciated.