Understanding Lagrange Multipliers and Constraint Equations

[Skrew]

I have been reading about Lagrange Multipliers, my book along with wiki and other resources I have read use an intuitive argument on why the max/min contour lines end up tangent to the constraint equation.
I don't really understand it, especially considering the obvious flaw as shown by the below,

F(x,y) = sin(x*y),

subject to, x^2 + y^2 = 6

The picture is the graph of the contour lines of F(x,y) along with the constraint.

The max and min values of F(x,y) are obviously not at the places the contour lines are tangent to the constraint.

So I was wondering if anyone could explain what's going?

 

[HallsofIvy]

? The maxima and minima of F(x,y), subject to the contstraint $x^2+ y^2= 6$ are $(\sqrt{3}, \sqrt{3})$, $(-\sqrt{3}, -\sqrt{3})$ (the maxima), $(\sqrt{3}, -\sqrt{3})$, and $(-\sqrt{3}, \sqrt{3})$ (the minima) obviously are where the level curves of F(x, y), specifically, sin(xy)= sin(3) and sin(xy)= -sin(3), are tangent to the graph of $x^2+ y^2= 6$.

I suspect that your graph just doesn't show the specific level curve sin(xy)= sin(3) or sin(xy)=-sin(3) which are the same as the graphs of xy= 3 and xy= -3. What sort of iteration did you use to graph the level curves?

 

[Skrew]

Uh point (2, sqrt(2)) produces a bigger maxima.

 

[arildno]

Both of you seem to have forgotten that the system:
$$\cos(xy)=0(1)$$
$$x^{2}+y^{2}=6(2)$$ can have real solutions.

The first is equivalent of saying:
$$xy=(2n+1)\frac{\pi}{2}$$ for some integer n

We may convert that system of equation into, for example:
$$x^{2}+y^{2}=6$$
$$(y-x)^{2}=\pm\sqrt{6-(2n+1)\pi}$$

Or,
$$y=x+\pm\sqrt{6-(2n+1)\pi}$$

Inserting in (2), we get:
$$2x^{2}\pm{2x}\sqrt{6-(2n+1)\pi}+6-(2n+1)\pi=6$$
which can be simplified as:
$$(x\pm\frac{\sqrt{6-(2n+1)\pi}}{2})^{2}=\frac{(2n+1)\pi}{2}+\frac{6-(2n+1)\pi}{4}$$
which simplifies to:
$$(x\pm\frac{\sqrt{6-(2n+1)\pi}}{2})^{2}=\frac{6+(2n+1)\pi}{4}$$
Taking the squar root, remembering that the "plus-minuses" may vary independently of each other, we get:
$$x=\pm\frac{\sqrt{6-(2n+1)\pi}}{2}\pm\frac{\sqrt{6+(2n+1)\pi}}{2}$$

Thus, we see that only n=0 and n=-1 are acceptable values.

Thus, we have:
$$x=\pm\frac{\sqrt{6+\pi}}{2}\pm\frac{\sqrt{6-\pi}}{2}$$

This can be simplified as:
$$x=\pm(\frac{\sqrt{6+\pi}}{2}+\frac{\sqrt{6-\pi}}{2}$$
OR
$$x=\pm(\frac{\sqrt{6+\pi}}{2}-\frac{\sqrt{6-\pi}}{2}$$
For each of these 4 solutions, y can be either of the 2 from the OTHER set, so that we have 8 solutions all in all.

 

[Skrew]

arildno, you can also easily see by the contour plot that there are 4 mins and 4 maxes, my example of (2, sqrt(2)) wasn't to indicate it was even a max or min, only to show that HallsofIvy was incorrect.

A more simple example showing the intuitive argument for the use of Lagrange multipliers is wrong can be seen through.

F(x,y) = sin(x) + 0y,

x^2+y^2 = 6

The highest level curve cuts through the constraints graph, it isn't tangent to it.

The intuitive argument works if the function is increaseing or decreasing throughout the "region" of the constraint but not otherwise, not if there is a max or min level curve on the constraints graph(which cuts through it).

 

[Mute]

Uh point (2, sqrt(2)) produces a bigger maxima.

No, it doesn't. $\sin(2\sqrt{2}) \approx 0.049$,while $\sin(\sqrt{3}\sqrt{3}) \approx 0.05233$.

Both of you seem to have forgotten that the system:
$$\cos(xy)=0(1)$$
$$x^{2}+y^{2}=6(2)$$ can have real solutions.

...

Uh, the problem isn't about that system of equations, it's a Lagrange multiplier problem, i.e., minimize the function F(x,y) = sin(xy) subject to the constraint c(x,y) = x^2 +y^2 - 6, which amounts to minimizing

$$G(x,y) = F(x,y) - \lambda c(x,y)$$

I'm not sure where your system of equations comes from, because the minimization procedure yields

$$\frac{\partial G}{\partial x} = y\cos(xy) - 2\lambda x = 0$$
$$\frac{\partial G}{\partial y} = x\cos(xy) - 2\lambda y = 0$$
which gives x^2 = y^2 when lambda isn't zero, which gives the $\pm \sqrt{3}$ solutions.

 

[Hurkyl]

which gives x^2 = y^2 when lambda isn't zero, which gives the $\pm \sqrt{3}$ solutions.

What about the solutions corresponding to when lambda is zero?

So I was wondering if anyone could explain what's going?

You're running into an issue of degeneracy. Roughly speaking, at the maximum, every unit vector points in a direction in which the objective function remains constant.

 

[arildno]

No, it doesn't. $\sin(2\sqrt{2}) \approx 0.049$,while $\sin(\sqrt{3}\sqrt{3}) \approx 0.05233$.



Uh, the problem isn't about that system of equations, it's a Lagrange multiplier problem, i.e., minimize the function F(x,y) = sin(xy) subject to the constraint c(x,y) = x^2 +y^2 - 6, which amounts to minimizing

$$G(x,y) = F(x,y) - \lambda c(x,y)$$

I'm not sure where your system of equations comes from, because the minimization procedure yields

$$\frac{\partial G}{\partial x} = y\cos(xy) - 2\lambda x = 0$$
$$\frac{\partial G}{\partial y} = x\cos(xy) - 2\lambda y = 0$$
which gives x^2 = y^2 when lambda isn't zero, which gives the $\pm \sqrt{3}$ solutions.

And why cannot lambda be zero?

The case of lambda=0 just means that the max/min under the constraint happens to be max/min WITHOUT the constraint as well.

Which is, in fact, the case here.

 

[Skrew]

No, it doesn't. $\sin(2\sqrt{2}) \approx 0.049$,while $\sin(\sqrt{3}\sqrt{3}) \approx 0.05233$.

I am using radians, sin(2*sqrt(2)) > sin(3).

You're running into an issue of degeneracy. Roughly speaking, at the maximum, every unit vector points in a direction in which the objective function remains constant.

I don't understand your explanation.

 

[Hurkyl]

I don't understand your explanation.

If you're at the maximum, then no matter what direction you point, the graph of the objective function is level in that direction.

In particular, this implies that the tangent line to the constraint function points in a direction in which the objective function is neither increasing nor decreasing. It just so happens that, in the degenerate case, there are lots of other directions with that property as well.

(Why is it degenerate? Well, for most points, all but 2 unit tangent vectors will be pointing in a direction where the function is either increasing or decreasing)


That the maximum is not isolated -- there is an entire curve of points that are maxima -- is another source of degeneracy. You probably wouldn't have noticed the first kind of degeneracy if it weren't also for the second kind of degeneracy to make it graphically obvious.

(Why is it degenerate? The maxima occurs inside the solution set to two equations in two variables -- usually such sets are a discrete set of points)

 

[Mute]

I am using radians, sin(2*sqrt(2)) > sin(3).

I thought I was using radians too! Turns out the calculator in Linux was in degree mode! My bad.

And why cannot lambda be zero?

The case of lambda=0 just means that the max/min under the constraint happens to be max/min WITHOUT the constraint as well.

Which is, in fact, the case here.

It was an oversight on my part to not check what system of equations lambda = 0 gave.