Derivation of Lagrange Multipliers Method

[Saladsamurai]

Hey folks. I have some more or less qualitative questions regarding optimization problems via Lagrange multipliers. I am following the http://en.wikipedia.org/wiki/Lagrange_multipliers" on this one and I am just a little confused by their wording.

In the first section titled "Introduction," they have the following:

Consider the two-dimensional problem introduced above:
maximize f(x,y)
subject to g(x,y) = c

We can visualize contours of f given by

f(x,y) = d

for various values of d, and the contour of g given by g(x,y) = c.
Suppose we walk along the contour line with g = c. In general the contour lines of f and g may be distinct, so following the contour line for g = c one could intersect with or cross the contour lines of f.

1) My first question (which might seem silly) is: What is g(x,y) ? Is it a line? Or a surface? It looks like a line in their picture but I am used to functions of the forms g(x,y) being representative of surfaces; though I feel like it could be ambiguous.

2) Second, when they say: "Suppose we walk along the contour line with g = c." They don't mean that g = c IS the contour line right? They are saying that AT g(x,y) one can follow a contour line. It just seems like if g(x,y) = c is not a surface, then the whole idea of a contour is a little weird to me. So perhaps if someone can answer my first question this would make more sense to me.


Sorry if these seem stupid, but it has been awhile.

 

[LCKurtz]

Look at reply #2 in the thread

https://www.physicsforums.com/showthread.php?t=394100

Work through the example I suggest there and see if it answers your question.

 

[Saladsamurai]

Look at reply #2 in the thread

https://www.physicsforums.com/showthread.php?t=394100

Work through the example I suggest there and see if it answers your question.

I will answer your first question.

This example might help you see that. Suppose you have a circular wire in the shape of the unit circle x²+y²=1. Suppose the temperature at a point (x,y) in the plane is given by T = f(x,y) = y + 2x, and you want to know what is the warmest point on the wire. Now make a sketch showing the circle and several of the level curves where

T = y + 2x = c

for various values of c. Plot them for at least c = 0, 1/2, 1, 3/2, 2. They will be parallel lines cutting through or near your wire, each of on which the temperature is constant. With this picture you will be able to estimate the max temperature on the wire although the level curve giving it may not have been drawn. But you should be able to see that the level curve that does the trick will be tangent to the circle. It has to at least touch the circle, and if it crosses the circle you can back away some to get a higher temperature. This means that the normal to the circle and the level curve must be parallel at the point giving max temperature, so their normals are proportional. That gives the gradient condition. The same idea works in 3D. Your text likely gives a mathematical argument to buttress this hueristic argument.

Hey LCKurtz

I am sketching this right now. Sketching level sets was never really my strong suit.
Is it as simple as letting c take on a certain value and then plotting that line? For example, let c = 1/2 and then plot the line y = 1/2 - 2x ? I am going to assume that the answer is yes and proceed.

And for this example, I assume that we are maximizing 'T' subject to the constraint that the point falls on the wire.

EDIT: Okay. I have plotted the circle and the line y = c - 2x for various values of c. I can see that 'T' would be maximized (and minimized) when the line y = c - 2x is tangent to the circle. I am having a couple of issues with this now.

1) Not sure how to generalize this idea beyond the simple circle constraint. That is if both my maximizing function f and my constraining function g are arbitrarily shaped surfaces, will the condition that g is 'tangent to' f be sufficient to say that f is at a maximum there?

2) I am not sure how to word this without sounding like an idiot but here goes anyway: The function T = y + 2x = c that you gave, is that a surface? That is, isn't it T(x,y) = y + 2x - c ? Or is it just T(x,y) = y + 2x ? I don't know why this is so confusing right now

 

[vela]

1) Not sure how to generalize this idea beyond the simple circle constraint. That is if both my maximizing function f and my constraining function g are arbitrarily shaped surfaces, will the condition that g is 'tangent to' f be sufficient to say that f is at a maximum there?

It could be a minimum as well.

2) I am not sure how to word this without sounding like an idiot but here goes anyway: The function T = y + 2x = c that you gave, is that a surface? That is, isn't it T(x,y) = y + 2x - c ? Or is it just T(x,y) = y + 2x ? I don't know why this is so confusing right now

T(x,y)=y+2x describes a surface. Think of T as the z coordinate. For each point in the xy plane, there's an associated temperature which you can think of as the height of the surface above the point (x,y). When you choose a specific value of T, you're looking at the intersection of that surface with the plane T=c.

 

[LCKurtz]

Hey LCKurtz

I am sketching this right now. Sketching level sets was never really my strong suit.
Is it as simple as letting c take on a certain value and then plotting that line? For example, let c = 1/2 and then plot the line y = 1/2 - 2x ? I am going to assume that the answer is yes and proceed.

Yes

And for this example, I assume that we are maximizing 'T' subject to the constraint that the point falls on the wire.

EDIT: Okay. I have plotted the circle and the line y = c - 2x for various values of c. I can see that 'T' would be maximized (and minimized) when the line y = c - 2x is tangent to the circle. I am having a couple of issues with this now.

1) Not sure how to generalize this idea beyond the simple circle constraint. That is if both my maximizing function f and my constraining function g are arbitrarily shaped surfaces, will the condition that g is 'tangent to' f be sufficient to say that f is at a maximum there?

2) I am not sure how to word this without sounding like an idiot but here goes anyway: The function T = y + 2x = c that you gave, is that a surface? That is, isn't it T(x,y) = y + 2x - c ? Or is it just T(x,y) = y + 2x ? I don't know why this is so confusing right now

This is a 2D example.

T = y+2x is the temperature at a point (x,y) in the plane. I don't find it helpful to interpret it as a surface although it can be thought of that way. The level curves in this example are straight lines in the plane on which the temperature is constant.: y+2x = c. It wouldn't matter if the level curves had other shapes. Try drawing a family of hyperbolas. As long as everything is smooth the level curves will be tangent at the extremes.

 

[Saladsamurai]

For anyone who is curious, I plotted T(x,y) = y + 2x and g(x,y) = x^2 + y^2 along with the plane z(x,y) = 1. The intersection of z(x,y,) = 1 with g(x,y) = x^2 + y^2 gives the level set g(x,y) = x^2 + y^2 = 1. Here are a couple of views:

and another view

And and one more

 

[LCKurtz]

Pretty pictures, but I don't think they add to the understanding of what is going on. The example problem is given temperature T(x,y) = x+2y in the plane, find the hottest point on the wire represented by the unit circle x²+y² = 1. The following plot shows the level curves for T varying from -3 to 3 including, in red, T =sqrt(5) which obviously gives the max.

Maple doesn't label the contours, but in this example the y intercepts of the contour lines give T for the lines.

 

[Saladsamurai]

Different people learn in different ways I guess The picture that you drew is indeed the exact same picture that I drew originally, but I found that it did not add to my understanding of the problem.

For me the problem was that T(x,y) is in fact a surface, whether "it helps to interpret it that way" or not. And the constraint, though in this case is a simple circle, is actually a level set on a surface. So for me, being able to see that for the constraint
g(x,y) = x^2 + y^2 = anything,
T(x,y) is maximal when the level set of g(x,y) is tangent to T(x,y).

Thanks for the help again!