Problem understading lagrange multipliers

[Narcol2000]

I'm trying to follow the idea behind Lagrange multipliers as given in the following wikipedia link.

http://en.wikipedia.org/wiki/Lagrange_multipliers

I follow the article right up until the point where it goes:

'To incorporate these conditions into one equation, we introduce an auxiliary function:'
*<insert formula seemingly plucked from random here>*

There doesn't seem to be any explanation or reason why this formula was chosen, and the 'justification' section just repeats what the constraints are, but still doesn't say where the formula comes from...

any ideas?

 

[atyy]

The true way is given in many books.
The quick and dirty way:

Thing to minimize: f(x,y)
Constraint equation: g(x,y)=A

We know we need to get the constraint equation into the thing to minimize, so we rearrange it to 0=A-g(x,y). We can always add zero to anything, so the thing to minimize becomes:

Thing to minimize taking constraint into account: f(x,y)+lamda(A-g(x,y))

where lamda is the Lagrange multiplier which you use to enforce the constraint after minimization.

 

[Narcol2000]

Thanks atyy, makes sense.
:)

 

[HallsofIvy]

Here's another way of thinking about it. Suppose you wanted to find a local maximum for f(x,y,z). One way to do that is to pick some starting point, find [itexs]\nabla f[/itex] at that point and "follow" that vector a short distance. Because $\nabla f$ points in the direction of fastest increase, that should take you closer to the maximum point (a short distance) because you don't want to overshoot). You can keep doing that as long as $\nabla f$ is not the 0 vector. That shows that a maximum (or minimum) must occur where $\nabla f= 0$.

Now suppose you are constrained to stay on the surface g(x,y,z)= constant. Now you can't "follow" the vector- it might point off the surface. But you can look at the projection of $\nabla f$ in the surface and get closer to the maximum value by going that way. You can do that until the projection of $\nabla f$ in the surface is 0- which is the same as saying that $\nabla f$ is perpendicular to the surface. Of course, $\nabla g$ is also perpendicular to the surface g(x,y,z)= constant. That means that a maximum (or minimum) will occur where $\nabla f$ and $\nabla g$ are parallel. And that is the same as saying one is a multiple of the other: $\nabla f= \lambda\nabla g$.