Understanding Stationary Points in Multivariable Functions

[unscientific]

Homework Statement

The problem as attached in the picture.

The Attempt at a Solution

So here's the problem that I was facing:

G_(t) has stationary points at t = 0, t = -1/2, t = -4

They correspond to (0,0), (a/4, -a) and (16a,-8a) in the function G_(t).

However, only (0,0) is a suitable stationary point in F(x,y).

Here are some of my conclusions:

Before the "condition" x = at², y = 2at was applied, (x,y) was free to have any possible value.

However, with the restriction implemented, again let's imagine F(x,y) as a terrain with many mountains, pits and saddles. By implementing this restriction we only see what lies along the "track" of h(x,y), taking the highest mountain and the lowest pit as the maximum and minimum. But this is not the case, as in the whole map of F(x,y) there may very well be even higher mountains and even deeper pits that just lie outside the track..

 

[unscientific]

Part 2: Lagrange multipliers

I attempted to use Lagrange multipliers to solve the problem above:

The problem is summed up as such:

F_(x,y) = x² + y² +3xy
subject to the condition h_(x,y) = y² -4ax = 0

dF = (2x+3y) dx + (2y + 3x) dy = 0

λdh = (-4a)λ dx + (2y)λ dy = 0d(F-λdh) = (2x + 3y + 4aλ) dx + (3xλ) dx = 0

Which reduces to these 3 equations:

y² - 4ax = 0 -----------(1)

3xλ = 0 ---------------------------(2)

2x + 3y + 4aλ = 0 -----------(3)

Solving (1) and (2) together, you get x = 0, y = 0. But plugging them into (3), you get λ = 0 which you sub back in, you get (x=0,y=0) or the solution (y=-4a), which is wrong since it implies x = 4a.

 

[unscientific]

But come to think about it, is any stationary point along "track" h(x,y) also a stationary point in F(x,y).

I know it may not be the highest mountain (maximum) or the deepest pit (minimum), but we have to agree that if a stationary point exists on "track" h(x,y), it got to exist on terrain F(x,y) right?

 

[Ray Vickson]

But come to think about it, is any stationary point along "track" h(x,y) also a stationary point in F(x,y).

I know it may not be the highest mountain (maximum) or the deepest pit (minimum), but we have to agree that if a stationary point exists on "track" h(x,y), it got to exist on terrain F(x,y) right?

Absolutely NOT! in the problem max/min f(x,y), subject to g(x,y) = c, the solution usually is at a NON-stationary point of f(x,y) in the 2-dimensional plane. In fact, f may not have any stationary points at all. For example, max or min (x+y), subject to x^2+y^2=2 has the obvious (and easily found) solutions: max at (1,1), min at (-1,-1). However, f(x,y) = x+y has no stationary points. For another example: min f(x,y) = (x-2)^2+(y-2)^2, subject to g(x,y) = x+y=1. The solution is at (x,y) = (1/2,1/2), but the only stationary point of f is at (2,2).

Along the curve g(x,y) = c we have, at (x0,y0) on the curve, that the "differentials" must satisfy dg = g_x dx + g_y dy = 0, where g_x = \partial f/\partial x|_{x=x_0,y=y_0}, etc. The the "differential" of f is df = f_x dx + f_y dy. If, for example, g_x \neq 0 we can solve dg = 0 to find dx: dx = -g_y/g_x \; dy, so along the curve we have df = f_x dx + f_y dy = \left( f_y - \frac{g_y}{g_x}f_x \right) dy. So, we need f_y - \frac{g_y}{g_x} f_x = 0.
Note that if we write \lambda = f_x/g_x, the condition df=0 becomes f_y - \lambda g_y = 0. This is just the Lagrange equation in the y-direction. Note also that the defintion of \lambda can be written as f_x - \lambda g_x = 0, which is the other Lagrange condition!

RGV

 

[unscientific]

Absolutely NOT! in the problem max/min f(x,y), subject to g(x,y) = c, the solution usually is at a NON-stationary point of f(x,y) in the 2-dimensional plane. In fact, f may not have any stationary points at all. For example, max or min (x+y), subject to x^2+y^2=2 has the obvious (and easily found) solutions: max at (1,1), min at (-1,-1). However, f(x,y) = x+y has no stationary points. For another example: min f(x,y) = (x-2)^2+(y-2)^2, subject to g(x,y) = x+y=1. The solution is at (x,y) = (1/2,1/2), but the only stationary point of f is at (2,2).

Along the curve g(x,y) = c we have, at (x0,y0) on the curve, that the "differentials" must satisfy dg = g_x dx + g_y dy = 0, where g_x = \partial f/\partial x|_{x=x_0,y=y_0}, etc. The the "differential" of f is df = f_x dx + f_y dy. If, for example, g_x \neq 0 we can solve dg = 0 to find dx: dx = -g_y/g_x \; dy, so along the curve we have df = f_x dx + f_y dy = \left( f_y - \frac{g_y}{g_x}f_x \right) dy. So, we need f_y - \frac{g_y}{g_x} f_x = 0.
Note that if we write \lambda = f_x/g_x, the condition df=0 becomes f_y - \lambda g_y = 0. This is just the Lagrange equation in the y-direction. Note also that the defintion of \lambda can be written as f_x - \lambda g_x = 0, which is the other Lagrange condition!

RGV

Thank you for the very thorough exaplanation with examples! However, using the "terrain and track" analogy, it seems rather intuitive to me that any maximum/minimum on g(x,y) must also lie on f(x,y)...

Any way to explain why it's not this case rather than using counter-examples?

 

[Ray Vickson]

You say " it seems rather intuitive to me that any maximum/minimum on g(x,y) must also lie on f(x,y)...", so I can't tell whether you are trying to maximize or minimize f or g. If your problem is max or min f(x,y), subject to g(x,y) = c, then, of course, we don't look for any max or min of g(x,y)---we are only interested in those points where g(x,y) = c, but if you are trying to maximize or minimize g(x,y), subject to f(x,y) = constant, then I have no idea what you are talking about. I don't know what you could possibly mean when you say "must also lie on f(x,y)". What does "lie on f(x,y)" mean? My previous response assumed (maybe incorrectly) that you thought a constrained optimum must also be a stationary point in the unconstrained problem, and that is not true. I can't figure out what you think IS true.

RGV

 

[unscientific]

Sorry for the confusion... what I meant was given a question to maximize f(x,y) given the constraint g(x,y) = c.

Treating f(x,y) as a height as a function of x,y for a terrain, won't every point on g(x,y) also be a point in f(x,y)? Given that f(x,y) alone is true for all x and for all y, so any point on g(x,y) be also a point in f(x,y)?

Therefore, any maximum detected in g(x,y) would also be a maximum on f(x,y) given the constraint.

I'm using the "terrain and track" analogy, where f(x,y) is the terrain and g(x,y) = c is a track that only can be taken.

I've attached a picture to help my explanation..

 

[HallsofIvy]

Thank you for the very thorough exaplanation with examples! However, using the "terrain and track" analogy, it seems rather intuitive to me that any maximum/minimum on g(x,y) must also lie on f(x,y)...

Any way to explain why it's not this case rather than using counter-examples?

Then you are thinking wrong. The (x, y, x) 'terrain' defined by z= f(x, y) is NOT the same as the 'terrain' defined by z= g(x,y). Of course, you can take the same x and y, but then z is different. In particular, a max or min for f(x,y) may not be as max or min for g(x,y).

 

[unscientific]

Then you are thinking wrong. The (x, y, x) 'terrain' defined by z= f(x, y) is NOT the same as the 'terrain' defined by z= g(x,y). Of course, you can take the same x and y, but then z is different. In particular, a max or min for f(x,y) may not be as max or min for g(x,y).

Yes I understand what you mean; you're saying even though in the terrain of g(x,y) a minimum appears at say (1,1) but this doesn't mean that this is a minimum at f(x,y) as you don't get the complete picture by just looking at g(x,y). Like, it may be possible that just with a small Δx you may find that the height goes deeper.

Secondly, for a constraint of g(x,y) = c, won't this imply that this is simply a flat terrain with a constant height of c?

 

[Ray Vickson]

Yes I understand what you mean; you're saying even though in the terrain of g(x,y) a minimum appears at say (1,1) but this doesn't mean that this is a minimum at f(x,y) as you don't get the complete picture by just looking at g(x,y). Like, it may be possible that just with a small Δx you may find that the height goes deeper.

Secondly, for a constraint of g(x,y) = c, won't this imply that this is simply a flat terrain with a constant height of c?

I think I might finally understand what you are attempting to say. Let me give it a try. You have a curve C in the XY-plane; it happens to be defined by an equation of the form g(x,y) = c (so is on a horizontal contour of the surface z= g(x,y)). Think of the curve C as a trail marked on a contour map of a region, but as viewed from above. The point (x,y) on C essentially gives you the latitude and longitude of a point on the trail, but the altitude of that point lies on the surface z = f(x,y). Viewed from above, the curve C might be a straight line (corresponding to a linear function g), but in 3 dimensions the trail is not straight: it goes up and down according to the local height of the land. Maybe that is what you were trying to say (but not in enough detail to be understandable by others), and if so, it is, of course, absolutely true, but not terribly useful: it does not really help you to solve the problem.

RGV

 

[unscientific]

I think I have better understood the problem now so let me conclude the points:

1. if h = f(x,y) is defined to be the height of a terrain, g(x,y) = c does not imply a flat plane of height c, since g(x,y) ≠ f(x,y) ≠ h.

2. Rather, g(x,y) = c tells you where you can go and where you cannot go by fixing a constraint between x and y, for example x + y = 1 or y² = 4ax.

3. This sort of determines the "path" or "trail" you can take along f(x,y) which at any point would satisfy the constraint and give h = f(x,y).

4. However, any maxima/minima along the trail does NOT imply that it is a maxima/minima of f(x,y). Think of a global minima in f(x,y) where the trail crosses the sloping-down-side but does not cross the bottom of the minima. It appears as a minima in g(x,y) but we know that it is not "yet" the minima in f(x,y).

 

[Ray Vickson]

I think I have better understood the problem now so let me conclude the points:

1. if h = f(x,y) is defined to be the height of a terrain, g(x,y) = c does not imply a flat plane of height c, since g(x,y) ≠ f(x,y) ≠ h.

2. Rather, g(x,y) = c tells you where you can go and where you cannot go by fixing a constraint between x and y, for example x + y = 1 or y² = 4ax.

3. This sort of determines the "path" or "trail" you can take along f(x,y) which at any point would satisfy the constraint and give h = f(x,y).

4. However, any maxima/minima along the trail does NOT imply that it is a maxima/minima of f(x,y). Think of a global minima in f(x,y) where the trail crosses the sloping-down-side but does not cross the bottom of the minima. It appears as a minima in g(x,y) but we know that it is not "yet" the minima in f(x,y).

No, No , No! Minima, or maxima, or anything else about g(x,y) are 100% irrelevant! The constraint g(x,y) = c only tells you what (x,y) values you are allowed to use. Any minima or maxima of g (if they even exist---which they did NOT in one of my previous examples) have nothing to do with the problem. Forevermore, *forget about maxima or minimal in g*; it is f you need to examine (but of course, the solution may not be a max of f either---but at least looking at f is looking at the correct function).

RGV

 

[unscientific]

No, No , No! Minima, or maxima, or anything else about g(x,y) are 100% irrelevant! The constraint g(x,y) = c only tells you what (x,y) values you are allowed to use. Any minima or maxima of g (if they even exist---which they did NOT in one of my previous examples) have nothing to do with the problem. Forevermore, *forget about maxima or minimal in g*; it is f you need to examine (but of course, the solution may not be a max of f either---but at least looking at f is looking at the correct function).

RGV

I see. Is it because by the constraint g(x,y) -c = 0 it only tells you what 'x' and what 'y' you are allowed to use and by that feature any maxima or minima of g(x,y) - c is due to the individual mechanics of g(x,y) - c not related to f(x,y) at all?