Where should I begin to eventually understand calculus of variations?

[V0ODO0CH1LD]

I am a engineering undergraduate. And my classical mechanics module was all based on Newtonian mechanics, but I got very curious about the hamiltonian and lagrangian formulations and decided to read up on those.

When I got to the principle of least action I couldn't understand much, mostly because I had never previously heard about calculus of variations. Assuming my mathematical background only includes high school stuff, "regular" calculus, vector calculus and linear algebra. Where should I begin to understand all about functionals and eventually get to minimize them using calculus of variations?

I tried looking into functional analysis, but it assumes topology is already known. Then I said "okay! let's learn topology", but it is assumes group theory is already known.. I looked a bit into group theory and it looks like the only prerequisite is linear algebra, mostly to generate examples. And I already know linear algebra!

So is that it? Can I begin my journey at group theory and gradually move up to functional analysis? Or is there something else I should already know..

Just an additional question: which field of mathematics looks into metric spaces? Because the book I bought on functional analysis talks about complete metric spaces in terms of metric spaces, and I don't know what those are..

 

[AlephZero]

You may have picked the wrong sort of book to get started. What you are describing is a rigorous approach to the subject and with the level of generality that is today's "state of the art" in math and theoretical physics.

On the other hand, if you went back to a textbook on "math for physics and engineering" written in the 1950s, you would find descriptions of calculus of variations that you could probably understand with a good background in calculus, and without any mention at all of metric spaces, group theory, etc.

Maybe that type of book still exists - but I learned this stuff so long ago I can't suggest any specific books to try.

(I'm not complaining about the progress of teaching mathematical generality and rigor here - but it doesn't help the OP much, unfortunately).

 

[V0ODO0CH1LD]

So what you mean is that I can actually learn calculus of variations without knowing what a functional is? I just ask that because taking derivatives with respect to variables that don't exist is a very foreign concept to me..

 

[ZombieFeynman]

I suggest looking at chapter 6 of Taylors classical mechanics book for a superficial overview of the subject, then trying some problems. I think this will give you what you seek.

 

[ZombieFeynman]

A functional is just a function which maps a space of functions to a space of real number (mathematicians will hopefully forgive my simplemindedness). A definite integral is a functional. It takes a function between two bounds and returns the area under that function.

 

[mathwonk]

To a mathematician (i.e. me), a functional is just a real valued function, i.e. a function with values in R, as distinguished from a function whose values may be vectors or some higher dimensional object.

Richard Courant has a famous 2 volume book on calculus and there is a short chapter in the second volume introducing the calculus of variations. I have not read it but my rough recollection from many years ago is that this name simply refers to differential calculus done on an infinite dimensional space, such as a space of functions.

Thus a basic problem would be to let the space be the space of all smooth curves joining two points of the plane and consider the length function on each curve, and look for the shortest curve joining the two points. This is a minimization problem, so should be approachable by taking the derivative of the length function or "functional" and setting it equal to zero. Hopefully one learns that the derivative is zero on a straight line joining the two points.

Another famous example is the "brachistochrone" problem of minimizing the time needed for a particle to slide down a curved path, joining two points, under the influence of gravity.

I believe this type of problem was solved by Euler. So if I myself were to begin the study of calculus of variations I would start by reading Courant's short chapter on the topic with its discussion of the basic differential equation of Euler, in Courant's calculus book, vol. 2.

There is also a short introduction to the subject on pages 182-186 of the advanced calculus book of Loomis and Sternberg, which used to be available free on the web, perhaps at Sternberg's page. It is short but Loomis makes some helpful remarks there about the special nature of the subject, and solves one of the typical problems.

A fuller treatment apparently occurs in chapter IV of vol.1 (230 pages), of the book on Methods of mathematical physics by Courant-Hilbert.

 

[AlephZero]

So what you mean is that I can actually learn calculus of variations without knowing what a functional is? I just ask that because taking derivatives with respect to variables that don't exist is a very foreign concept to me..

Sure. You will have to learn a bit about how functionals "behave", but you don't need to know what they are called. As Mathwonk said, the classic problems can be stated as "find the function that satisfies some condition" - usually a minimum or maximum condition, like "show that the curve with shortest length that enclosing a given (fixed) area is a circle".

For the way Euler, Bernouilli, Lagrange, etc solved these problems, the key step in the argument is usually of the form:

If $\int_a^b f(x)g(x)\,dx$ has some property for every possible function $g(x)$, that tells you something interesting about $f(x)$.

If you know about "virtual work" from your mechanics courses, you should see this is a very similar idea (think of $g(x)$ as an "arbitrary small displacement").

 

[WannabeNewton]

I second the Taylor recommendation if this is a first exposure but also check out the book by Calkin called "Lagrangian and Hamiltonian Mechanics". You are jumping ahead if you want to learn functional analysis (which is not exactly what I think you think it means) and topology (which doesn't require any group theory to learn). Metric spaces are covered in real analysis courses.

 

[LeonhardEuler]

I was exactly in the same situation as you a few years ago. I read "Classical Mechanics" by Goldstein. It has a good introduction to the math behind the Hamiltonian and Lagrangian formulations. I also read Wikipedia. Between those sources, I got a decent understanding of the topics.

Later I also read "An Introduction to the Calculus of Variations" by L. A. Pars. That was a good book. But for me at least, it was better to have something concrete in mind when I first learned the subject, so I was better off starting with Goldstein.

I learned the topic purely out of curiosity, but it has been useful to me too in my research. It is an interesting topic and I hope you enjoy learning it!

 

[V0ODO0CH1LD]

okay, I have one question that might give the insight necessary for me to go on with the least action principle, at least for a while. If I have a function $f$ of both $x$ and $y$, where $y$ is also a function of $x$, and I take its partial derivative with respect to $x$ it going to look like:

\frac{df}{dx}=\frac{\partial{f}}{\partial{x}}+\frac{\partial{f}}{\partial{y}}\frac{dy}{dx}

But the thing with calculus of variations is that $y$ actually is the derivative of $x$ with respect to some parameter $t$ and I also to find the stationary points of $f$ with respect to the function of $t$ I am trying to find, in this case $x$. Right?

So that equation is going to look like:

0=\frac{\partial{f}}{\partial{x}}+\frac{\partial{f}}{\partial{\dot{x}}}\frac{d\dot{x}}{dx}

Is that what I am trying to find the stationary point of? Or is it the integral of that with respect to $t$? In that case; how do I get it to look like eulers equation*?

*0=\frac{\partial{f}}{\partial{x}}-\frac{d}{dt}\frac{\partial{f}}{\partial{\dot{x}}}

 

[LeonhardEuler]

okay, I have one question that might give the insight necessary for me to go on with the least action principle, at least for a while. If I have a function $f$ of both $x$ and $y$, where $y$ is also a function of $x$, and I take its partial derivative with respect to $x$ it going to look like:

\frac{df}{dx}=\frac{\partial{f}}{\partial{x}}+\frac{\partial{f}}{\partial{y}}\frac{dy}{dx}

But the thing with calculus of variations is that $y$ actually is the derivative of $x$ with respect to some parameter $t$ and I also to find the stationary points of $f$ with respect to the function of $t$ I am trying to find, in this case $x$. Right?

So that equation is going to look like:

0=\frac{\partial{f}}{\partial{x}}+\frac{\partial{f}}{\partial{\dot{x}}}\frac{d\dot{x}}{dx}

Is that what I am trying to find the stationary point of? Or is it the integral of that with respect to $t$? In that case; how do I get it to look like eulers equation*?

*0=\frac{\partial{f}}{\partial{x}}-\frac{d}{dt}\frac{\partial{f}}{\partial{\dot{x}}}

The problem you set up is not one that the Euler Lagrange equation applies to.

In your set up, f is a function of two variables, and you subject the variables to an additional constraint, which is that one is a function of some variable t, and the other is the derivative of it. The maximization you are setting up is finding the particular values of the two inputs that maximize f subject to this constraint. This is an ordinary constrained maximization problem, and the solution is a pair of numbers, not a function.

The Euler Lagrange equation refers to a specific set up. The functional that maps functions to real numbers is of the form
I(x) = \int_{a}^{b}L(x,\dot{x})dt
And the Euler-Lagrange equation for this is
\frac{\partial L}{\partial x}=-\frac{d}{dt}\frac{\partial L}{\partial \dot{x}}
Notice that the Euler-Lagrange equation specifically deals with the integrand L, which was not present in your example.

 

[V0ODO0CH1LD]

So the functional is the definite integral $\int_a^bdt$ of a function $L$ of another function $x$ and its derivative with respect to some parameter $t$, that is $\dot{x}$? And that is defined as a functional because it takes a function and maps it to the real numbers? What I still don't get is that to minimize a function using its derivative, even if that function is a function of another function (a functional), the derivative has to be taken with respect to a variable. But what is that variable? What is the derivative of $I(x)$ being taken with respect to? $x$?

 

[LeonhardEuler]

So the functional is the definite integral $\int_a^bdt$ of a function $L$ of another function $x$ and its derivative with respect to some parameter $t$, that is $\dot{x}$? And that is defined as a functional because it takes a function and maps it to the real numbers?

Yes. And the function x(t) is unknown. The solution to the problem is the entire function x(t), not some number.

What I still don't get is that to minimize a function using its derivative, even if that function is a function of another function (a functional), the derivative has to be taken with respect to a variable. But what is that variable? What is the derivative of $I(x)$ being taken with respect to? $x$?

That is the difference between the calculus of variations and a regular minimization problem. The idea to take the derivative and set it to zero to minimize a function is that the function is neither increasing nor decreasing an an extreme point. You test for this with the derivative, which tells you how the function changes in response to an increase or decrease in its argument.

This same logic does not carry over completely to variational problems. Now the functional takes a function as an argument. With a function of a real number, there are only two ways to make a small change in the argument: increase it a little, or decrease it a little. Not so for a functional. There are an infinite number of ways to make a small change to the function you input. Where the logic is the same, though, is that at an extreme point, the functional must be stationary for any small change in the argument.

The idea of the proof of the Euler Lagrange equation is this: We have some functional I, and some function f, and we want to see if f makes I stationary. We introduce a perturbation function, \eta(t), which we can make arbitrarily small by multiplying by a small real number \epsilon. Then we look at
I(f+\epsilon\eta(t))=\int_{a}^{b}L(f+\epsilon\eta(t),\dot{f}+\epsilon \dot{\eta}(t))dt
Since the limits are fixed, we also require \eta(a)=\eta(b)=0. We can take the derivative now with respect to \epsilon and set that to 0. That is
\frac{d}{d\epsilon}I(f+\epsilon\eta(t))|_{\epsilon=0}=0
We need the derivative to be zero for all functions \eta(t) subject to the constraint I mentioned, and also usually we would restrict ourselves to looking at functions with continuous first derivatives.

You can look here:
http://en.wikipedia.org/wiki/Euler–Lagrange_equation
to see how the rest of the proof goes (it is in a section that you need to press "show" to expand).