• Support PF! Buy your school textbooks, materials and every day products via PF Here!

Minimization Operator Problem

1,682
50
1. The problem statement, all variables and given/known data
Given a Hilbert space $$V = \left\{ f\in L_2[0,1] | \int_0^1 f(x)\, dx = 0\right\},B(f,g) = \langle f,g\rangle,l(f) = \int_0^1 x f(x) \, dx$$ find the minimum of $$B(u,u)+2l(u)$$.

2. Relevant equations
In my text I found a variational theorem stating this minimization problem is equivalent to solving
$$
B(f,g)+l(f) = 0
$$
for fixed ##g## and all ##f##.

3. The attempt at a solution
Plugging in values yields
$$
\int_0^1f(g+x)\, dx = 0.
$$
But from here I'm not sure how to handle the answer. Any suggestions? I'm guessing either ##g=-x## or else ##f## must be orthogonal to the function ##g+x## on the interval ##[0,1]##. Unsure how to proceed.
 

Ray Vickson

Science Advisor
Homework Helper
Dearly Missed
10,705
1,710
1. The problem statement, all variables and given/known data
Given a Hilbert space $$V = \left\{ f\in L_2[0,1] | \int_0^1 f(x)\, dx = 0\right\},B(f,g) = \langle f,g\rangle,l(f) = \int_0^1 x f(x) \, dx$$ find the minimum of $$B(u,u)+2l(u)$$.

2. Relevant equations
In my text I found a variational theorem stating this minimization problem is equivalent to solving
$$
B(f,g)+l(f) = 0
$$
for fixed ##g## and all ##f##.

3. The attempt at a solution
Plugging in values yields
$$
\int_0^1f(g+x)\, dx = 0.
$$
But from here I'm not sure how to handle the answer. Any suggestions? I'm guessing either ##g=-x## or else ##f## must be orthogonal to the function ##g+x## on the interval ##[0,1]##. Unsure how to proceed.
You have
$$B(u,u) + 2 l(u) = \int_0^1 u^2(x) \, dx + 2\int_0^1 x u(x) \, dx = \int_0^1[ u(x)^2 + 2 x u(x)] \, dx.$$ The integrand equals ## (u(x)+x)^2 -x^2##
 
1,682
50
You have
$$B(u,u) + 2 l(u) = \int_0^1 u^2(x) \, dx + 2\int_0^1 x u(x) \, dx = \int_0^1[ u(x)^2 + 2 x u(x)] \, dx.$$ The integrand equals ## (u(x)+x)^2 -x^2##
Right, this is what I must minimize (this integral). I'm not sure how to do that. But the theorem I wrote only requires the integral be zero. Am I missing something?
 

fresh_42

Mentor
Insights Author
2018 Award
9,999
6,745
Right, this is what I must minimize (this integral). I'm not sure how to do that. But the theorem I wrote only requires the integral be zero. Am I missing something?
The integral over ##x^2## is a constant, so the minimization is the one of the integral ##(u(x)+x)^2##. Now the question is, where is ##u## from to make it minimal?
 
1,682
50
The integral over ##x^2## is a constant, so the minimization is the one of the integral ##(u(x)+x)^2##. Now the question is, where is ##u## from to make it minimal?
Okay, I see what you're saying. Isn't ##u \in L_2[0,1] : \int_0^1 u\, dx = 0##?

Since the integrand is squared is must be true that the integral can be no less than zero. I'd say this implies ##u = -x##, except that ##\langle -x,1\rangle \neq 0##. Ideas?
 

fresh_42

Mentor
Insights Author
2018 Award
9,999
6,745
Okay, I see what you're saying. Isn't ##u \in L_2[0,1] : \int_0^1 u\, dx = 0##?

Since the integrand is squared is must be true that the integral can be no less than zero. Doesn't this imply ##u = -x##?
That's the question. If ##u \in L^2([0,1])## then ##u=-x## is the solution. If ##u \in V## then it's probably ##u=0##, since ##-x\notin V##, but that needs to be proven. It's the question: which element of ##V## is closest to ##x## in ##L^2([0,1]).##
 
1,682
50
That's the question. If ##u \in L^2([0,1])## then ##u=-x## is the solution, if ##u \in V## then it's probably ##u=0##, since ##-x\notin V##, but that needs to be proven. It's the question: which element of ##V## is closest to ##x## in ##L^2([0,1]).##
Okay yea, I agree with you. But how do you know ##u\in V \implies u=0##?

The closest is likely ##u=1/2-x##?
 

fresh_42

Mentor
Insights Author
2018 Award
9,999
6,745
Okay yea, I agree with you. But how do you know ##u\in V \implies u=0##?

The closest is likely ##u=1/2-x##?
You're right, my guess - and it was one - isn't closer than yours. That's why I said it needs a proof. Btw, yours as well. However, we have Euclidean spaces here, so the closest distance is a straight line and the minimization should be easy.
 
1,682
50
You're right, my guess - and it was one - isn't closer than yours. That's why I said it needs a proof. Btw, yours as well. However, we have Euclidean spaces here, so the closest distance is a straight line and the minimization should be easy.
Can you elaborate on the straight line concept? I think you see something I don't.
 

fresh_42

Mentor
Insights Author
2018 Award
9,999
6,745
Can you elaborate on the straight line concept? I think you see something I don't.
What we have is a hyperspace ##V## and a point ##x## outside. The shortest distance is at the basis of a normal vector of ##V## pointing to ##x##, or a perpendicular vector to ##V##. (I would look up the Gram-Schmidt procedure, but maybe it's easier, e.g. Lagrange multiplier, basis of ##V##, power series for ##u## - just to name a few ideas). Your solution is probably correct, but why?
 
Last edited:
1,682
50
What we have is a hyperspace ##V## and a point ##x## outside. The shortest distance is at the basis of a normal vector of ##V## pointing to ##x##, or a perpendicular vector to ##V##. (I would look up the Gram-Schmidt procedure, but maybe it's easier, e.g. Lagrange multiplier, basis of ##V##, power series for ##u## - just to name a few ideas). Your solution is probably correct, but why?
You lost me. I am familiar with Gram-Schmidt (reconstructing orthonormal basis functions from ones that are not necessarily that) and Lagrange multipliers (optimization with a constraint, setting the gradients equal, yielding system of algebraic equations) and of course power series, which for you would perhaps be ##u = 1/2 - x + HOT## (Higher Order Terms).

But how do these help/are relevant?
 

fresh_42

Mentor
Insights Author
2018 Award
9,999
6,745
I don't know. How do we find ##u \in V## such that the distance to ##x## is minimal? The major difficulty is to describe ##u## somehow. A basis is a possibility. Legendre polynomials work on ##L^2([-1,1])##, but I'm sure there is also a known basis for ##V##. Or you prove, that any other vector ##u(x)= (\frac{1}{2}-x) + v(x)## is necessarily further away, unless ##v(x)=0##. We could write ##u(x)=(\frac{1}{2}-x) + \lambda v(x)## and treat ##\lambda## as Lagrange multiplier.

Since I don't have a solution in mind, I just name ideas.

Edit: ##\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n\; , \;n>0## looks like a basis.

Edit##^2##: The first basis vector is ##2x-1##, and if we solve ##u^2+ux=0## (the condition that ##u## in ##V## is the perpendicular point to ##x##) with ##u=\lambda\cdot (2x-1)##, then we get ##\lambda \in \{\,0,-\frac{1}{2}\,\}## and ##u(x)=-\frac{1}{2}(2x-1)## is exactly your solution.
 
Last edited:

Ray Vickson

Science Advisor
Homework Helper
Dearly Missed
10,705
1,710
Okay, I see what you're saying. Isn't ##u \in L_2[0,1] : \int_0^1 u\, dx = 0##?

Since the integrand is squared is must be true that the integral can be no less than zero. I'd say this implies ##u = -x##, except that ##\langle -x,1\rangle \neq 0##. Ideas?
Just to clarify: is your problem (1) or (2) below?
$$\begin{array}{cc}
(1) & \min B(u,u) + 2 l(u)\\
&\text{s.t.}\; u \in L_2[0,1] \\
\\
(2) & \min B(u,u) + 2 l(u) \\
&\text{s.t.} \; u \in V
\end{array}$$
If it is (2), then it becomes
$$\min \int_0^1 (u(x) + x)^2 \, dx \\
\text{subject to} \int_0^1 u(x) \, dx = 0
$$
 
1,682
50
I don't know. How do we find ##u \in V## such that the distance to ##x## is minimal? The major difficulty is to describe ##u## somehow. A basis is a possibility. Legendre polynomials work on ##L^2([-1,1])##, but I'm sure there is also a known basis for ##V##. Or you prove, that any other vector ##u(x)= (\frac{1}{2}-x) + v(x)## is necessarily further away, unless ##v(x)=0##. We could write ##u(x)=(\frac{1}{2}-x) + \lambda v(x)## and treat ##\lambda## as Lagrange multiplier.

Since I don't have a solution in mind, I just name ideas.

Edit: ##\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n\; , \;n>0## looks like a basis.

Edit##^2##: The first basis vector is ##2x-1##, and if we solve ##u^2+ux=0## (the condition that ##u## in ##V## is the perpendicular point to ##x##) with ##u=\lambda\cdot (2x-1)##, then we get ##\lambda \in \{\,0,-\frac{1}{2}\,\}## and ##u(x)=-\frac{1}{2}(2x-1)## is exactly your solution.
How did you know the basis function was ##\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n\; , \;n>0##? Otherwise I think I follow you, but let me summarize: Given the aforementioned basis, we take the first term to approximate ##u : u=\lambda(2x-1)## and substitute this into the equation ##\langle u,u+x\rangle = 0## and solve for the coefficient ##\lambda## (note ##\langle u,u+x\rangle = 0## is equivalent to ##B(u,u) + l(u) = 0##). The linearly independent choice is ##\lambda \neq 0 \implies \lambda = -1/2##. But I don't see how this minimizes ##B(u,u) + 2l(u) = 0##.
 
1,682
50
Just to clarify: is your problem (1) or (2) below?
$$\begin{array}{cc}
(1) & \min B(u,u) + 2 l(u)\\
&\text{s.t.}\; u \in L_2[0,1] \\
\\
(2) & \min B(u,u) + 2 l(u) \\
&\text{s.t.} \; u \in V
\end{array}$$
If it is (2), then it becomes
$$\min \int_0^1 (u(x) + x)^2 \, dx \\
\text{subject to} \int_0^1 u(x) \, dx = 0
$$
(2) is the correct problem (though it seems fresh_42 solved (1) too). In general I am not sure how to solve this sort of Lagrange multiplier integral equation. I'm happy to learn another technique, though this seems similar to what fresh_42 wrote.

My guess is Lagrange multiplier implies solving the two equations $$\int_0^1 u(x) \, dx = 0\\
(u(x) + x)^2 = \lambda u(x)$$ but this doesn't feel right since the integrals have boundaries.
 

fresh_42

Mentor
Insights Author
2018 Award
9,999
6,745
How did you know the basis function was ##\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n\; , \;n>0##?
I looked at the Legendre polynomials and modified a formula for them (Rodrigues formula).
Otherwise I think I follow you, but let me summarize: Given the aforementioned basis, we take the first term to approximate ##u : u=\lambda(2x-1)## and substitute this into the equation ##\langle u,u+x\rangle = 0## and solve for the coefficient ##\lambda## (note ##\langle u,u+x\rangle = 0## is equivalent to ##B(u,u) + l(u) = 0##). The linearly independent choice is ##\lambda \neq 0 \implies \lambda = -1/2##. But I don't see how this minimizes ##B(u,u) + 2l(u) = 0##.
It doesn't show that's the optimum, it only indicates a way to do so. You have to contribute a little, too.
Say we name the polynomials ##P_n=\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n##. Then we already know that the shortest distance between the straight ##\lambda P_1## and the point ##x## is at ##w## for ##\lambda = \frac{1}{2}##. The situation is as follows:


upload_2019-3-19_19-23-20.png

Now which possibilities do we have? First of all, I think the basis polynomials are orthogonal, so this might help in calculations. I haven't calculated their norm either, but I don't think we have to norm them in case they aren't already. I also don't know whether they form a basis. They should be linear independent, but do they span ##V##? Do we even need this? Anyway, we want to show ##w=u##. So our options are e.g.
  • Calculate the angels, as ##u## is the unique perpendicular to all vectors in ##V ##. Does ##w## have this property?
  • Vary ##w + \mu P## for some vector ##P\neq P_1## and show that the distance becomes larger for ##\mu \neq 0##.
 

Attachments

Last edited:
1,682
50
I looked at the Legendre polynomials and modified a formula for them (Rodrigues formula).

It doesn't show that's the optimum, it only indicates a way to do so. You have to contribute a little, too.
Say we name the polynomials ##P_n=\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n##. Then we already know that the shortest distance between the straight ##\lambda P_1## and the point ##x## is at ##w## for ##\lambda = \frac{1}{2}##. The situation is as follows:


View attachment 240529
Now which possibilities do we have? First of all, I think the basis polynomials are orthogonal, so this might help in calculations. I haven't calculated their norm either, but I don't think we have to norm them in case they aren't already. Anyway, we want to show ##w=u##. So our options are e.g.
  • Calculate the angels, as ##u## is the unique perpendicular to all vectors in ##V ##. Does ##w## have this property?
  • Vary ##w + \mu P## for some vector ##P\neq P_1## and show that the distance becomes larger for ##\mu \neq 0##.
Wow, that was a lot. I'm trying to follow you: so are you saying the space ##V## is constructed from the basis you present, and that the shortest distance from a point ##x## to ##V## is a straight line with shortest length, and therefore must be orthogonal to all other basis for ##n \neq 1##?

Then is seems like ##\int_0^1 P_n P_1 \, dx = 0## implies ##P_1## is orthogonal to the rest of ##V##?
 

fresh_42

Mentor
Insights Author
2018 Award
9,999
6,745
Wow, that was a lot. I'm trying to follow you: so are you saying the space ##V## is constructed from the basis you present, and that the shortest distance from a point ##x## to ##V## is a straight line with shortest length, and therefore must be orthogonal to all other basis for ##n \neq 1##?

Then is seems like ##\int_0^1 P_n P_1 \, dx = 0## implies ##P_1## is orthogonal to the rest of ##V##?
I edited my previous post while it was in your cache. We don't know enough yet about the properties of the ##P_n##. I think orthogonality is easy to show, but I haven't done any property checks for them. I would try to show that ##w## is optimal (2nd option). We know that ##P_1## can be expanded to a orthonormal system, which should do.
 

Ray Vickson

Science Advisor
Homework Helper
Dearly Missed
10,705
1,710
(2) is the correct problem (though it seems fresh_42 solved (1) too). In general I am not sure how to solve this sort of Lagrange multiplier integral equation. I'm happy to learn another technique, though this seems similar to what fresh_42 wrote.

My guess is Lagrange multiplier implies solving the two equations $$\int_0^1 u(x) \, dx = 0\\
(u(x) + x)^2 = \lambda u(x)$$ but this doesn't feel right since the integrals have boundaries.
What you have is similar to a constrained calculus-of-variations problem, but without any ##u'## terms present in the integral. You could write ##I = \int_0^1 [(x + u(x))^2 - \lambda u(x) ] \, dx## and look for an unconstrained minimum of ##I##. However, if I were doing it I would proceed much differently, as follows.

If you think of "discretizing" the problem (approximating the integrals by finite sums) you will have a problem of the form
$$\begin{array}{cl}
\min & \sum_{i=1}^n (x_i + u_i)^2 \\
\text{such that}& \sum_{i=1}^n u_i = 0
\end{array}$$
Here, ##x_2, x_2, \ldots, x_n \in [0,1]## are equally-spaced points we use to approximate the integral by a sum, and the ##u_i## are supposed to be ##u(x_i)##. Basically, we are replacing the integral ##\int_0^1 F(x) \, dx## by ## \sum_{i=1}^n \Delta x F(x_i),## then dropping the (constant) ##\Delta x,## which will not affect the position of the minimum.

Note that the ##u_i## are just some independent variables to be determined. Before we optimize they are independent of the ##x_i##, but after optimizing they might become functions of the ##x_i##. After we determine the ##u_i## we can get information about ##u(x)## by setting ##u(x_i) = u_i.##

The finite optimization problem above has an elementary solution in terms of the Lagrange multiplier method, so one can easily obtain a discrete version of ##u(x).## This is then easy to extend to the continuous domain.
 
Last edited:
1,682
50
Thank you both for responding. I am going to submit what I have so far (just don't have time to look further into what you both suggested, though I will when I get time, as I love learning new methods). As such I'm marking this thread solved. I will post the professors solution (if he gives one).

Thanks again for taking the time to help! I really appreciate your expertise!
 

Want to reply to this thread?

"Minimization Operator Problem" You must log in or register to reply here.

Physics Forums Values

We Value Quality
• Topics based on mainstream science
• Proper English grammar and spelling
We Value Civility
• Positive and compassionate attitudes
• Patience while debating
We Value Productivity
• Disciplined to remain on-topic
• Recognition of own weaknesses
• Solo and co-op problem solving
Top