Solving the Minimization Operator Problem

[member 428835]

Homework Statement

Given a Hilbert space $$V = \left\{ f\in L_2[0,1] | \int_0^1 f(x)\, dx = 0\right\},B(f,g) = \langle f,g\rangle,l(f) = \int_0^1 x f(x) \, dx$$ find the minimum of $$B(u,u)+2l(u)$$.

Homework Equations

In my text I found a variational theorem stating this minimization problem is equivalent to solving
$$
B(f,g)+l(f) = 0
$$
for fixed $g$ and all $f$.

The Attempt at a Solution

Plugging in values yields
$$
\int_0^1f(g+x)\, dx = 0.
$$
But from here I'm not sure how to handle the answer. Any suggestions? I'm guessing either $g=-x$ or else $f$ must be orthogonal to the function $g+x$ on the interval $[0,1]$. Unsure how to proceed.

 

[Ray Vickson]

Homework Statement

Given a Hilbert space $$V = \left\{ f\in L_2[0,1] | \int_0^1 f(x)\, dx = 0\right\},B(f,g) = \langle f,g\rangle,l(f) = \int_0^1 x f(x) \, dx$$ find the minimum of $$B(u,u)+2l(u)$$.

Homework Equations

In my text I found a variational theorem stating this minimization problem is equivalent to solving
$$
B(f,g)+l(f) = 0
$$
for fixed $g$ and all $f$.

The Attempt at a Solution

Plugging in values yields
$$
\int_0^1f(g+x)\, dx = 0.
$$
But from here I'm not sure how to handle the answer. Any suggestions? I'm guessing either $g=-x$ or else $f$ must be orthogonal to the function $g+x$ on the interval $[0,1]$. Unsure how to proceed.

You have
$$B(u,u) + 2 l(u) = \int_0^1 u^2(x) \, dx + 2\int_0^1 x u(x) \, dx = \int_0^1[ u(x)^2 + 2 x u(x)] \, dx.$$ The integrand equals $(u(x)+x)^2 -x^2$

 

[member 428835]

You have
$$B(u,u) + 2 l(u) = \int_0^1 u^2(x) \, dx + 2\int_0^1 x u(x) \, dx = \int_0^1[ u(x)^2 + 2 x u(x)] \, dx.$$ The integrand equals $(u(x)+x)^2 -x^2$

Right, this is what I must minimize (this integral). I'm not sure how to do that. But the theorem I wrote only requires the integral be zero. Am I missing something?

 

[fresh_42]

Right, this is what I must minimize (this integral). I'm not sure how to do that. But the theorem I wrote only requires the integral be zero. Am I missing something?

The integral over $x^2$ is a constant, so the minimization is the one of the integral $(u(x)+x)^2$. Now the question is, where is $u$ from to make it minimal?

 

[member 428835]

The integral over $x^2$ is a constant, so the minimization is the one of the integral $(u(x)+x)^2$. Now the question is, where is $u$ from to make it minimal?

Okay, I see what you're saying. Isn't $u \in L_2[0,1] : \int_0^1 u\, dx = 0$?

Since the integrand is squared is must be true that the integral can be no less than zero. I'd say this implies $u = -x$, except that $\langle -x,1\rangle \neq 0$. Ideas?

 

[fresh_42]

Okay, I see what you're saying. Isn't $u \in L_2[0,1] : \int_0^1 u\, dx = 0$?

Since the integrand is squared is must be true that the integral can be no less than zero. Doesn't this imply $u = -x$?

That's the question. If $u \in L^2([0,1])$ then $u=-x$ is the solution. If $u \in V$ then it's probably $u=0$, since $-x\notin V$, but that needs to be proven. It's the question: which element of $V$ is closest to $x$ in $L^2([0,1]).$

 

[member 428835]

That's the question. If $u \in L^2([0,1])$ then $u=-x$ is the solution, if $u \in V$ then it's probably $u=0$, since $-x\notin V$, but that needs to be proven. It's the question: which element of $V$ is closest to $x$ in $L^2([0,1]).$

Okay yea, I agree with you. But how do you know $u\in V \implies u=0$?

The closest is likely $u=1/2-x$?

 

[fresh_42]

Okay yea, I agree with you. But how do you know $u\in V \implies u=0$?

The closest is likely $u=1/2-x$?

You're right, my guess - and it was one - isn't closer than yours. That's why I said it needs a proof. Btw, yours as well. However, we have Euclidean spaces here, so the closest distance is a straight line and the minimization should be easy.

 

[member 428835]

You're right, my guess - and it was one - isn't closer than yours. That's why I said it needs a proof. Btw, yours as well. However, we have Euclidean spaces here, so the closest distance is a straight line and the minimization should be easy.

Can you elaborate on the straight line concept? I think you see something I don't.

 

[fresh_42]

Can you elaborate on the straight line concept? I think you see something I don't.

What we have is a hyperspace $V$ and a point $x$ outside. The shortest distance is at the basis of a normal vector of $V$ pointing to $x$, or a perpendicular vector to $V$. (I would look up the Gram-Schmidt procedure, but maybe it's easier, e.g. Lagrange multiplier, basis of $V$, power series for $u$ - just to name a few ideas). Your solution is probably correct, but why?

 

[member 428835]

What we have is a hyperspace $V$ and a point $x$ outside. The shortest distance is at the basis of a normal vector of $V$ pointing to $x$, or a perpendicular vector to $V$. (I would look up the Gram-Schmidt procedure, but maybe it's easier, e.g. Lagrange multiplier, basis of $V$, power series for $u$ - just to name a few ideas). Your solution is probably correct, but why?

You lost me. I am familiar with Gram-Schmidt (reconstructing orthonormal basis functions from ones that are not necessarily that) and Lagrange multipliers (optimization with a constraint, setting the gradients equal, yielding system of algebraic equations) and of course power series, which for you would perhaps be $u = 1/2 - x + HOT$ (Higher Order Terms).

But how do these help/are relevant?

 

[fresh_42]

I don't know. How do we find $u \in V$ such that the distance to $x$ is minimal? The major difficulty is to describe $u$ somehow. A basis is a possibility. Legendre polynomials work on $L^2([-1,1])$, but I'm sure there is also a known basis for $V$. Or you prove, that any other vector $u(x)= (\frac{1}{2}-x) + v(x)$ is necessarily further away, unless $v(x)=0$. We could write $u(x)=(\frac{1}{2}-x) + \lambda v(x)$ and treat $\lambda$ as Lagrange multiplier.

Since I don't have a solution in mind, I just name ideas.

Edit: $\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n\; , \;n>0$ looks like a basis.

Edit$^2$: The first basis vector is $2x-1$, and if we solve $u^2+ux=0$ (the condition that $u$ in $V$ is the perpendicular point to $x$) with $u=\lambda\cdot (2x-1)$, then we get $\lambda \in \{\,0,-\frac{1}{2}\,\}$ and $u(x)=-\frac{1}{2}(2x-1)$ is exactly your solution.

 

[Ray Vickson]

Okay, I see what you're saying. Isn't $u \in L_2[0,1] : \int_0^1 u\, dx = 0$?

Since the integrand is squared is must be true that the integral can be no less than zero. I'd say this implies $u = -x$, except that $\langle -x,1\rangle \neq 0$. Ideas?

Just to clarify: is your problem (1) or (2) below?
$$\begin{array}{cc}
(1) & \min B(u,u) + 2 l(u)\\
&\text{s.t.}\; u \in L_2[0,1] \\
\\
(2) & \min B(u,u) + 2 l(u) \\
&\text{s.t.} \; u \in V
\end{array}$$
If it is (2), then it becomes
$$\min \int_0^1 (u(x) + x)^2 \, dx \\
\text{subject to} \int_0^1 u(x) \, dx = 0
$$

 

[member 428835]

I don't know. How do we find $u \in V$ such that the distance to $x$ is minimal? The major difficulty is to describe $u$ somehow. A basis is a possibility. Legendre polynomials work on $L^2([-1,1])$, but I'm sure there is also a known basis for $V$. Or you prove, that any other vector $u(x)= (\frac{1}{2}-x) + v(x)$ is necessarily further away, unless $v(x)=0$. We could write $u(x)=(\frac{1}{2}-x) + \lambda v(x)$ and treat $\lambda$ as Lagrange multiplier.

Since I don't have a solution in mind, I just name ideas.

Edit: $\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n\; , \;n>0$ looks like a basis.

Edit$^2$: The first basis vector is $2x-1$, and if we solve $u^2+ux=0$ (the condition that $u$ in $V$ is the perpendicular point to $x$) with $u=\lambda\cdot (2x-1)$, then we get $\lambda \in \{\,0,-\frac{1}{2}\,\}$ and $u(x)=-\frac{1}{2}(2x-1)$ is exactly your solution.

How did you know the basis function was $\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n\; , \;n>0$? Otherwise I think I follow you, but let me summarize: Given the aforementioned basis, we take the first term to approximate $u : u=\lambda(2x-1)$ and substitute this into the equation $\langle u,u+x\rangle = 0$ and solve for the coefficient $\lambda$ (note $\langle u,u+x\rangle = 0$ is equivalent to $B(u,u) + l(u) = 0$). The linearly independent choice is $\lambda \neq 0 \implies \lambda = -1/2$. But I don't see how this minimizes $B(u,u) + 2l(u) = 0$.

 

[member 428835]

Just to clarify: is your problem (1) or (2) below?
$$\begin{array}{cc}
(1) & \min B(u,u) + 2 l(u)\\
&\text{s.t.}\; u \in L_2[0,1] \\
\\
(2) & \min B(u,u) + 2 l(u) \\
&\text{s.t.} \; u \in V
\end{array}$$
If it is (2), then it becomes
$$\min \int_0^1 (u(x) + x)^2 \, dx \\
\text{subject to} \int_0^1 u(x) \, dx = 0
$$

(2) is the correct problem (though it seems fresh_42 solved (1) too). In general I am not sure how to solve this sort of Lagrange multiplier integral equation. I'm happy to learn another technique, though this seems similar to what fresh_42 wrote.

My guess is Lagrange multiplier implies solving the two equations $$\int_0^1 u(x) \, dx = 0\\
(u(x) + x)^2 = \lambda u(x)$$ but this doesn't feel right since the integrals have boundaries.

 

[fresh_42]

How did you know the basis function was $\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n\; , \;n>0$?

I looked at the Legendre polynomials and modified a formula for them (Rodrigues formula).

Otherwise I think I follow you, but let me summarize: Given the aforementioned basis, we take the first term to approximate $u : u=\lambda(2x-1)$ and substitute this into the equation $\langle u,u+x\rangle = 0$ and solve for the coefficient $\lambda$ (note $\langle u,u+x\rangle = 0$ is equivalent to $B(u,u) + l(u) = 0$). The linearly independent choice is $\lambda \neq 0 \implies \lambda = -1/2$. But I don't see how this minimizes $B(u,u) + 2l(u) = 0$.

It doesn't show that's the optimum, it only indicates a way to do so. You have to contribute a little, too.
Say we name the polynomials $P_n=\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n$. Then we already know that the shortest distance between the straight $\lambda P_1$ and the point $x$ is at $w$ for $\lambda = \frac{1}{2}$. The situation is as follows:

Now which possibilities do we have? First of all, I think the basis polynomials are orthogonal, so this might help in calculations. I haven't calculated their norm either, but I don't think we have to norm them in case they aren't already. I also don't know whether they form a basis. They should be linear independent, but do they span $V$? Do we even need this? Anyway, we want to show $w=u$. So our options are e.g.

• Calculate the angels, as $u$ is the unique perpendicular to all vectors in $V$. Does $w$ have this property?
• Vary $w + \mu P$ for some vector $P\neq P_1$ and show that the distance becomes larger for $\mu \neq 0$.

 

[member 428835]

I looked at the Legendre polynomials and modified a formula for them (Rodrigues formula).

It doesn't show that's the optimum, it only indicates a way to do so. You have to contribute a little, too.
Say we name the polynomials $P_n=\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n$. Then we already know that the shortest distance between the straight $\lambda P_1$ and the point $x$ is at $w$ for $\lambda = \frac{1}{2}$. The situation is as follows:View attachment 240529
Now which possibilities do we have? First of all, I think the basis polynomials are orthogonal, so this might help in calculations. I haven't calculated their norm either, but I don't think we have to norm them in case they aren't already. Anyway, we want to show $w=u$. So our options are e.g.

• Calculate the angels, as $u$ is the unique perpendicular to all vectors in $V$. Does $w$ have this property?
• Vary $w + \mu P$ for some vector $P\neq P_1$ and show that the distance becomes larger for $\mu \neq 0$.

Wow, that was a lot. I'm trying to follow you: so are you saying the space $V$ is constructed from the basis you present, and that the shortest distance from a point $x$ to $V$ is a straight line with shortest length, and therefore must be orthogonal to all other basis for $n \neq 1$?

Then is seems like $\int_0^1 P_n P_1 \, dx = 0$ implies $P_1$ is orthogonal to the rest of $V$?

 

[fresh_42]

Wow, that was a lot. I'm trying to follow you: so are you saying the space $V$ is constructed from the basis you present, and that the shortest distance from a point $x$ to $V$ is a straight line with shortest length, and therefore must be orthogonal to all other basis for $n \neq 1$?

Then is seems like $\int_0^1 P_n P_1 \, dx = 0$ implies $P_1$ is orthogonal to the rest of $V$?

I edited my previous post while it was in your cache. We don't know enough yet about the properties of the $P_n$. I think orthogonality is easy to show, but I haven't done any property checks for them. I would try to show that $w$ is optimal (2nd option). We know that $P_1$ can be expanded to a orthonormal system, which should do.

 

[Ray Vickson]

(2) is the correct problem (though it seems fresh_42 solved (1) too). In general I am not sure how to solve this sort of Lagrange multiplier integral equation. I'm happy to learn another technique, though this seems similar to what fresh_42 wrote.

My guess is Lagrange multiplier implies solving the two equations $$\int_0^1 u(x) \, dx = 0\\
(u(x) + x)^2 = \lambda u(x)$$ but this doesn't feel right since the integrals have boundaries.

What you have is similar to a constrained calculus-of-variations problem, but without any $u'$ terms present in the integral. You could write $I = \int_0^1 [(x + u(x))^2 - \lambda u(x) ] \, dx$ and look for an unconstrained minimum of $I$. However, if I were doing it I would proceed much differently, as follows.

If you think of "discretizing" the problem (approximating the integrals by finite sums) you will have a problem of the form
$$\begin{array}{cl}
\min & \sum_{i=1}^n (x_i + u_i)^2 \\
\text{such that}& \sum_{i=1}^n u_i = 0
\end{array}$$
Here, $x_2, x_2, \ldots, x_n \in [0,1]$ are equally-spaced points we use to approximate the integral by a sum, and the $u_i$ are supposed to be $u(x_i)$. Basically, we are replacing the integral $\int_0^1 F(x) \, dx$ by $\sum_{i=1}^n \Delta x F(x_i),$ then dropping the (constant) $\Delta x,$ which will not affect the position of the minimum.

Note that the $u_i$ are just some independent variables to be determined. Before we optimize they are independent of the $x_i$, but after optimizing they might become functions of the $x_i$. After we determine the $u_i$ we can get information about $u(x)$ by setting $u(x_i) = u_i.$

The finite optimization problem above has an elementary solution in terms of the Lagrange multiplier method, so one can easily obtain a discrete version of $u(x).$ This is then easy to extend to the continuous domain.

 

[member 428835]

Thank you both for responding. I am going to submit what I have so far (just don't have time to look further into what you both suggested, though I will when I get time, as I love learning new methods). As such I'm marking this thread solved. I will post the professors solution (if he gives one).

Thanks again for taking the time to help! I really appreciate your expertise!