Solving the Minimization Operator Problem

You lost me. I am familiar with Gram-Schmidt (reconstructing orthonormal basis functions from ones that are not necessarily that) and Lagrange multipliers (optimization with a constraint, setting the gradients equal, yielding system of algebraic equations) and of course power series, which for you would perhaps be ##u = 1/2 - x + HOT## (Higher Order Terms).But how do these help/are relevant?I don't know. How do we find ##u \in V## such that the distance to ##x## is minimal? The major difficulty is to describe ##u## somehow. A basis is a possibility. Legendre polynomials work on ##L^2([-1,1])
  • #1
member 428835

Homework Statement


Given a Hilbert space $$V = \left\{ f\in L_2[0,1] | \int_0^1 f(x)\, dx = 0\right\},B(f,g) = \langle f,g\rangle,l(f) = \int_0^1 x f(x) \, dx$$ find the minimum of $$B(u,u)+2l(u)$$.

Homework Equations


In my text I found a variational theorem stating this minimization problem is equivalent to solving
$$
B(f,g)+l(f) = 0
$$
for fixed ##g## and all ##f##.

The Attempt at a Solution


Plugging in values yields
$$
\int_0^1f(g+x)\, dx = 0.
$$
But from here I'm not sure how to handle the answer. Any suggestions? I'm guessing either ##g=-x## or else ##f## must be orthogonal to the function ##g+x## on the interval ##[0,1]##. Unsure how to proceed.
 
Physics news on Phys.org
  • #2
joshmccraney said:

Homework Statement


Given a Hilbert space $$V = \left\{ f\in L_2[0,1] | \int_0^1 f(x)\, dx = 0\right\},B(f,g) = \langle f,g\rangle,l(f) = \int_0^1 x f(x) \, dx$$ find the minimum of $$B(u,u)+2l(u)$$.

Homework Equations


In my text I found a variational theorem stating this minimization problem is equivalent to solving
$$
B(f,g)+l(f) = 0
$$
for fixed ##g## and all ##f##.

The Attempt at a Solution


Plugging in values yields
$$
\int_0^1f(g+x)\, dx = 0.
$$
But from here I'm not sure how to handle the answer. Any suggestions? I'm guessing either ##g=-x## or else ##f## must be orthogonal to the function ##g+x## on the interval ##[0,1]##. Unsure how to proceed.

You have
$$B(u,u) + 2 l(u) = \int_0^1 u^2(x) \, dx + 2\int_0^1 x u(x) \, dx = \int_0^1[ u(x)^2 + 2 x u(x)] \, dx.$$ The integrand equals ## (u(x)+x)^2 -x^2##
 
  • #3
Ray Vickson said:
You have
$$B(u,u) + 2 l(u) = \int_0^1 u^2(x) \, dx + 2\int_0^1 x u(x) \, dx = \int_0^1[ u(x)^2 + 2 x u(x)] \, dx.$$ The integrand equals ## (u(x)+x)^2 -x^2##
Right, this is what I must minimize (this integral). I'm not sure how to do that. But the theorem I wrote only requires the integral be zero. Am I missing something?
 
  • #4
joshmccraney said:
Right, this is what I must minimize (this integral). I'm not sure how to do that. But the theorem I wrote only requires the integral be zero. Am I missing something?
The integral over ##x^2## is a constant, so the minimization is the one of the integral ##(u(x)+x)^2##. Now the question is, where is ##u## from to make it minimal?
 
  • #5
fresh_42 said:
The integral over ##x^2## is a constant, so the minimization is the one of the integral ##(u(x)+x)^2##. Now the question is, where is ##u## from to make it minimal?
Okay, I see what you're saying. Isn't ##u \in L_2[0,1] : \int_0^1 u\, dx = 0##?

Since the integrand is squared is must be true that the integral can be no less than zero. I'd say this implies ##u = -x##, except that ##\langle -x,1\rangle \neq 0##. Ideas?
 
  • #6
joshmccraney said:
Okay, I see what you're saying. Isn't ##u \in L_2[0,1] : \int_0^1 u\, dx = 0##?

Since the integrand is squared is must be true that the integral can be no less than zero. Doesn't this imply ##u = -x##?
That's the question. If ##u \in L^2([0,1])## then ##u=-x## is the solution. If ##u \in V## then it's probably ##u=0##, since ##-x\notin V##, but that needs to be proven. It's the question: which element of ##V## is closest to ##x## in ##L^2([0,1]).##
 
  • Like
Likes member 428835
  • #7
fresh_42 said:
That's the question. If ##u \in L^2([0,1])## then ##u=-x## is the solution, if ##u \in V## then it's probably ##u=0##, since ##-x\notin V##, but that needs to be proven. It's the question: which element of ##V## is closest to ##x## in ##L^2([0,1]).##
Okay yea, I agree with you. But how do you know ##u\in V \implies u=0##?

The closest is likely ##u=1/2-x##?
 
  • #8
joshmccraney said:
Okay yea, I agree with you. But how do you know ##u\in V \implies u=0##?

The closest is likely ##u=1/2-x##?
You're right, my guess - and it was one - isn't closer than yours. That's why I said it needs a proof. Btw, yours as well. However, we have Euclidean spaces here, so the closest distance is a straight line and the minimization should be easy.
 
  • #9
fresh_42 said:
You're right, my guess - and it was one - isn't closer than yours. That's why I said it needs a proof. Btw, yours as well. However, we have Euclidean spaces here, so the closest distance is a straight line and the minimization should be easy.
Can you elaborate on the straight line concept? I think you see something I don't.
 
  • #10
joshmccraney said:
Can you elaborate on the straight line concept? I think you see something I don't.
What we have is a hyperspace ##V## and a point ##x## outside. The shortest distance is at the basis of a normal vector of ##V## pointing to ##x##, or a perpendicular vector to ##V##. (I would look up the Gram-Schmidt procedure, but maybe it's easier, e.g. Lagrange multiplier, basis of ##V##, power series for ##u## - just to name a few ideas). Your solution is probably correct, but why?
 
Last edited:
  • #11
fresh_42 said:
What we have is a hyperspace ##V## and a point ##x## outside. The shortest distance is at the basis of a normal vector of ##V## pointing to ##x##, or a perpendicular vector to ##V##. (I would look up the Gram-Schmidt procedure, but maybe it's easier, e.g. Lagrange multiplier, basis of ##V##, power series for ##u## - just to name a few ideas). Your solution is probably correct, but why?
You lost me. I am familiar with Gram-Schmidt (reconstructing orthonormal basis functions from ones that are not necessarily that) and Lagrange multipliers (optimization with a constraint, setting the gradients equal, yielding system of algebraic equations) and of course power series, which for you would perhaps be ##u = 1/2 - x + HOT## (Higher Order Terms).

But how do these help/are relevant?
 
  • #12
I don't know. How do we find ##u \in V## such that the distance to ##x## is minimal? The major difficulty is to describe ##u## somehow. A basis is a possibility. Legendre polynomials work on ##L^2([-1,1])##, but I'm sure there is also a known basis for ##V##. Or you prove, that any other vector ##u(x)= (\frac{1}{2}-x) + v(x)## is necessarily further away, unless ##v(x)=0##. We could write ##u(x)=(\frac{1}{2}-x) + \lambda v(x)## and treat ##\lambda## as Lagrange multiplier.

Since I don't have a solution in mind, I just name ideas.

Edit: ##\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n\; , \;n>0## looks like a basis.

Edit##^2##: The first basis vector is ##2x-1##, and if we solve ##u^2+ux=0## (the condition that ##u## in ##V## is the perpendicular point to ##x##) with ##u=\lambda\cdot (2x-1)##, then we get ##\lambda \in \{\,0,-\frac{1}{2}\,\}## and ##u(x)=-\frac{1}{2}(2x-1)## is exactly your solution.
 
Last edited:
  • #13
joshmccraney said:
Okay, I see what you're saying. Isn't ##u \in L_2[0,1] : \int_0^1 u\, dx = 0##?

Since the integrand is squared is must be true that the integral can be no less than zero. I'd say this implies ##u = -x##, except that ##\langle -x,1\rangle \neq 0##. Ideas?

Just to clarify: is your problem (1) or (2) below?
$$\begin{array}{cc}
(1) & \min B(u,u) + 2 l(u)\\
&\text{s.t.}\; u \in L_2[0,1] \\
\\
(2) & \min B(u,u) + 2 l(u) \\
&\text{s.t.} \; u \in V
\end{array}$$
If it is (2), then it becomes
$$\min \int_0^1 (u(x) + x)^2 \, dx \\
\text{subject to} \int_0^1 u(x) \, dx = 0
$$
 
  • #14
fresh_42 said:
I don't know. How do we find ##u \in V## such that the distance to ##x## is minimal? The major difficulty is to describe ##u## somehow. A basis is a possibility. Legendre polynomials work on ##L^2([-1,1])##, but I'm sure there is also a known basis for ##V##. Or you prove, that any other vector ##u(x)= (\frac{1}{2}-x) + v(x)## is necessarily further away, unless ##v(x)=0##. We could write ##u(x)=(\frac{1}{2}-x) + \lambda v(x)## and treat ##\lambda## as Lagrange multiplier.

Since I don't have a solution in mind, I just name ideas.

Edit: ##\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n\; , \;n>0## looks like a basis.

Edit##^2##: The first basis vector is ##2x-1##, and if we solve ##u^2+ux=0## (the condition that ##u## in ##V## is the perpendicular point to ##x##) with ##u=\lambda\cdot (2x-1)##, then we get ##\lambda \in \{\,0,-\frac{1}{2}\,\}## and ##u(x)=-\frac{1}{2}(2x-1)## is exactly your solution.
How did you know the basis function was ##\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n\; , \;n>0##? Otherwise I think I follow you, but let me summarize: Given the aforementioned basis, we take the first term to approximate ##u : u=\lambda(2x-1)## and substitute this into the equation ##\langle u,u+x\rangle = 0## and solve for the coefficient ##\lambda## (note ##\langle u,u+x\rangle = 0## is equivalent to ##B(u,u) + l(u) = 0##). The linearly independent choice is ##\lambda \neq 0 \implies \lambda = -1/2##. But I don't see how this minimizes ##B(u,u) + 2l(u) = 0##.
 
  • #15
Ray Vickson said:
Just to clarify: is your problem (1) or (2) below?
$$\begin{array}{cc}
(1) & \min B(u,u) + 2 l(u)\\
&\text{s.t.}\; u \in L_2[0,1] \\
\\
(2) & \min B(u,u) + 2 l(u) \\
&\text{s.t.} \; u \in V
\end{array}$$
If it is (2), then it becomes
$$\min \int_0^1 (u(x) + x)^2 \, dx \\
\text{subject to} \int_0^1 u(x) \, dx = 0
$$
(2) is the correct problem (though it seems fresh_42 solved (1) too). In general I am not sure how to solve this sort of Lagrange multiplier integral equation. I'm happy to learn another technique, though this seems similar to what fresh_42 wrote.

My guess is Lagrange multiplier implies solving the two equations $$\int_0^1 u(x) \, dx = 0\\
(u(x) + x)^2 = \lambda u(x)$$ but this doesn't feel right since the integrals have boundaries.
 
  • #16
joshmccraney said:
How did you know the basis function was ##\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n\; , \;n>0##?
I looked at the Legendre polynomials and modified a formula for them (Rodrigues formula).
Otherwise I think I follow you, but let me summarize: Given the aforementioned basis, we take the first term to approximate ##u : u=\lambda(2x-1)## and substitute this into the equation ##\langle u,u+x\rangle = 0## and solve for the coefficient ##\lambda## (note ##\langle u,u+x\rangle = 0## is equivalent to ##B(u,u) + l(u) = 0##). The linearly independent choice is ##\lambda \neq 0 \implies \lambda = -1/2##. But I don't see how this minimizes ##B(u,u) + 2l(u) = 0##.
It doesn't show that's the optimum, it only indicates a way to do so. You have to contribute a little, too.
Say we name the polynomials ##P_n=\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n##. Then we already know that the shortest distance between the straight ##\lambda P_1## and the point ##x## is at ##w## for ##\lambda = \frac{1}{2}##. The situation is as follows:
upload_2019-3-19_19-23-20.png

Now which possibilities do we have? First of all, I think the basis polynomials are orthogonal, so this might help in calculations. I haven't calculated their norm either, but I don't think we have to norm them in case they aren't already. I also don't know whether they form a basis. They should be linear independent, but do they span ##V##? Do we even need this? Anyway, we want to show ##w=u##. So our options are e.g.
  • Calculate the angels, as ##u## is the unique perpendicular to all vectors in ##V ##. Does ##w## have this property?
  • Vary ##w + \mu P## for some vector ##P\neq P_1## and show that the distance becomes larger for ##\mu \neq 0##.
 

Attachments

  • upload_2019-3-19_19-21-9.png
    upload_2019-3-19_19-21-9.png
    3.8 KB · Views: 209
  • upload_2019-3-19_19-23-20.png
    upload_2019-3-19_19-23-20.png
    3.9 KB · Views: 451
Last edited:
  • #17
fresh_42 said:
I looked at the Legendre polynomials and modified a formula for them (Rodrigues formula).

It doesn't show that's the optimum, it only indicates a way to do so. You have to contribute a little, too.
Say we name the polynomials ##P_n=\dfrac{1}{n!} \dfrac{d^n}{dx^n} (x^2-x)^n##. Then we already know that the shortest distance between the straight ##\lambda P_1## and the point ##x## is at ##w## for ##\lambda = \frac{1}{2}##. The situation is as follows:View attachment 240529
Now which possibilities do we have? First of all, I think the basis polynomials are orthogonal, so this might help in calculations. I haven't calculated their norm either, but I don't think we have to norm them in case they aren't already. Anyway, we want to show ##w=u##. So our options are e.g.
  • Calculate the angels, as ##u## is the unique perpendicular to all vectors in ##V ##. Does ##w## have this property?
  • Vary ##w + \mu P## for some vector ##P\neq P_1## and show that the distance becomes larger for ##\mu \neq 0##.
Wow, that was a lot. I'm trying to follow you: so are you saying the space ##V## is constructed from the basis you present, and that the shortest distance from a point ##x## to ##V## is a straight line with shortest length, and therefore must be orthogonal to all other basis for ##n \neq 1##?

Then is seems like ##\int_0^1 P_n P_1 \, dx = 0## implies ##P_1## is orthogonal to the rest of ##V##?
 
  • #18
joshmccraney said:
Wow, that was a lot. I'm trying to follow you: so are you saying the space ##V## is constructed from the basis you present, and that the shortest distance from a point ##x## to ##V## is a straight line with shortest length, and therefore must be orthogonal to all other basis for ##n \neq 1##?

Then is seems like ##\int_0^1 P_n P_1 \, dx = 0## implies ##P_1## is orthogonal to the rest of ##V##?
I edited my previous post while it was in your cache. We don't know enough yet about the properties of the ##P_n##. I think orthogonality is easy to show, but I haven't done any property checks for them. I would try to show that ##w## is optimal (2nd option). We know that ##P_1## can be expanded to a orthonormal system, which should do.
 
  • #19
joshmccraney said:
(2) is the correct problem (though it seems fresh_42 solved (1) too). In general I am not sure how to solve this sort of Lagrange multiplier integral equation. I'm happy to learn another technique, though this seems similar to what fresh_42 wrote.

My guess is Lagrange multiplier implies solving the two equations $$\int_0^1 u(x) \, dx = 0\\
(u(x) + x)^2 = \lambda u(x)$$ but this doesn't feel right since the integrals have boundaries.

What you have is similar to a constrained calculus-of-variations problem, but without any ##u'## terms present in the integral. You could write ##I = \int_0^1 [(x + u(x))^2 - \lambda u(x) ] \, dx## and look for an unconstrained minimum of ##I##. However, if I were doing it I would proceed much differently, as follows.

If you think of "discretizing" the problem (approximating the integrals by finite sums) you will have a problem of the form
$$\begin{array}{cl}
\min & \sum_{i=1}^n (x_i + u_i)^2 \\
\text{such that}& \sum_{i=1}^n u_i = 0
\end{array}$$
Here, ##x_2, x_2, \ldots, x_n \in [0,1]## are equally-spaced points we use to approximate the integral by a sum, and the ##u_i## are supposed to be ##u(x_i)##. Basically, we are replacing the integral ##\int_0^1 F(x) \, dx## by ## \sum_{i=1}^n \Delta x F(x_i),## then dropping the (constant) ##\Delta x,## which will not affect the position of the minimum.

Note that the ##u_i## are just some independent variables to be determined. Before we optimize they are independent of the ##x_i##, but after optimizing they might become functions of the ##x_i##. After we determine the ##u_i## we can get information about ##u(x)## by setting ##u(x_i) = u_i.##

The finite optimization problem above has an elementary solution in terms of the Lagrange multiplier method, so one can easily obtain a discrete version of ##u(x).## This is then easy to extend to the continuous domain.
 
Last edited:
  • #20
Thank you both for responding. I am going to submit what I have so far (just don't have time to look further into what you both suggested, though I will when I get time, as I love learning new methods). As such I'm marking this thread solved. I will post the professors solution (if he gives one).

Thanks again for taking the time to help! I really appreciate your expertise!
 

1. What is a minimization operator problem?

A minimization operator problem is a mathematical optimization problem where the goal is to find the minimum value of a given function with respect to a set of variables. It is used in various fields such as economics, engineering, and computer science to solve real-world problems.

2. How is a minimization operator problem different from a maximization problem?

A minimization operator problem seeks to find the minimum value of a function, while a maximization problem seeks to find the maximum value. In other words, a minimization problem is trying to minimize the cost or error, while a maximization problem is trying to maximize the profit or accuracy.

3. What are some common techniques used to solve minimization operator problems?

Some common techniques used to solve minimization operator problems include gradient descent, Newton's method, and the simplex method. These methods involve iteratively updating the variables to approach the minimum value of the function.

4. How is a minimization operator problem used in machine learning?

In machine learning, a minimization operator problem is often used to train models by minimizing the cost or error function. This involves adjusting the model's parameters to improve its performance on a given dataset.

5. Can a minimization operator problem have multiple solutions?

Yes, a minimization operator problem can have multiple solutions. This can occur when there are multiple local minima in the function, and the algorithm used to solve the problem gets stuck in one of them. To find the global minimum, different starting points or more advanced optimization techniques may be necessary.

Similar threads

  • Calculus and Beyond Homework Help
Replies
2
Views
276
  • Calculus and Beyond Homework Help
Replies
3
Views
163
  • Calculus and Beyond Homework Help
Replies
10
Views
1K
  • Calculus and Beyond Homework Help
Replies
6
Views
272
  • Calculus and Beyond Homework Help
Replies
15
Views
1K
  • Calculus and Beyond Homework Help
Replies
5
Views
1K
  • Calculus and Beyond Homework Help
Replies
1
Views
599
  • Advanced Physics Homework Help
Replies
11
Views
1K
  • Calculus and Beyond Homework Help
Replies
1
Views
663
  • Calculus and Beyond Homework Help
Replies
3
Views
487
Back
Top