Showing fourier coefficients reduce mean square error best

[eschiesser]

Homework Statement

So I'm supposed to show that a finite Fourier approximation is the optimal approximation for a given function.
I am to suppose we have a given set of functions \phi _k(x),k=1,2,\text{...}N defined on a\leq x\leq b.

They are orthogonal \int _a^b\phi _m(x)\phi _n(x)dx=0 \text{ for } m\neq n

and are normalized \int _a^b\left[\phi _m(x)\right]{}^2dx=1

a general approximation for f(x) in terms of these N functions is
f_{\text{app}} (x)=\sum _{m=1}^N \gamma _m\phi _m(x)

One possible choice of coefficients is the Fourier coefficients defined by:
f_m=\int _a^bf(x)\phi _m(x)dx

The mean square error is defined as:

E=\int_a^b \left[f(x)-f_{\text{app}} (x)\right]{}^2 \, dx=\int_a^b \left[f(x)-\sum _{m=1}^N \gamma _m\phi _m(x)\right]{}^2 \, dx

I am supposed to show that the Fourier coefficients would be the optimal choice of \gamma _m to minimize E

The Attempt at a Solution

Thus far, I have carried out the square in the integrand of the error term, used the idea of orthogonality, and substituted the Fourier coefficients in for gamma, but from there I am stuck! Here is what I have...

E=\int _a^b[f(x)]^2dx-2\int _a^bf(x)\left[\sum _{m=1}^N \int _a^bf(x)\phi _m(x)dx\right]\phi _m(x)dx+\left[\sum _{m=1}^N \int _a^bf(x)\phi _m(x)dx\right]{}^2

From here, I am stuck! is there some kind of simplification that I am missing? this is very frustrating. Any help/nudge would be appreciated.

-Eric

 

[Billy Bob]

Double check your squaring of \left[f(x)-\sum _{m=1}^N \gamma _m\phi _m(x)\right]{}^2 because your last formula for E looks suspect. Did you confuse f_m and gamma_m?



The key trick (eventually) is to use -2\gamma_m f_m+\gamma_m^2 = -f_m^2 + (f_m - \gamma_m)^2

 

[eschiesser]

if I am understanding the problem correctly, which is not an assumption I would bet any substantial amount of money on, I thought that the point was to show that f_m is the best substitution for \gamma_m, e.g. show that f_m=\gamma_m reduces E better than any other substitution for \gamma_m.

So I everywhere I see \gamma_m i want to substitute the expression for f_m and show somehow that this is minimizes E, correct?

 

[Billy Bob]

No, wait until the end to imagine replacing gamma_m with f_m. Do the algebra I suggested and you should end up with an expression

E = expression involving both gamma_m and f_m

If you do enough algebra (correctly), you will see that this E is clearly minimized when gamma_m = f_m.

Sorry I'm being vague. It's not really too hard.