Showing fourier coefficients reduce mean square error best

[eschiesser]

Homework Statement

So I'm supposed to show that a finite Fourier approximation is the optimal approximation for a given function.
I am to suppose we have a given set of functions $$\phi _k(x),k=1,2,\text{...}N$$ defined on $$a\leq x\leq b$$.

They are orthogonal $$\int _a^b\phi _m(x)\phi _n(x)dx=0 \text{ for } m\neq n$$

and are normalized $$\int _a^b\left[\phi _m(x)\right]{}^2dx=1$$

a general approximation for f(x) in terms of these N functions is
$$f_{\text{app}} (x)=\sum _{m=1}^N \gamma _m\phi _m(x)$$

One possible choice of coefficients is the Fourier coefficients defined by:
$$f_m=\int _a^bf(x)\phi _m(x)dx$$

The mean square error is defined as:

$$E=\int_a^b \left[f(x)-f_{\text{app}} (x)\right]{}^2 \, dx=\int_a^b \left[f(x)-\sum _{m=1}^N \gamma _m\phi _m(x)\right]{}^2 \, dx$$

I am supposed to show that the Fourier coefficients would be the optimal choice of $$\gamma _m$$ to minimize E

The Attempt at a Solution

Thus far, I have carried out the square in the integrand of the error term, used the idea of orthogonality, and substituted the Fourier coefficients in for gamma, but from there I am stuck! Here is what I have...

$$E=\int _a^b[f(x)]^2dx-2\int _a^bf(x)\left[\sum _{m=1}^N \int _a^bf(x)\phi _m(x)dx\right]\phi _m(x)dx+\left[\sum _{m=1}^N \int _a^bf(x)\phi _m(x)dx\right]{}^2$$

From here, I am stuck! is there some kind of simplification that I am missing? this is very frustrating. Any help/nudge would be appreciated.

-Eric

 

[Billy Bob]

Double check your squaring of $$\left[f(x)-\sum _{m=1}^N \gamma _m\phi _m(x)\right]{}^2$$ because your last formula for E looks suspect. Did you confuse f_m and gamma_m?



The key trick (eventually) is to use $$-2\gamma_m f_m+\gamma_m^2 = -f_m^2 + (f_m - \gamma_m)^2$$

 

[eschiesser]

if I am understanding the problem correctly, which is not an assumption I would bet any substantial amount of money on, I thought that the point was to show that $$f_m$$ is the best substitution for $$\gamma_m$$, e.g. show that $$f_m=\gamma_m$$ reduces E better than any other substitution for $$\gamma_m$$.

So I everywhere I see $$\gamma_m$$ i want to substitute the expression for $$f_m$$ and show somehow that this is minimizes E, correct?

 

[Billy Bob]

No, wait until the end to imagine replacing gamma_m with f_m. Do the algebra I suggested and you should end up with an expression

E = expression involving both gamma_m and f_m

If you do enough algebra (correctly), you will see that this E is clearly minimized when gamma_m = f_m.

Sorry I'm being vague. It's not really too hard.