Understanding the Real Inner Product in Linear Algebra and Fourier Analysis

[member 428835]

Sure, this could also appropriately be placed under linear alg. this being said, can anyone give me an intuitive explanation for the real inner product? i realize it as:

$$<f(x),g(x)>\doteqdot\int_a^bf(x)g(x)dx$$

where i can think of this as an "infinite" dot product along a to b. geometrically (and intuitively) what does this represent about two functions, say in $$R^3$$ for ease of visualization.

for what its worth this has shown up in a Fourier analysis course

 

[Simon Bridge]

It is the same as a dot product for discrete-valued vectors only for continuous ones.
http://en.wikipedia.org/wiki/Dot_product#Inner_product

It may help if you examined specific vectors for f and g ...

 

[HallsofIvy]

A little more generally, we define the dot product specifically on the vector space Rⁿ by $\sum_{i= 1}^n u_iv_i$, the sum of the products of corresponding components (or on Cⁿ as $\sum_{i=1}^n u_iv_i^*$ where "*" indicate the complex conjugate). In any vector space, V, we say that an "inner product" is function from VxV to the underlying field (real numbers or complex numbers) satisfying
1) <v, v> is a positive real number.
2) <au+ bv, w>= a<u, w>+ b<v, w>
3) <u, v>= <v, u> if the underlying field is the real numbers, <u, v> is the complex conjugate of <v, u> if the underlying field is the complex numbers.

It is easy to show that the dot products in Rⁿ and Cⁿ do satisfy those and are "inner products". It is also fairly easy to show that any n dimensional vector space, of dimension n, is isomorphic to Rⁿ, the specific isomorphism given by a selected ordered basis. And so any inner product is equivalent to a dot product, given a specific basis.

But, as I said, that requires a finite dimension which is why Simon Bridge said "It is the same as a dot product for discrete-valued vectors only for continuous ones." The "infinite dimensional vector spaces" typically used are the "function spaces".

 

[chiro]

You might want to review the Riemann Integral and see the analog between that infinite sum and the infinite sum of a vector in l^2 (little l^2 as opposed to big L^2).

 

[homeomorphic]

Well, vaguely, you could say it's sort of measuring how much the two functions are "in synch" with each other, much as the ordinary dot product measures how much two vectors are "in synch" with each other. The dot product of vectors can be interpreted as projecting one vector onto the other and then multiplying the length of the the projection with the length of the vector being projected to. So, in some sense, it's measuring how much one vector points in the direction of the other, with longer vectors being given more weight.

What you really need to get an intuition for is not so much the inner product, but the function space--the idea that functions can be thought of as some sort of vectors because an inner product is something you do to vectors. One way to say it is that functions can be added and multiplied by scalars. Roughly, speaking vectors are things that can be added together and multiplied by scalars. That's probably a bit unsatisfying, though.

One thing that might be a little more satisfying is the realization that if you look at a finite vector subspace of the space of functions, you get the vector subspaces that we know and love, namely R^n. Remembering that we are thinking of functions as vectors, we could take the the span, that is all linear combinations, of the functions, 1, x, and x^2. We get all the second degree (or lower) polynomials. There are 3 parameters, so it is a 3-dimensional space. So, it's R^3, a vector space we know and love.

As another example, you could take the interval from 0 to 1, divide it into n pieces, and consider all the functions that are constant on each piece. We get R^n.

Well, I'm getting tired, but if you keep thinking along those lines, you'll see how the inner product of functions is related to the one in R^n. Just think of doing a discrete approximation.

 

[Simon Bridge]

vaguely, you could say it's sort of measuring how much the two functions are "in synch" with each other

Oooh - that's good: I'll have to use it.
Agree with comments on vector spaces.