- #1

- 55

- 4

You are using an out of date browser. It may not display this or other websites correctly.

You should upgrade or use an alternative browser.

You should upgrade or use an alternative browser.

- B
- Thread starter waves and change
- Start date

In summary, the conversation discusses the geometric interpretation of orthogonal functions and their advantages in representing a function as a linear combination of other functions. There are three possibilities for representing a function, which depend on the shapes of the graphs of the functions involved. Using orthogonal functions eliminates the possibility of getting multiple representations of a function. Additionally, there is a pictorial argument for the cross correlation between two sinusoidal functions, which supports the mathematical proof.

- #1

- 55

- 4

Mathematics news on Phys.org

- #2

Science Advisor

- 7,859

- 1,591

Suppose we want to represent a function f(x) as a linear combination of the sum of three functions a(x),b(x),c(x). There are 3 possibilities

1) f(x) cannot be represented as a linear combination of a(x), b(x), c(x).

2) f(x) can be represented as a linear combination of a(x), b(x), c(x) in more than one way

3) f(x) can be represented as a linear combination of a(x), b(x), c(x) in exactly one way.

It's fair to say that these possibilities depend on the shapes of the graphs of f(x), a(x), b(x), c(x). So there is a geometric aspect to situation. I don't think there is a simplistic way to visualize this geometric aspect.

The advantage of using orthogonal functions a(x), b(x), c(x) is that if it comes down to a choice between case 2) and case 3), we get get case 3). If we don't have orthogonal functions, we might have a situation such as

5a(x) +4b(x) = c(x). Then if we have f(x) = 5a(x) + 4b(x) + 10 c(x), we also have f(x) = 11 c(x), getting two different representations of f(x).

The general idea is that a function cannot be expressed as a linear combination of other functions that are orthogonal to it. So (intuitively) if we express a function f(x) as a linear combination of functions that are orthogonal to each other, we don't have to worry about getting two different representations of f(x).

If a(x) = sin(2x), you can "see" that a(x) cannot be represented as a linear combination of sin(1x) and sin(3x) because the periods won't work out. Perhaps the best we can do as far as seeing a set of functions are mutually independent is to perceive some reason why each one cannot be written as a linear combination of the others. "Orthogonal" is, in a manner of speaking, an extreme case of independence. One might think of orthogonal functions as being "uncorrelated".

1) f(x) cannot be represented as a linear combination of a(x), b(x), c(x).

2) f(x) can be represented as a linear combination of a(x), b(x), c(x) in more than one way

3) f(x) can be represented as a linear combination of a(x), b(x), c(x) in exactly one way.

It's fair to say that these possibilities depend on the shapes of the graphs of f(x), a(x), b(x), c(x). So there is a geometric aspect to situation. I don't think there is a simplistic way to visualize this geometric aspect.

The advantage of using orthogonal functions a(x), b(x), c(x) is that if it comes down to a choice between case 2) and case 3), we get get case 3). If we don't have orthogonal functions, we might have a situation such as

5a(x) +4b(x) = c(x). Then if we have f(x) = 5a(x) + 4b(x) + 10 c(x), we also have f(x) = 11 c(x), getting two different representations of f(x).

The general idea is that a function cannot be expressed as a linear combination of other functions that are orthogonal to it. So (intuitively) if we express a function f(x) as a linear combination of functions that are orthogonal to each other, we don't have to worry about getting two different representations of f(x).

If a(x) = sin(2x), you can "see" that a(x) cannot be represented as a linear combination of sin(1x) and sin(3x) because the periods won't work out. Perhaps the best we can do as far as seeing a set of functions are mutually independent is to perceive some reason why each one cannot be written as a linear combination of the others. "Orthogonal" is, in a manner of speaking, an extreme case of independence. One might think of orthogonal functions as being "uncorrelated".

Last edited:

- #3

Science Advisor

Gold Member

- 28,641

- 6,711

Here is a 'geometrical' and pictorial argument to support the mathematical one. Plot two equal amplitude sinusoids along the same axis (from - infinity to + infinity. (If they have periods of n and m where n and m are rational and different, you can limit the range to the repeat length of the 'beat'). Take any point along the axis and examine the two values (amplitudes) and there will always be another point, somewhere along the axis, with the same two amplitudes but a difference in sign. So the the sum of the products of that pair values over the total interval will be zero. The sum of all those produces will be zero. If n=m, the sinusoids are always in step and there is no cancellation.waves and change said:is there geometric interpretation/picture of this phenomena?

This is just an arm waving description of the cross correlation between the two functions.

Last edited:

- #4

- 55

- 4

- #5

- 55

- 4

Stephen Tashi said:

1) f(x) cannot be represented as a linear combination of a(x), b(x), c(x).

2) f(x) can be represented as a linear combination of a(x), b(x), c(x) in more than one way

3) f(x) can be represented as a linear combination of a(x), b(x), c(x) in exactly one way.

It's fair to say that these possibilities depend on the shapes of the graphs of f(x), a(x), b(x), c(x). So there is a geometric aspect to situation. I don't think there is a simplistic way to visualize this geometric aspect.

The advantage of using orthogonal functions a(x), b(x), c(x) is that if it comes down to a choice between case 2) and case 3), we get get case 3). If we don't have orthogonal functions, we might have a situation such as

5a(x) +4b(x) = c(x). Then if we have f(x) = 5a(x) + 4b(x) + 10 c(x), we also have f(x) = 11 c(x), getting two different representations of f(x).

The general idea is that a function cannot be expressed as a linear combination of other functions that are orthogonal to it. So (intuitively) if we express a function f(x) as a linear combination of functions that are orthogonal to each other, we don't have to worry about getting two different representations of f(x).

If a(x) = sin(2x), you can "see" that a(x) cannot be represented as a linear combination of sin(1x) and sin(3x) because the periods won't work out. Perhaps the best we can do as far as seeing a set of functions are mutually independent is to perceive some reason why each one cannot be written as a linear combination of the others. "Orthogonal" is, in a manner of speaking, an extreme case of independence. One might think of orthogonal functions as being "uncorrelated".

Thanks for the insight but nonetheless thinking of functions as being uncorrelated doesn’t really do justice geometrically. You can easily picture 2 vectors being uncorrelated by having a 90 degree angle between them. I’m trying to imagine an analogue with regards to sine functions ...

Thank you

- #6

Science Advisor

- 7,859

- 1,591

waves and change said:You can easily picture 2 vectors being uncorrelated by having a 90 degree angle between them..

If

I’m trying to imagine an analogue with regards to sine functions ...

Just imagine the sine functions approximated by data at 1000 points.

Perhaps what you mean is that we can easily visualize a pair of 2 or 3 dimensional vectors as being "uncorrelated" when there is a 90 degree angle between them (?).

- #7

- 55

- 4

Stephen Tashi said:Ifyoucan picture that for cases like a pair of thousand-dimensional vectors, then you're in business!

Just imagine the sine functions approximated by data at 1000 points.

Perhaps what you mean is that we can easily visualize a pair of 2 or 3 dimensional vectors as being "uncorrelated" when there is a 90 degree angle between them (?).

You used the word uncorrelated I’m trying to see if there is any geometric insight here. Imagining a sine function at 1000 points ? Don’t see what information that conveys ?

- #8

Science Advisor

Gold Member

- 28,641

- 6,711

You wanted a geometric description. Personally I would rather stick with the Maths - which can easily be proved - and that is the "quantitative" way. My answer would have to be that, for a graphical demonstration, you could take two sinusoids (sketched) and, by inspection (as they say) you can pick a point on the x-axis and scan across to find another point that will satisfy the requirement of same amplitudes and opposite sign. Pick f and 2f for a start and the picture shows what's going on clearly. It's not rigorous but simple geometrical proofs seldom are. I'm not sure why you feel the need to go to deeply into the geometrical way when the standard textbook analytical approach to deriving the FT does it all.waves and change said:

- #9

- 55

- 4

sophiecentaur said:You wanted a geometric description. Personally I would rather stick with the Maths - which can easily be proved - and that is the "quantitative" way. My answer would have to be that, for a graphical demonstration, you could take two sinusoids (sketched) and, by inspection (as they say) you can pick a point on the x-axis and scan across to find another point that will satisfy the requirement of same amplitudes and opposite sign. Pick f and 2f for a start and the picture shows what's going on clearly. It's not rigorous but simple geometrical proofs seldom are. I'm not sure why you feel the need to go to deeply into the geometrical way when the standard textbook analytical approach to deriving the FT does it all.

Because I don’t just read a textbook see a proof and robotically memorize it. Geometric interpretations can give deep insight when you “ take a step back “ in my opinion. Thank you for the “ geometric insight “ any sources you know of that give insight into this?

- #10

Science Advisor

Gold Member

- 28,641

- 6,711

I agree that robotically memorising a proof is totally pointless. I suspect this is a cop out and that you don't actually want to get to grips with this. You will never have a good grasp of Physics is you are not prepared to address it in the language that is best suited to it (Maths). I am sure that you use Maths at some level and that you actually find it useful. It is unlikely that you use a pictorial method for working out your personal budget and that you use Arithmetic or even algebraic equations for much of Science.waves and change said:Because I don’t just read a textbook see a proof and robotically memorize it.

Calculus and the transforms that it allows you to perform will give a depth of understanding that pictures never will.

- #11

- 55

- 4

sophiecentaur said:I agree that robotically memorising a proof is totally pointless. I suspect this is a cop out and that you don't actually want to get to grips with this. You will never have a good grasp of Physics is you are not prepared to address it in the language that is best suited to it (Maths). I am sure that you use Maths at some level and that you actually find it useful. It is unlikely that you use a pictorial method for working out your personal budget and that you use Arithmetic or even algebraic equations for much of Science.

Calculus and the transforms that it allows you to perform will give a depth of understanding that pictures never will.

Who said I don’t want to address phenomena mathematically. You don’t know anything about me yet want to make these blanket sarcastic statements about my budget. I love math and studied math and physics in college. I’m wasting my time with someone not worth my time. I found a few good sources on the geometric interpretations of Fourier analysis.

Thanks anyway

- #12

Science Advisor

Gold Member

- 28,641

- 6,711

No need to take offence. The fact that you talked in terms of learning like a robot implied to me that you didn't want to appreciate the beauty of the way transforms work. You wrote as if it held no magic for you.waves and change said:Who said I don’t want to address phenomena mathematically. You don’t know anything about me yet want to make these blanket sarcastic statements about my budget. I love math and studied math and physics in college. I’m wasting my time with someone not worth my time. I found a few good sources on the geometric interpretations of Fourier analysis.

Thanks anyway

You did, in your first post.waves and change said:Who said I don’t want to address phenomena mathematically.

- #13

- 55

- 4

- #14

- 55

- 4

sophiecentaur said:No need to take offence. The fact that you talked in terms of learning like a robot implied to me that you didn't want to appreciate the beauty of the way transforms work. You wrote as if it held no magic for you.

You did, in your first post.

Oh said in my post that I don’t want to address phenomena mathematically? Haha I guess English isn’t your forte ... I said simply that I was looking for a geometrical interpretation of a specific topic.. Fourier transform. If you can’t differentiate between those two statements... you have a problem ...

- #15

Science Advisor

Gold Member

- 28,641

- 6,711

If you really are into the Maths then why did you talk about memorising a proof like a robot? I don't know of any robots with an insight into Maths (yet). I'm sorry but I can only respond to what your write.waves and change said:Oh said in my post that I don’t want to address phenomena mathematically? Haha I guess English isn’t your forte ... I said simply that I was looking for a geometrical interpretation of a specific topic.. Fourier transform. If you can’t differentiate between those two statements... you have a problem ...

Perhaps we should stop bickering now. I am glad you have found what you wanted.

- #16

- 55

- 4

- #17

Science Advisor

- 7,859

- 1,591

waves and change said:You used the word uncorrelated I’m trying to see if there is any geometric insight here. Imagining a sine function at 1000 points ? Don’t see what information that conveys ?

That confirms what I suspected. When youj said it was easy to tell if two vectors were uncorrelated, you meant vectors in 2 or 3 dimensions.

You are probably familar with doing linear regression. For example if X = height and Y = weight and we have such paired data from a sample of people, we can ask what the "best" way to approximate Y as AX + B, meaning what are the values of A and B that minimize the square error ##\sum (Y_i - AX_i - B)^2 ## If the best value of A = 0 then X and Y are uncorrelated.

Consider the two 3-dimensional vectors X = (1,4,2) and Y = (6,-1,-1). Treat them as paired data (1,6), (4,-1), (2,-1). If you plot these points perhaps you can have some geometric intuition that the best value of A is A = 0. (i.e a horizontal regression line does better than a sloped one).

I haven't tried to do a similar thing for 1000 points of two functions. It might be revealing. For example for ##f(x_i) = sin(x_i)## and ##g(x) = sin(2 x_i)##, what does a plot of the data ##(f(x_i), g(x_i))## look like for evenly spaced ##x_i## in ##[0,2\pi]##?

- #18

- 55

- 4

Thank you! I will attempt this later on!

- #19

Mentor

- 37,009

- 13,709

It makes sense to have them orthogonal here - it is the most symmetric case.

- #20

Science Advisor

- 2,297

- 793

If you are up for a final math exam (at the master level) you better have memorized the main point the proof. The proof of several theorems have been painstakingly worked out by exceptional mathematicians over the course of several years. I do not think that you will be able to come up with your own proof at the spur of the moment standing before an examination board.waves and change said:Because I don’t just read a textbook see a proof and robotically memorize it.

- #21

Gold Member

- 223

- 47

waves and change said:

This picture might help show how the signal amplitude is plotted against frequency and time. This is from a Google Image search.

- #22

- 55

- 4

Svein said:If you are up for a final math exam (at the master level) you better have memorized the main point the proof. The proof of several theorems have been painstakingly worked out by exceptional mathematicians over the course of several years. I do not think that you will be able to come up with your own proof at the spur of the moment standing before an examination board.

What relevance this post had? ...None

- #23

- 55

- 4

Stephen Tashi said:That confirms what I suspected. When youj said it was easy to tell if two vectors were uncorrelated, you meant vectors in 2 or 3 dimensions.

You are probably familar with doing linear regression. For example if X = height and Y = weight and we have such paired data from a sample of people, we can ask what the "best" way to approximate Y as AX + B, meaning what are the values of A and B that minimize the square error ##\sum (Y_i - AX_i - B)^2 ## If the best value of A = 0 then X and Y are uncorrelated.

Consider the two 3-dimensional vectors X = (1,4,2) and Y = (6,-1,-1). Treat them as paired data (1,6), (4,-1), (2,-1). If you plot these points perhaps you can have some geometric intuition that the best value of A is A = 0. (i.e a horizontal regression line does better than a sloped one).

I haven't tried to do a similar thing for 1000 points of two functions. It might be revealing. For example for ##f(x_i) = sin(x_i)## and ##g(x) = sin(2 x_i)##, what does a plot of the data ##(f(x_i), g(x_i))## look like for evenly spaced ##x_i## in ##[0,2\pi]##?

Thank you for the insight in your posts which steered me in the right direction and I was able to get a grasp of the geometrical aspect after finding a few good online papers. Thanks again!

This thread can be closed

Share:

- Replies
- 7

- Views
- 1K

- Replies
- 26

- Views
- 1K

- Replies
- 12

- Views
- 2K

- Replies
- 2

- Views
- 2K

- Replies
- 3

- Views
- 952

- Replies
- 12

- Views
- 837

- Replies
- 13

- Views
- 2K

- Replies
- 7

- Views
- 1K

- Replies
- 4

- Views
- 2K

- Replies
- 15

- Views
- 621