I Orthogonal Basis of Periodic Functions: Beyond Sines and Cosines

[QuantumCuriosity42]

TL;DR Summary

Investigating if sines and cosines are the only orthogonal basis for periodic functions that generate the entire function space.

Hello everyone,

I've been delving deep into the realm of periodic functions and their properties. One of the fundamental concepts I've come across is the use of sines and cosines as an orthogonal basis for representing any functions. This is evident in Fourier series expansions, where any function can be represented as a sum of sines and cosines of different frequencies.

However, this leads me to a question: Are sines and cosines the only orthogonal basis for periodic functions that can generate the entire function space? Specifically, can we consider other periodic functions, like square pulses or triangular waves of various frequencies, as potential candidates for an orthogonal basis?

Would love to hear insights and if there are any relevant literature or studies that have explored this topic further.

Thank you in advance!

 

[anuttarasammyak]

You can expand a periodic function as a series of any complete orthogonal set of base functions. Sinusoidal base function is an example often used.

 

[QuantumCuriosity42]

You can expand a periodic function as a series of any complete orthogonal set of base functions. Sinusoidal base function is an example often used.

To clarify, I was referring to expanding any function (whether periodic or not) in terms of an orthogonal basis of periodic functions.

 

[WWGD]

Note these are Schauder bases, not Hamel bases, in that we deal with convergence, given these are infinite sums. Those not only Linear Algebra but Topology is also involved.

 

[anuttarasammyak]

To clarify, I was referring to expanding any function (whether periodic or not) in terms of an orthogonal basis of periodic functions.

If periodic, the function is expresed as series, e.g. Fourier series.
If not periodic, the function is expressed as integral. e.g. Fourier integral.

 

[QuantumCuriosity42]

If periodic, the function is expresed as series, e.g. Fourier series.
If not periodic, the function is expressed as integral. e.g. Fourier integral.

Yes, but my question is what other basis are apart from the one Fourier transform uses (sine and cosine).

 

[DaveE]

Laplace and Wavelet come to mind, but I think there are many.

 

[QuantumCuriosity42]

Laplace and Wavelet come to mind, but I think there are many.

Those don't use a basis of periodic functions.

 

[WWGD]

Wouldn't Parsevals Theorem apply?

 

[QuantumCuriosity42]

Wouldn't Parsevals Theorem apply?

What do you mean?

 

[anuttarasammyak]

Yes, but my question is what other basis are apart from the one Fourier transform uses (sine and cosine).

How about this set as an exasmple ?

 

[QuantumCuriosity42]

How about this set as an exasmple ?
View attachment 334783

Is that set ortogonal and generates the space of all possible functions? I don't know. Is there any proof like there is for the Fourier transform basis?
Moreover, I would prefer to find a basis of continous functions like sin and cos.

 

[anuttarasammyak]

Is that set ortogonal and generates the space of all possible functions? I don't know. Is there any proof like there is for the Fourier transform basis?
Moreover, I would prefer to find a basis of continous functions like sin and cos.

I just give you a hint or suggestion which might be of your interest (I hope so). It is your problem and you can do it.

For smooth functions you like, instead of sin x and cos x bases, using their combination
\sin x + \cos x
\sin x - \cos x
is an easy way to demonstrate.

 

[QuantumCuriosity42]

I just give you a hint or suggestion which might be of your interest (I hope so). It is your problem and you can do it.

For smooth functions, instead of sin x and cos x, using their combination
\sin x + \cos x
\sin x - \cos x
is an easy way to demonstrate.

Thanks, but I don't have a level of math good enough to prove that.
By the way, it is not a problem, just that my question originated because I wonder why in physics we always decompose waves in sinusoids, when maybe there are other valid basis of periodic functions. That is, there is no "fundamental frequency", the frequency depends in the basis of periodic functions used.

 

[anuttarasammyak]

when maybe there are other valid basis of periodic functions.

To clarify your queations what other canditates are in your mind ?

 

[QuantumCuriosity42]

To clarify your queations what is an example in your mind ?

I don't know an example basis different from cos(kx) with k being a real number. Which is the one the Fourier transform uses (ignoring the imaginary part).
But my question was precisely the example you are asking me to provide, or I am not understanding you correctly. But for example, maybe all triangular (or pulse) shaped functions of different frequencies also form a basis, I don't know.

 

[anuttarasammyak]

Other than Trigonometric Functions, Some example of periodic base functions in physics are Bessel Functions, Legendre Polynomials, Chebyshev Polynomials, Hermite Polynomials and Walsh Functions. You can see what they are and how they are used in websites.

I think among these functions, trigonometric fundtion is most easy and most universal due to its simple mathematics of
\frac{d}{dx}e^{ikx}=ik e^{ikx}
[\frac{d^2}{dx^2}+k^2]e^{ikx}=0
what is solution of equation of motion of harmonic oscillator.

[EDIT]My bad, Bessel functions and others are not periodic itself.

 

[QuantumCuriosity42]

Other than Trigonometric Functions, Some example of periodic base functions in physics are Bessel Functions, Legendre Polynomials, Chebyshev Polynomials, Hermite Polynomials and Walsh Functions. You can see what they are and how they are used in websites.

I think among these functions, trigonometric fundtion is most easy and most universal due to its simple mathematics of
\frac{d}{dx}e^{ikx}=ik e^{ikx}
[\frac{d^2}{dx^2}+k^2]e^{ikx}=0
what is solution of equation of motion of harmonic oscillator.

I think all the functions you listed are not periodic ?
And even if they were, why some things in nature work in terms of the frequencies of sines (like E=h*f), even if trigonometric function are easy to work with. It does not make sense that nature chose the same thing that is easy or convenient for us.

 

[anuttarasammyak]

In a map North-South axis and East-West axis is given and we appoint number coordinates to a place.
We can use another orothogonal axis, e.g. NW-SE axis and NE-SW axis and appoint another number coorinate to the same place. The both maps work equally. Convenience or whch we are familiar with is another issue.

We have a same situation in expansion of functions. sinusoidal bases are common in use but any orther bases, #11 as an example, work equally.

 

[anuttarasammyak]

With two different base of $\{\phi_n\},\{\psi_n\}$, F is expanded as
F(x)=\sum_n p_n\phi_n(x)
F(x)=\sum_n q_n\psi_n(x)
and base function is also expressed by the other base,
\phi_n(x)=\sum_m r_{nm}\psi_m(x)
So
F(x)=\sum_n p_n\sum_m r_{nm}\psi_m(x)=\sum_m \sum_n p_n r_{nm}\psi_m(x)=\sum_n(\sum_m p_m r_{mn} )\psi_n(x)
Thus
q_n=\sum_m r_{mn} p_m

We can go from one expansion to the other in this way. As an example $\{\phi_n\}$ is sinusoids ,$\{\psi_n\}$ is base in #11.

 

[DaveE]

For smooth functions you like, instead of sin x and cos x bases, using their combination
sin⁡x+cos⁡x
sin⁡x−cos⁡x
is an easy way to demonstrate.

Thanks, but I don't have a level of math good enough to prove that.

This is the Hartley transform which only uses real numbered coefficients for real valued functions.

 

[DaveE]

I think among these functions, trigonometric fundtion is most easy and most universal due to its simple mathematics

Yes. The sinusoids are eigenfunctions for linear systems, which is VERY convenient. This allows you to do spectral analysis at single frequencies.

 

[QuantumCuriosity42]

Yes. The sinusoids are eigenfunctions for linear systems, which is VERY convenient. This allows you to do spectral analysis at single frequencies.

Could you explain that more deeply? Why the basis of #11 doesn't allow you to do spectral analysis at single frequencies?

 

[QuantumCuriosity42]

How about this set as an exasmple ?
View attachment 334783

Well, now that I notice, those functions aren't orthogonal? For example the dot product between 1 and 3 is less than 0.

 

[DaveE]

Could you explain that more deeply? Why the basis of #11 doesn't allow you to do spectral analysis at single frequencies?

Because your basis functions contain different frequencies which LTI systems respond differently to. Each response wouldn't have just a single gain and phase, but an infinite set of them for each harmonic frequency too. I suppose you could do it with a bunch of math. Since the output is no longer a simple modification of the input, then you'll have to deconvolve the whole mess. Then you're back to using sinusoids. I guess the phrase "single frequency" kind of requires sinusoids by definition.

But, honestly, I haven't thought much about this. Maybe in a square wave world the engineers understand the square wave domain like we understand the frequency domain. We are trained with delta functions, step functions and sinusoids because the math is easy. Each illuminates system behaviors in ways we understand, maybe just because that's what we are used to.

 

[marcusl]

To expand on Dave’s post, sines and cosines are the eigenfunctions of linear time-invariant systems, and, since this class of system is ubiquitous, Fourier analysis is everywhere.

There are other classes of periodic function for more complex systems, like the elliptic functions that are doubly periodic.

 

[anuttarasammyak]

Well, now that I notice, those functions aren't orthogonal? For example the dot product between 1 and 3 is less than 0.

You are right. My bad. Thanks.
Insted of my failure attempt, a good example of bases for periodic function other than sinusoids is Legendre polynomials which is illustrated in https://upload.wikimedia.org/wikipe...mials6.svg/360px-Legendrepolynomials6.svg.png

I think all the functions you listed are not periodic ?

[EDIT]My bad, Bessel functions and others are not periodic itself.

To make it periodic is not difficult. For an example new Pn(x) would be
P_n(2(\frac{x+1}{2}-[\frac{x+1}{2}])-1)
so that it is periodic outside of [-1,1], where [ ] is floor function. Normalization is easily done also. Completeness is proved though it is complicated to me.

And even if they were, why some things in nature work in terms of the frequencies of sines (like E=h*f), even if trigonometric function are easy to work with. It does not make sense that nature chose the same thing that is easy or convenient for us.

That discussion on nature seems beyond OP which is a question on mathematics: Investigating if sines and cosines are the only orthogonal basis for periodic functions that generate the entire function space. The answer is we have plural bases other than sinusoids.

 

[QuantumCuriosity42]

You are right. My bad. Thanks.
Insted of my failure attempt, a good example of bases for periodic function other than sinusoids is Legendre polynomials which is illustrated in https://upload.wikimedia.org/wikipe...mials6.svg/360px-Legendrepolynomials6.svg.pngTo make it periodic is not difficult. For an example new Pn(x) would be
P_n(2(\frac{x+1}{2}-[\frac{x+1}{2}])-1)
so that it is periodic outside of [-1,1], where [ ] is floor function. Normalization is easily done also. Completeness is proved though it is complicated to me.
That discussion on nature seems beyond OP which is a question on mathematics: Investigating if sines and cosines are the only orthogonal basis for periodic functions that generate the entire function space. The answer is we have plural bases other than sinusoids.

So if we have more possible basis of periodic functions that generate the entire function space, why is it that in nature for example, the colors of light depend on the frequencies of the armonic decomposition (in sines) of the wave? It is too much of a coincidence and it does not make any sense to me.
Also, that basis of periodic legendre polynomials to decompose functions has a well known name like Fourier transform? Or can you reference the proof of completnees you talked about?

 

[QuantumCuriosity42]

Because your basis functions contain different frequencies which LTI systems respond differently to. Each response wouldn't have just a single gain and phase, but an infinite set of them for each harmonic frequency too. I suppose you could do it with a bunch of math. Since the output is no longer a simple modification of the input, then you'll have to deconvolve the whole mess. Then you're back to using sinusoids. I guess the phrase "single frequency" kind of requires sinusoids by definition.

But, honestly, I haven't thought much about this. Maybe in a square wave world the engineers understand the square wave domain like we understand the frequency domain. We are trained with delta functions, step functions and sinusoids because the math is easy. Each illuminates system behaviors in ways we understand, maybe just because that's what we are used to.

That is interesting to know, but LTI systems are more of an engineering rather than physics concept right?
It looks like some nature properties like I said in my previous comment also depend on armonic frequencies.
Could you provide some references about how LTI systems only behave linearly with sines?

 

[anuttarasammyak]

Also, that basis of periodic legendre polynomials to decompose functions has a well known name like Fourier transform? Or can you reference the proof of completnees you talked about?

https://en.wikipedia.org/wiki/Legendre_polynomials would give you an indroductory ideas.

 

[QuantumCuriosity42]

https://en.wikipedia.org/wiki/Legendre_polynomials would give you an indroductory ideas.

But that completeness is only proved on [-1, 1] interval, I don't know if it is valid to extend it to (-inf, +inf), and I don't think it is valid to repeat that polinomial more times to fill the (-inf, +inf) interval. And even if we did that, then how do you change the frequency of the extended Legendre polynomial to the full interval (-inf, +inf)?
To decompose a function in terms of fundamental frequencies you need to prove ortogonality between all versions of the same function but each time with a different frequency. For example, cos(mx) is ortogonal with cos(nx), for all values of n. That is why it forms a basis that can be used in the Fourier transform.

 

[anuttarasammyak]

For any priodic function of
f(x)=f(x+L)
we can concentrate our investigation on [0, L] and forget outside because they behave same with inside. The period [-1,1] for Legendre polynomials is easily transformed to cover [0, L].

Why you say$(-\infty,+\infty )$? Function of $L=\infty$ is not periodic.

 

[QuantumCuriosity42]

For any priodic function of
f(x)=f(x+L)
we can concentrate our investigation on [0, L] and forget outside because they behave the same with inside.
The period [-1,1] for Legendre polynomials to to cover [0, L].
Why you say (-\infty,+infty )? function of $L=\infty$ is no more periodic.

Ah I see, I didn't know that.
I didn't mean to say L=inf, I meant, a Legendre polynomial only is defined on the interval [-1, 1], while cosine function is defined in (-inf, +inf), and we can use any frequency like cos(kx), k being the frequency.
To do the equivalent thing with Legendre poynomials, we should choose only one Legendre polynomial, extend it to (-inf, +inf), and having a way of changing its frequency, how could we do that? Is it even possible?

All of this with the intention of finding another infinite basis of ortogonal and periodic functions, each of a different frequency, like cos(nx) is ortogonal with cos(mx). To see if there are more or the only one is the one formed by infinites cosines of different frequencies (or sines, but they are the same). These are called the "fundamental frequencies". I want to know if they are called fundamental because they are convenient to work with, or really because they are the only basis possible that is ortogonal for all possible versions of the function with different frequencies.

 

[QuantumCuriosity42]

In fact, now that I think about it, Legendre polynomial ortogonality is proved between all Legendre polynomials. But I don't think it could be proved in that way I say after choosing one, and proving its ortogonality with different versions of itself, each with a different frequency. In order to have another suitable basis of fundamental frequencies to have a similar "Fourier transform", but with other basis of fundamental frequencies which aren't those of sines.

 

[anuttarasammyak]

To do the equivalent thing with Legendre poynomials, we should choose only one Legendre polynomial, extend it to (-inf, +inf), and having a way of changing its frequency, how could we do that? Is it even possible?

I have already shown

To make it periodic is not difficult. For an example new Pn(x) would be
Pn(2(x+12−[x+12])−1)
so that it is periodic outside of [-1,1], where [ ] is floor function.

which is periodic with period L=2 and is defined in $(-\infty,+\infty)$.
I have no idea of "frequency" you say.

 

[QuantumCuriosity42]

I have already shown

which is periodic with period L=2 and is defined in $(-\infty,+\infty)$.

What is n, and what is x there? Can you explain what that expression shows?
I don't see a way to change its frequency.

 

[anuttarasammyak]

What is n, and what is x there? Can you explain what that expression shows?
I don't see a way to change its frequency.

n is number attached to Legendre polunomials

Graph of the variable of polynomials

So the graph of Legendre Polynomial for n=7 is

You see it periodic in all the region.

 

[QuantumCuriosity42]

n is number attached to Legendre polunomials

Graph of the variable of polynomials
View attachment 334829

So the graph of Legendre Polynomial for n=7 is

View attachment 334830

You see it periodic in all the region.

Ah I see now, thanks. But I still can't see how you could change its frequency with an argument of the function, like cos(kx), being k the frequency.

 

[anuttarasammyak]

Ah I see now, thanks. But I still can't see how you could change its frequency

I have no idea of "frequency" you say.

 

[QuantumCuriosity42]

@anuttarasammyak What I mean is that, we can change the frequency of a cosine ( cos(k*x) ) just by changing its parameter k (it is its frequency), cos(m*x) is ortogonal with cos(n*x), for all m not equal to n.
That is why we can use them as a basis formed by infinite cosines each with a different frequency in the Fourier transform.
What I asked was how can we decompose all possible functions with another basis of periodic functions which are not cosines. And you proposed to use a Legendre polynomial, with that extension to (-inf, +inf), but we lack the ability to change its frequency, like we do for cos(k*x). And for them to form a basis it would also be needed to prove the ortogonality between that extendend polynomial and all possible versions of it with different frequencies.
My question is that I think there is no way to change the frequency of a Legendre polynomial (or the extended one you proposed), like we do for cos(k*x), where k is the frequency and we can change it.

 

[anuttarasammyak]

My question is that I think there is no way to change the frequency of a Legendre polynomial (or the extended one you proposed), like we do for cos(k*x), where k is the frequency and we can change it.

Fourier series, in words of https://en.wikipedia.org/wiki/Fourier_series , has discrete set of frequencies or more precisely wave number of
\{2\pi n /P\}
Their numbers are countable infinite as well as n of Legendre polynomials. What' the difference you feel ?

 

[QuantumCuriosity42]

Fourier series, in words of https://en.wikipedia.org/wiki/Fourier_series , has discrete set of frequencies or more precisely wave number of
{2\pi n /P}
Their numbers are countable infinite as well as n of Legendre polynomials. What' the difference you feel ?

I am not talking about Fourier series, which are only valid to decompose periodic functions. I am refering to Fourier transform, which lets us decompose any function, wheter periodic or not.

With respect to Legendre polynomials, I am not talking about forming a basis with all Legendre polynomials, I mean to do it just with one of them. But with infinite versions of that one, and each one being the same original chosed polynomial but with different frequency. The same way that in Fourier Transform the basis is formed by infinite versions of the same cosine function, but each one with a different frequency.

 

[anuttarasammyak]

Now I know your interest is beyond OP topic of

Orthogonal Basis of Periodic Functions: Beyond Sines and Cosines​

 

[QuantumCuriosity42]

Now I know your interest is beyond OP topic of

Orthogonal Basis of Periodic Functions: Beyond Sines and Cosines​

Should I create a new thread?

 

[hutchphd]

One of the few useful pieces I remember from math classes:
Sturm–Liouville theory is the general study of Sturm–Liouville problems. In particular, for a "regular" Sturm–Liouville problem, it can be shown that there are an infinite number of eigenvalues each with a unique eigenfunction, and that these eigenfunctions form an orthonormal basis of a certain Hilbert space of functions.
Most useful functions are found within this rubric IMHO.

 

[DaveE]

LTI systems are more of an engineering rather than physics concept right?

Sorry, I don't really understand the difference; so sure, whatever... But I do see physicists using Fourier Transforms A LOT.

Could you provide some references about how LTI systems only behave linearly with sines?

Nope, that can't be done. Linear systems are linear for any input signal, not just sine waves. Consider that an input function can be expressed as a Fourier series (or transform), which is composed of sinusoids. If it's linear for sinusoids, it's also linear for linear combinations of sinusoids, which is nearly everything.

 

[marcusl]

That is interesting to know, but LTI systems are more of an engineering rather than physics concept right?
It looks like some nature properties like I said in my previous comment also depend on armonic frequencies.

LTI applies in physics as well. The propagation of light through any linear time-invariant medium (air, glass or colored filters) is an example. IWe can demonstrate that sines and cosines are the eigenfunctions (the natural modes or natural ways to characterize) an LTI system:
The output y(t) of an LTI system is given by the convolution of an input x(t) with the system's impulse response h(t) according to$$y(t)=x(t)*h(t)=\int^\infty_{-\infty}{x(t-\tau)h(\tau)d\tau}$$h on the right is independent of the time of day t, hence the characterization that the system is not time varying.
Now let the input be a wave of a single frequency $$x(t)=x(ω,t)=A(\omega)exp(i\omega t)$$where A is a complex number, so that x(t) consists of both a sine and cosine wave of the same frequency ω but of independent amplitudes. Then
$$y(t)=A(\omega)exp(i\omega t)\int^\infty_{-\infty}{exp(-i\omega \tau)h(\tau)d\tau}$$The integral is just the Fourier transform of the impulse response H(ω), which is called the frequency response of the system. Thus $$y(t)=H(ω)x(ω,t)$$We say that the complex exponential (sine/cosine, if you prefer) function x(ω,t) is a characteristic function of the LTI system and H(ω) is the corresponding characteristic value; these are also called eigenfunctions and eigenvalues. The frequency response or eigenvalue spectrum H(ω) completely characterizes the response of the system to any arbitrary input because every input can be decomposed into sines and cosines via Fourier transformation, and the system response to each frequency component is known from the frequency response. With more advanced math, it can be shown that this is the only set of eigenfunctions for an LTI system and that they are all orthogonal to each other.

The connection to light color is that light is a collection of electromagnetic waves, which are sines and cosines by definition. Propagation media such as air, glass or colored filters are LTI so decomposing light into its constituent frequencies (colors) and applying the frequency response function for the medium gives the output.

Another class of physical system is linear and spatially (rather than temporally) non-varying. They are treated mathematically in the same way but using spatially varying waves and spatial frequency responses.

 

[WWGD]

Well, now that I notice, those functions aren't orthogonal? For example the dot product between 1 and 3 is less than 0.

This is not really a problem, as Gram-Schmidt allows us to orthogonalize a basis, even an infinite one. In an inner-product space, of course.

 

[QuantumCuriosity42]

One of the few useful pieces I remember from math classes:
Sturm–Liouville theory is the general study of Sturm–Liouville problems. In particular, for a "regular" Sturm–Liouville problem, it can be shown that there are an infinite number of eigenvalues each with a unique eigenfunction, and that these eigenfunctions form an orthonormal basis of a certain Hilbert space of functions.
Most useful functions are found within this rubric IMHO.

I don't see what that has to do with my question about a basis formed by infinite versions of the same periodic function, each one with a different frequencies (like cos(kx)).

 

[hutchphd]

The point is that this is not unique, and should not therefore be expected a priori to be particularly special. You need to broaden your view.