How to determine if distributions are correlated?

  • Context: Graduate 
  • Thread starter Thread starter ggk
  • Start date Start date
  • Tags Tags
    Distributions
Click For Summary
SUMMARY

This discussion focuses on determining the correlation between two frequency spurs identified through Fast Fourier Transforms (FFT) in Matlab. The user isolates two spurs, referred to as spur1 and spur2, and computes their respective probability distribution functions (PDFs): PDFspur1, PDFspur2, and PDFspur12. The user proposes that if spur1 and spur2 are statistically independent, then PDFspur12 should equal the convolution of PDFspur1 and PDFspur2. Feedback is sought on this method and alternative approaches for establishing correlation.

PREREQUISITES
  • Understanding of Fast Fourier Transforms (FFT) in Matlab
  • Knowledge of probability distribution functions (PDFs)
  • Familiarity with convolution operations in statistical analysis
  • Basic concepts of signal processing and correlation
NEXT STEPS
  • Research the mathematical properties of convolution in relation to probability distributions
  • Explore statistical independence tests for probability distributions
  • Learn about correlation coefficients and their applications in signal analysis
  • Investigate advanced signal processing techniques for identifying correlations in noisy data
USEFUL FOR

Data analysts, signal processing engineers, and researchers working with frequency analysis and correlation in noisy datasets.

ggk
Messages
2
Reaction score
0
Hi Everyone,

I am analyzing real data using fast-fourier transforms (FFT) in Matlab. The FFT magnitude spectrum show some background noise floor with several sharp spurs popping up high out of the background noise. I need to figure out conclusively which of these spurs are correlated with which other spurs (if any).

To simplify this problem let me just analyze two spurs, to see if they are correlated or not. Let me call them spur1 and spur2. I process the data to obtain three probability distribution functions (PDFs):

1) I isolate spur1 and do an inverse FFT only on spur1 to obtain its' respective real-time waveform (a sinusoid of a certain frequency). I take the PDF of this waveform (PDFspur1).

2) I isolate spur2 and do an inverse FFT only on spur2 to obtain its' respective real-time waveform (a sinusoid of a different frequency). I take the PDF of this waveform (PDFspur2).

3) I isolate both spur1 and spur2 from the rest of the spectrum and take an inverse FFT on the spectrum containing both spur1 and spur2. This results in a real-time waveform whose PDF I'll call PDFspur12.

I want to conclusively determine if spur1 and spur2 are correlated with each other. How do I do it?

One thought I have is, if the distibutions (PFDs) are statistically independent (that is, uncorrelated) then PDFspur12 should EQUAL PDFspur1 CONVOLVED with PDFspur2. If they are NOT equal, then they are not correlated.

I think this is mathematically sound, but I'd appreciate any comments/feedback, especially if you know a better/faster/more conclusive way to determine correlation. -GK
 
Physics news on Phys.org
what do you mean by two signals of different frequencies being correlated?
 
Correlation meaning the spurs share the same origin (source). For example, a square-wave has odd harmonics. If you see these harmonics in the FFT magnitude spectrum, you can tell they are correlated because each spur has the same phase-angle and they are integer multiples of the fundamental frequency. In my case, however, there are so many spurs that phase-angle and multipler (if it's known, which isn't always known in my case) aren't sufficient to determine correlation.
 

Similar threads

  • · Replies 10 ·
Replies
10
Views
3K
  • · Replies 12 ·
Replies
12
Views
2K
  • · Replies 9 ·
Replies
9
Views
2K
  • · Replies 13 ·
Replies
13
Views
3K
  • · Replies 16 ·
Replies
16
Views
5K
  • · Replies 12 ·
Replies
12
Views
5K
  • · Replies 0 ·
Replies
0
Views
3K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 14 ·
Replies
14
Views
4K