Updating PDF for anomaly detection in seasonal time series using FFT

[wess80]

Hi,

I am currently investigating environmental time series and algorithms to determine when an 'unexpected' event/reading has occurred in the series. I am currently constructing the gaussian probability density function (pdf) based on historical readings and checking if 'new' readings are acceptable according to a threshold in the pdf. As time goes by the pdf threshold becomes too large. Also, my time series contains daily and seasonal components.

I have received advice to use spectral analysis (using an FFT) on new month's data to compute the new month's periodicities. I have done so and now am faced with the question of how and what to do with the periodic frequencies in order to help determine what/when an unexpected reading is (according to the distribution).

I was thinking that perhaps it is possible to update a time series' pdf according to it's Fourier transform?

Could some one guide on this.

Thanks,

Wess

 

[fresh_42]

I can only tell what economists usually do. They clean the data from seasonal effects, i.e. they norm the data. If you know the seasonal (monthly) effects, you can divide the data by the norm of those effects. However, if e.g. the seasonal effects are of the form $A+B(t)$ and current data are $C(t)$, i.e. a constant socket $A$ and fluctuations $B(t)$ on top, then you will only use $B(t)$ for normalization: $C_0(t):= A+ \dfrac{C(t)-A}{B(t)}.$