Smoothing algorithm for times series and slope measurement

AI Thread Summary
For analyzing time series data with frequent fluctuations, using smoothing algorithms like splines or LOESS can help identify overall trends while accommodating smaller movements. The choice of method depends on the intended use; splines may risk overfitting if used for predictive modeling, while LOESS can effectively connect data points for visualization. To measure the slope of the smoothed line, applying a second LOESS model with a larger smoothing parameter can capture long-term trends. It’s important to evaluate whether the minor fluctuations in the data are significant for your analysis. Selecting the right smoothing technique is crucial for accurate interpretation of the underlying trends.
nickmath
Messages
1
Reaction score
0
Hi,

I'm doing an experiment that gathers one data point approximately each second. I have plotted all the datapoints on a graph. The graph has many tops and bottoms. Now due to the fact that there are so many data points and that I can visualle tell that there are several overal trends: one period my data is going up, the next period it is level again for a period of time and the next period it can be down again. And in each of these periods the data can go up, down or level again, although the overal direction is up, down or level. I have searched for different smoothing methods that can follow all these small movements, but at the same time (using a second smoothing algorithm/parameter) can follow the overal movement. I was thinking about using splines or lowess. Are splines or lowess able to do this?

Second problem I'm having is, after having applied a smoothing algorithm, I need to be able to measure the slope of the smoothing line. I need to know what the slope/curvature of that smoothing line is at any giving moment in time. All the pointers you can give me are really welcome. Thanks.

Kind regards,

Nick
 
Physics news on Phys.org
Foremost: I am an undergraduate CS student and therefore my advise should be taken with a skeptical ear.

I think it depends on what you want to use the smoothed values for.
If you want to do some sort of predictions or make a general model of the data I think what you want to do is model the data with Minimum Description Length using the order of a polynomial and a polynomial that fits the data. Doing a spline will probably overfit your data if this is what you want to smooth for.

If you want to make a nicer visualization of the data, spline might do the trick for you. I'm not really sure though - I'm hesitant to agree that it would do a very good job.
 
Are you sure that all of these little ups and downs are meaningful data? I guess LOESS would be as good as any method for connecting the dots. If you are interested in long-term trends, you could just fit another LOESS model, with a larger smoothing parameter.
 
I was reading documentation about the soundness and completeness of logic formal systems. Consider the following $$\vdash_S \phi$$ where ##S## is the proof-system making part the formal system and ##\phi## is a wff (well formed formula) of the formal language. Note the blank on left of the turnstile symbol ##\vdash_S##, as far as I can tell it actually represents the empty set. So what does it mean ? I guess it actually means ##\phi## is a theorem of the formal system, i.e. there is a...
Back
Top