Optimal Approach for Analyzing Non-Smooth Experimental Data

[t.m.p.c]

Hi,

I need your help. From experiments I got data set which I need to analyze. The problem is that my data is not smooth. I tried to fit my data using a polynomial equation, but the fitting was not good enough. I also tried to smooth, spline... but got very different final results. Can anyone tell me which will be the best way (the more credible) to analyze my data?
Thanks in advance.

Regards,

tmpc

 

[Hurkyl]

Question #1: Does theoretically analyzing the data source suggest how the results should be distributed?

Question #2: How are you judging whether or not the fit is good enough?

Question #3: Do you have all of the variables accounted for?

Question #4: Have you tried anything exponential or logarithmic?


Some bad news: at this point, since you've been trying lots of things, it might not be possible to credibly analyze your data^*. Finding a fit after a dozen false starts is far less significant than finding a fit on the first try. It might be that the best you can do is, once you find something that fits your current data, and then do a new experiment to (in)validate how well it works.

Maybe you can do some tricks to salvage this dataset, like using 75% of the data to find a fit and 25% to (in)validate it... but I'm not the person who can judge such things.



*: More accurately, it might not be possible to draw any credible conclusions. The analysis "we tried X, Y, and Z to analyze the data, without success" is definitely credible and accurate. It is important to know things like "these variables aren't linearly related", although it's not a flashy result.