Finding specific points in data array

Click For Summary
SUMMARY

This discussion focuses on finding specific points in a noisy data array, particularly for plotting Temperature versus Time using Python. The user employs a moving average function to smooth the temperature data extracted from a CSV file named 'ANP-Heat-BA1.csv'. To locate the averaged temperature at a specific time point, such as 500 seconds, various methods are suggested, including linear interpolation, polynomial fitting, and spline fitting. The conversation emphasizes the importance of defining a model to handle data points that may not directly correspond to the available time values.

PREREQUISITES
  • Proficiency in Python programming, specifically with libraries like Pandas and NumPy.
  • Understanding of moving averages and their application in data smoothing.
  • Familiarity with interpolation techniques, including linear interpolation and spline fitting.
  • Knowledge of data visualization concepts, particularly using plotting libraries like Matplotlib.
NEXT STEPS
  • Learn how to implement linear interpolation in Python using the SciPy library.
  • Explore polynomial fitting techniques with NumPy's polyfit function.
  • Investigate spline fitting methods using the SciPy.interpolate module.
  • Study data visualization best practices for plotting time series data with Matplotlib.
USEFUL FOR

Data scientists, Python developers, and researchers working with time series data who need to analyze and visualize noisy datasets effectively.

lee403
Messages
16
Reaction score
1
I have this really noisy data and I'm wanting to plot Temperature v. Time. I used this function to calculate a moving average. Here's some sample code.

Python:
BA1='ANP-Heat-BA1.csv'

#Time Trial 1
time1=pd.read_csv(BA1,skiprows=0)
time1=time1['Time(s)']

#Temperature 
tmp1=pd.read_csv(BA1,skiprows=0)
tmp1=tmp1['Temp']

def movingaverage (values, window):
    weights = np.repeat(1.0, window)/window
    sma = np.convolve(values, weights, 'valid')
    return sma
   
tmp1MA=movingaverage(tmp1, 10)

p.plot(time1[len(time1)-len(tmp1MA):],tmp1MA)

The moving average function seems to work fine at removing some of the random noise, but now I want to find the point in the array tmp1MA that corresponds to a given time1 value. For example, I would like be able to find the averaged temperature at time 500.
 
Hey lee403.

You'd have to define a model and then look at the expectation of that random variable.

What model are you using?
 
You have the moving average values, MAi for a specific set of time points ti. Any particular value of time, t, might be exactly one of the ti values, or in between two ti values or above all of the tis or below all of the tis.

There are a few things you need to decide. Do you want to extrapolate above or use the value of the highest ti? Same question for t below the lowest ti. Do you want to linearly interpolate for between tis or fit a polynomial and use that? You can also do a spline fit to the values and evaluate the spline at t.
 

Similar threads

Replies
3
Views
1K
  • · Replies 17 ·
Replies
17
Views
4K
  • · Replies 11 ·
Replies
11
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
Replies
7
Views
3K
  • · Replies 7 ·
Replies
7
Views
8K
  • · Replies 10 ·
Replies
10
Views
26K
  • · Replies 0 ·
Replies
0
Views
2K
  • · Replies 1 ·
Replies
1
Views
1K
  • · Replies 10 ·
Replies
10
Views
11K