What to plot as the errors for a moving average plot of temperature?

In summary: The author has been monitoring the temperature of a mirror surface placed outside at night. The temperature is measured at 10s intervals and so, as you can imagine, a plot of the data for one period is quite noisy. The author has therefore decided to smooth the data by plotting a moving average over a ten-minute time period. The smoothing process averages the temperature over the time period, but includes an allowance for sensor error. The author is interested in portraying comparisons in the behaviour of several mirrors, so the use of error bars is important. The author also wants to know how fast the temperature of the mirror can change. To form a probability model for the sensor errors, the author would need to know the range of temperatures that the surface
  • #1
sian130
9
0
I have been monitoring the temperature of a mirror surface placed outside at night. The temperature is measured at 10s intervals and so, as you can imagine, a plot of the data for one period is quite noisy. I have therefore decided to "smooth" the data by plotting a moving average over a ten-minute time period.

This is where I need some help. I need to plot some error bars. I've tried plotting the standard deviation of the data in the ten-minute time periods used for the moving averages, but it doesn't look right. Because the temperature has, in some cases, remained constant for up to two hours, I have ended up with almost no error bars except for little ovals of errors where the temperature changes significantly.

Would it be better practice to simply plot the uncertainty in the measurement caused by the apparatus instead? The temperature probe I am using has an uncertainty of +/-0.5 degrees. Or is there a way of combining the uncertainty of the measurement with the standard deviation? I just don't think it's correct to have large proportions of my graphs with "zero" errors, especially as I know that there is an uncertainty from the temperature probe.

Any advice, or suggestions for potentially useful books on the subject would be much appreciated.
 
Physics news on Phys.org
  • #2
Unless you formulate a clear probability model for how the data is generated, it is meaningless to talk about "error bars". (It's also meaningless to talk about "error bars" unless you define what an "error bar" is supposed to indicate. For example, if the temperature at time 5200 secs is 21.2 C and I draw error bars at plus or minus 1.4 C around that point, what fact is that supposed to convey? )

Don't let the use of statistics lobotomize your knowledge of the physics of the situation. You didn't explain any physical reason that you decided to use a moving average. Is the moving average supposed to be a better indication of the actual temperature? Is there some physical reason that the mirror can't change temperature as fast as the raw data shows? Can you use the physical data of the situation to establish any bound on how fast the the actual temperature of the mirror can change?

Yes, it is possible to formulate a probability model that would incorporate the known statistics of the sensor errors. To make a good model of those errors can get complicated if the errors are stated as percentages and the magnitude of the measurements varies by an order of magnitude. It is also hard to make a good model of this if the error is almost always in the last significant digit of the sensor reading - meaning the last significant digit of the data to whatever base the raw sensor output gives, not necessarily in the last significant digit of the data once it has been processed to a base 10 number.
 
  • #3
Thank you for your response.

Well I'd like to convey the uncertainty in the measurement at each point in the time period, which will be a combination of the uncertainty from the moving average calculation (i.e. the standard deviation) and also the uncertainty associated with the apparatus, which is quoted as being +/-0.5 deg C. This is what I want to convey by using "error bars".

Ideally I wouldn't have plotted a moving average, but the purpose of my graph is compare several different mirror surface temperatures on one graph, so in order to make a clearer comparison between surfaces, I decided to smooth the data to make the graph less noisy. The precise values of temperature every 10s or so are not important to me, rather I'm more interested in showing that, for example, one of the mirrors consistently hovers 3degC above all the others. So the moving average has no physical significance, but as the purpose of the graphs is to portray comparisons in the behaviour of several surfaces it is important that the graph is visually clear.

As for establishing how fast the temperature of the surface can change, I don't know, but I know that it can change by about half a degree or so in 10s! I have started to model the situation in a computer program but I haven't modeled heat transfer in anything smaller than 1 second intervals, as I'm more concerned with changes in temperature on a longer timescales.

How would I go about formulating a probability model? A typical temperature range of the surface is about 5degC (say, from 0degC to -5degC over the course of 12 hours). The temperature probes have an uncertainty as quoted by the manufacturer of +/-0.5degC, valid in the range -60degC to +60degC.

Thank you!
 
  • #4
sian130 said:
Thank you for your response.

Well I'd like to convey the uncertainty in the measurement at each point in the time period, which will be a combination of the uncertainty from the moving average calculation (i.e. the standard deviation) and also the uncertainty associated with the apparatus, which is quoted as being +/-0.5 deg C. This is what I want to convey by using "error bars".

Let's see if I got that.

[itex] T_m(t_i) = [/itex] measured temperature at time [itex] t_i [/itex]
[itex] A_m(t_i) = \frac{ \sum_{k=i-n}^{i+n} T_m(t_i)}{2n+1} [/itex] = moving average of measured temperatures at [itex] t_i [/itex]
[itex] T(t_i) = [/itex] actual temperature at time [itex] t_i [/itex]
[itex] A(t_i) = \frac{\sum_{k=i-n}^{i+n} T(t_i)} {2n+1} [/itex] = actual moving average temperature at time [itex] t_i [/itex].

We assume
[itex] T_m(t_i) = T(t_i) + E(t_i) [/itex] where the "error" [itex] E(t_i) [/itex] is a random variable (for each [itex] t_i [/itex]).

We assume that [itex] E(t_i) = E_{moth}(t_i) + E_{meas}(t_i) [/itex] where [itex] E_{moth}(t_i) [/itex] is the "error" made by mother nature and [itex] E_{meas}(t_i) [/itex] is the error made by our measurement process.

Let [itex] D(t_i) = A(t_i) - A_m(t_i) [/itex]. You want to estimate the standard deviation of [itex]D(t_i)[/itex] ? You also want to express this standard deviation as a function of two other standard deviations - one due to mother nature and one due to measurement error?
so in order to make a clearer comparison between surfaces

How would I go about formulating a probability model?

Generally speaking you would make a model for the physical process using deterministic equations from physics plus some random variables representing "uncontrolled" physical inputs to these equations. The equations would have various unknown parameters. Then you would investigate whether it is possible to estimate the values of these parameters from the measured data and other known facts.

It is not clear to me whether the goal of the comparison "between surfaces" is intended to be precisely how their temperatures would change with the same inputs or whether the thing being compared is some property (like reflectivity, specific heat etc.) that is not identical to temperature but merely related to measured temperatures. A physical model might clarify what property of the mirror surface concerns you. For example, suppose I am interested in the position of an object and I have data on a varying Force that is applied to it. If I measure the statistics of the force (such as standard deviation) , I can't conclude that the statistics of the position are directly proportional to it. (F = ma, not F = mx). There is some "inertia" to heat transfer isn't there? So if the real quantity of interest is not temperature itself, the equations relating temperature to the quantity would be useful.
 
  • #5
Stephen Tashi said:
Let's see if I got that.

[itex] T_m(t_i) = [/itex] measured temperature at time [itex] t_i [/itex]
[itex] A_m(t_i) = \frac{ \sum_{k=i-n}^{i+n} T_m(t_i)}{2n+1} [/itex] = moving average of measured temperatures at [itex] t_i [/itex]
[itex] T(t_i) = [/itex] actual temperature at time [itex] t_i [/itex]
[itex] A(t_i) = \frac{\sum_{k=i-n}^{i+n} T(t_i)} {2n+1} [/itex] = actual moving average temperature at time [itex] t_i [/itex].

We assume
[itex] T_m(t_i) = T(t_i) + E(t_i) [/itex] where the "error" [itex] E(t_i) [/itex] is a random variable (for each [itex] t_i [/itex]).

We assume that [itex] E(t_i) = E_{moth}(t_i) + E_{meas}(t_i) [/itex] where [itex] E_{moth}(t_i) [/itex] is the "error" made by mother nature and [itex] E_{meas}(t_i) [/itex] is the error made by our measurement process.

Let [itex] D(t_i) = A(t_i) - A_m(t_i) [/itex]. You want to estimate the standard deviation of [itex]D(t_i)[/itex] ? You also want to express this standard deviation as a function of two other standard deviations - one due to mother nature and one due to measurement error?


Well, all I want is the value of the error, i.e. [itex] E(t_i) = E_{moth}(t_i) + E_{meas}(t_i) [/itex] . In that case, can I simply add the standard deviation (which I am taking to be the error caused by mother nature, i.e. [itex] E_{moth}(t_i) [/itex]) to the measurement error, [itex] E_{meas}(t_i) [/itex] ?


Stephen Tashi said:
It is not clear to me whether the goal of the comparison "between surfaces" is intended to be precisely how their temperatures would change with the same inputs or whether the thing being compared is some property (like reflectivity, specific heat etc.) that is not identical to temperature but merely related to measured temperatures.

I'm interested in how their temperatures would change with the same inputs, i.e. with the same weather conditions (e.g. ambient temperature and wind speed). My aim is to see which mirror surfaces are more likely to experience condensation on cold, clear nights. Therefore, if I know how the temperatures are likely to change, then I can predict condensation if the temperature falls below the dew point temperature on a given day. That's why I'm doing the experiment - I'm plotting the temperatures throughout the night and then trying to tweak my model until it simulates the observations. So far I have a model which allows me to vary emissivity, thermal conductivity, mass and area of the mirrors. But my original question just refers to the actual experimental data - I want to present my graphs with error bars which portray the uncertainties in the measurement.
 
  • #6
sian130 said:
Well, all I want is the value of the error, i.e. [itex] E(t_i) = E_{moth}(t_i) + E_{meas}(t_i) [/itex] . In that case, can I simply add the standard deviation (which I am taking to be the error caused by mother nature, i.e. [itex] E_{moth}(t_i) [/itex]) to the measurement error, [itex] E_{meas}(t_i) [/itex] ?

OK, but the standard deviation of the error [itex] E(t_i) [/itex] isn't a direct indicator of how the moving average curve itself would vary if you could run the experiment over and over again under the same conditions.

You have a situation where it's almost irresistible to do certain arithmetic and this urge would drive most people to make the following assumptions.

1. Assume the [itex] E_{moth}(t_i) [/itex] each have mean zero and are independent and identically distributed random variables for all [itex] t_i [/itex].

2. Assume [itex] E_{meas}(t_i) [/itex] each have mean zero and are independent and identically distributed random variables for all [itex] t_i [/itex].

and we may have to make more assumptions , but the general idea is to use the deviations we can measure [itex] D_m(t_i) = T_m(t_i) - A_m(t_i) [/itex] and argue that the variance of these deviations is approximately the variance of [itex] D(t_i) = T_m(t_i) - A(t_i) [/itex].

Compute [itex] D_m [/itex] for all [itex] t_i [/itex] and consider this one big set of data for the same random variable [itex] D_m [/itex]. Find the variance of [itex] D_m [/itex].

It is the variances of sums of independent random variables that can be computed as sums (not their standard deviations). So [itex] Var(D_m) = Var(D) = Var(E_{moth}) + Var(E_{meas}) [/itex].

Then you have the problem of translating the measurement device specifications into a numerical value for [itex] Var(E_{meas}) [/itex].

If you can do that, you can solve for [itex] Var(E_{moth}) [/itex].


A person with very serious intentions would check things like whether [itex] D_m [/itex] really appears to have mean zero and whether [itex] D_m(t_i) [/itex] really is identically distributed for all times [itex] t_i [/itex]. For example, do the devations tend to be positive when temperatures are falling?


My aim is to see which mirror surfaces are more likely to experience condensation on cold, clear nights.

That's a clear goal. It focuses attention on a certain range of temperatures. I assume your data is for temperatures in that range. But is the question of interest a yes-or-no question? Or is there a more-or-less aspect to the question of condensation?

trying to tweak my model until it simulates the observations. So far I have a model which allows me to vary emissivity, thermal conductivity, mass and area of the mirrors

According to physical laws, should the temperature-vs-environment behavior of the mirror change once some condensation does form on it? Is the formation of "some" condensation something that promotes even more condensation at the same temperature?
 
  • #7
Thank you for your response again. I am working through this at the moment...please bear with me!
 

1. What is a moving average plot of temperature?

A moving average plot of temperature is a way of visualizing the trend of temperature data over time. It involves calculating the average temperature over a certain period of time and plotting it against the time period, which helps to smooth out any short-term fluctuations and highlight long-term patterns.

2. Why is it important to plot errors for a moving average plot of temperature?

Plotting errors for a moving average plot of temperature is important because it allows us to see the level of uncertainty or variance in the data. This is especially useful when looking at long-term trends, as it helps to assess the reliability of the data and any potential sources of error.

3. What are the different types of errors that can be plotted for a moving average plot of temperature?

The most common types of errors that can be plotted for a moving average plot of temperature include standard error, standard deviation, and confidence intervals. These errors provide information about the range of values within which the true average temperature is likely to fall.

4. How do you determine the appropriate error type to plot for a moving average plot of temperature?

The appropriate error type to plot for a moving average plot of temperature depends on the specific research question and the nature of the data. For example, if the goal is to compare temperature trends between different time periods, standard error or standard deviation may be more suitable. If the focus is on the accuracy of the overall trend, confidence intervals may be more appropriate.

5. Can the errors for a moving average plot of temperature change over time?

Yes, the errors for a moving average plot of temperature can change over time. This is because the errors are based on the data and can be affected by factors such as sample size, measurement accuracy, and changes in the climate. It is important to regularly re-evaluate the errors and adjust the moving average plot accordingly to ensure the most accurate representation of the data.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
781
  • Set Theory, Logic, Probability, Statistics
Replies
18
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
794
  • Set Theory, Logic, Probability, Statistics
Replies
21
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
1K
  • Set Theory, Logic, Probability, Statistics
2
Replies
40
Views
3K
  • Set Theory, Logic, Probability, Statistics
Replies
3
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
19
Views
5K
  • Set Theory, Logic, Probability, Statistics
Replies
5
Views
971
  • Set Theory, Logic, Probability, Statistics
Replies
12
Views
1K
Back
Top