Register to reply

What to plot as the errors for a moving average plot of temperature?

by sian130
Tags: average, errors, moving, plot, temperature
Share this thread:
sian130
#1
Feb13-12, 10:53 AM
P: 9
I have been monitoring the temperature of a mirror surface placed outside at night. The temperature is measured at 10s intervals and so, as you can imagine, a plot of the data for one period is quite noisy. I have therefore decided to "smooth" the data by plotting a moving average over a ten-minute time period.

This is where I need some help. I need to plot some error bars. I've tried plotting the standard deviation of the data in the ten-minute time periods used for the moving averages, but it doesn't look right. Because the temperature has, in some cases, remained constant for up to two hours, I have ended up with almost no error bars except for little ovals of errors where the temperature changes significantly.

Would it be better practice to simply plot the uncertainty in the measurement caused by the apparatus instead? The temperature probe I am using has an uncertainty of +/-0.5 degrees. Or is there a way of combining the uncertainty of the measurement with the standard deviation? I just don't think it's correct to have large proportions of my graphs with "zero" errors, especially as I know that there is an uncertainty from the temperature probe.

Any advice, or suggestions for potentially useful books on the subject would be much appreciated.
Phys.Org News Partner Science news on Phys.org
Bees able to spot which flowers offer best rewards before landing
Classic Lewis Carroll character inspires new ecological model
When cooperation counts: Researchers find sperm benefit from grouping together in mice
Stephen Tashi
#2
Feb14-12, 10:40 AM
Sci Advisor
P: 3,255
Unless you formulate a clear probability model for how the data is generated, it is meaningless to talk about "error bars". (It's also meaningless to talk about "error bars" unless you define what an "error bar" is supposed to indicate. For example, if the temperature at time 5200 secs is 21.2 C and I draw error bars at plus or minus 1.4 C around that point, what fact is that supposed to convey? )

Don't let the use of statistics lobotomize your knowledge of the physics of the situation. You didn't explain any physical reason that you decided to use a moving average. Is the moving average supposed to be a better indication of the actual temperature? Is there some physical reason that the mirror can't change temperature as fast as the raw data shows? Can you use the physical data of the situation to establish any bound on how fast the the actual temperature of the mirror can change?

Yes, it is possible to formulate a probability model that would incorporate the known statistics of the sensor errors. To make a good model of those errors can get complicated if the errors are stated as percentages and the magnitude of the measurements varies by an order of magnitude. It is also hard to make a good model of this if the error is almost always in the last significant digit of the sensor reading - meaning the last significant digit of the data to whatever base the raw sensor output gives, not necessarily in the last significant digit of the data once it has been processed to a base 10 number.
sian130
#3
Feb14-12, 11:34 AM
P: 9
Thank you for your response.

Well I'd like to convey the uncertainty in the measurement at each point in the time period, which will be a combination of the uncertainty from the moving average calculation (i.e. the standard deviation) and also the uncertainty associated with the apparatus, which is quoted as being +/-0.5 deg C. This is what I want to convey by using "error bars".

Ideally I wouldn't have plotted a moving average, but the purpose of my graph is compare several different mirror surface temperatures on one graph, so in order to make a clearer comparison between surfaces, I decided to smooth the data to make the graph less noisy. The precise values of temperature every 10s or so are not important to me, rather I'm more interested in showing that, for example, one of the mirrors consistently hovers 3degC above all the others. So the moving average has no physical significance, but as the purpose of the graphs is to portray comparisons in the behaviour of several surfaces it is important that the graph is visually clear.

As for establishing how fast the temperature of the surface can change, I don't know, but I know that it can change by about half a degree or so in 10s! I have started to model the situation in a computer program but I haven't modelled heat transfer in anything smaller than 1 second intervals, as I'm more concerned with changes in temperature on a longer timescales.

How would I go about formulating a probability model? A typical temperature range of the surface is about 5degC (say, from 0degC to -5degC over the course of 12 hours). The temperature probes have an uncertainty as quoted by the manufacturer of +/-0.5degC, valid in the range -60degC to +60degC.

Thank you!

Stephen Tashi
#4
Feb14-12, 09:14 PM
Sci Advisor
P: 3,255
What to plot as the errors for a moving average plot of temperature?

Quote Quote by sian130 View Post
Thank you for your response.

Well I'd like to convey the uncertainty in the measurement at each point in the time period, which will be a combination of the uncertainty from the moving average calculation (i.e. the standard deviation) and also the uncertainty associated with the apparatus, which is quoted as being +/-0.5 deg C. This is what I want to convey by using "error bars".
Let's see if I got that.

[itex] T_m(t_i) = [/itex] measured temperature at time [itex] t_i [/itex]
[itex] A_m(t_i) = \frac{ \sum_{k=i-n}^{i+n} T_m(t_i)}{2n+1} [/itex] = moving average of measured temperatures at [itex] t_i [/itex]
[itex] T(t_i) = [/itex] actual temperature at time [itex] t_i [/itex]
[itex] A(t_i) = \frac{\sum_{k=i-n}^{i+n} T(t_i)} {2n+1} [/itex] = actual moving average temperature at time [itex] t_i [/itex].

We assume
[itex] T_m(t_i) = T(t_i) + E(t_i) [/itex] where the "error" [itex] E(t_i) [/itex] is a random variable (for each [itex] t_i [/itex]).

We assume that [itex] E(t_i) = E_{moth}(t_i) + E_{meas}(t_i) [/itex] where [itex] E_{moth}(t_i) [/itex] is the "error" made by mother nature and [itex] E_{meas}(t_i) [/itex] is the error made by our measurement process.

Let [itex] D(t_i) = A(t_i) - A_m(t_i) [/itex]. You want to estimate the standard deviation of [itex]D(t_i)[/itex] ? You also want to express this standard deviation as a function of two other standard deviations - one due to mother nature and one due to measurement error?



so in order to make a clearer comparison between surfaces
How would I go about formulating a probability model?
Generally speaking you would make a model for the physical process using deterministic equations from physics plus some random variables representing "uncontrolled" physical inputs to these equations. The equations would have various unknown parameters. Then you would investigate whether it is possible to estimate the values of these parameters from the measured data and other known facts.

It is not clear to me whether the goal of the comparison "between surfaces" is intended to be precisely how their temperatures would change with the same inputs or whether the thing being compared is some property (like reflectivity, specific heat etc.) that is not identical to temperature but merely related to measured temperatures. A physical model might clarify what property of the mirror surface concerns you. For example, suppose I am interested in the position of an object and I have data on a varying Force that is applied to it. If I measure the statistics of the force (such as standard deviation) , I can't conclude that the statistics of the position are directly proportional to it. (F = ma, not F = mx). There is some "inertia" to heat transfer isn't there? So if the real quantity of interest is not temperature itself, the equations relating temperature to the quantity would be useful.
sian130
#5
Feb16-12, 09:09 AM
P: 9
Quote Quote by Stephen Tashi View Post
Let's see if I got that.

[itex] T_m(t_i) = [/itex] measured temperature at time [itex] t_i [/itex]
[itex] A_m(t_i) = \frac{ \sum_{k=i-n}^{i+n} T_m(t_i)}{2n+1} [/itex] = moving average of measured temperatures at [itex] t_i [/itex]
[itex] T(t_i) = [/itex] actual temperature at time [itex] t_i [/itex]
[itex] A(t_i) = \frac{\sum_{k=i-n}^{i+n} T(t_i)} {2n+1} [/itex] = actual moving average temperature at time [itex] t_i [/itex].

We assume
[itex] T_m(t_i) = T(t_i) + E(t_i) [/itex] where the "error" [itex] E(t_i) [/itex] is a random variable (for each [itex] t_i [/itex]).

We assume that [itex] E(t_i) = E_{moth}(t_i) + E_{meas}(t_i) [/itex] where [itex] E_{moth}(t_i) [/itex] is the "error" made by mother nature and [itex] E_{meas}(t_i) [/itex] is the error made by our measurement process.

Let [itex] D(t_i) = A(t_i) - A_m(t_i) [/itex]. You want to estimate the standard deviation of [itex]D(t_i)[/itex] ? You also want to express this standard deviation as a function of two other standard deviations - one due to mother nature and one due to measurement error?

Well, all I want is the value of the error, i.e. [itex] E(t_i) = E_{moth}(t_i) + E_{meas}(t_i) [/itex] . In that case, can I simply add the standard deviation (which I am taking to be the error caused by mother nature, i.e. [itex] E_{moth}(t_i) [/itex]) to the measurement error, [itex] E_{meas}(t_i) [/itex] ?


Quote Quote by Stephen Tashi View Post
It is not clear to me whether the goal of the comparison "between surfaces" is intended to be precisely how their temperatures would change with the same inputs or whether the thing being compared is some property (like reflectivity, specific heat etc.) that is not identical to temperature but merely related to measured temperatures.
I'm interested in how their temperatures would change with the same inputs, i.e. with the same weather conditions (e.g. ambient temperature and wind speed). My aim is to see which mirror surfaces are more likely to experience condensation on cold, clear nights. Therefore, if I know how the temperatures are likely to change, then I can predict condensation if the temperature falls below the dew point temperature on a given day. That's why I'm doing the experiment - I'm plotting the temperatures throughout the night and then trying to tweak my model until it simulates the observations. So far I have a model which allows me to vary emissivity, thermal conductivity, mass and area of the mirrors. But my original question just refers to the actual experimental data - I want to present my graphs with error bars which portray the uncertainties in the measurement.
Stephen Tashi
#6
Feb16-12, 11:34 AM
Sci Advisor
P: 3,255
Quote Quote by sian130 View Post
Well, all I want is the value of the error, i.e. [itex] E(t_i) = E_{moth}(t_i) + E_{meas}(t_i) [/itex] . In that case, can I simply add the standard deviation (which I am taking to be the error caused by mother nature, i.e. [itex] E_{moth}(t_i) [/itex]) to the measurement error, [itex] E_{meas}(t_i) [/itex] ?
OK, but the standard deviation of the error [itex] E(t_i) [/itex] isn't a direct indicator of how the moving average curve itself would vary if you could run the experiment over and over again under the same conditions.

You have a situation where it's almost irresistible to do certain arithmetic and this urge would drive most people to make the following assumptions.

1. Assume the [itex] E_{moth}(t_i) [/itex] each have mean zero and are independent and identically distributed random variables for all [itex] t_i [/itex].

2. Assume [itex] E_{meas}(t_i) [/itex] each have mean zero and are independent and identically distributed random variables for all [itex] t_i [/itex].

and we may have to make more assumptions , but the general idea is to use the deviations we can measure [itex] D_m(t_i) = T_m(t_i) - A_m(t_i) [/itex] and argue that the variance of these deviations is approximately the variance of [itex] D(t_i) = T_m(t_i) - A(t_i) [/itex].

Compute [itex] D_m [/itex] for all [itex] t_i [/itex] and consider this one big set of data for the same random variable [itex] D_m [/itex]. Find the variance of [itex] D_m [/itex].

It is the variances of sums of independent random variables that can be computed as sums (not their standard deviations). So [itex] Var(D_m) = Var(D) = Var(E_{moth}) + Var(E_{meas}) [/itex].

Then you have the problem of translating the measurement device specifications into a numerical value for [itex] Var(E_{meas}) [/itex].

If you can do that, you can solve for [itex] Var(E_{moth}) [/itex].


A person with very serious intentions would check things like whether [itex] D_m [/itex] really appears to have mean zero and whether [itex] D_m(t_i) [/itex] really is identically distributed for all times [itex] t_i [/itex]. For example, do the devations tend to be positive when temperatures are falling?


My aim is to see which mirror surfaces are more likely to experience condensation on cold, clear nights.
That's a clear goal. It focuses attention on a certain range of temperatures. I assume your data is for temperatures in that range. But is the question of interest a yes-or-no question? Or is there a more-or-less aspect to the question of condensation?

trying to tweak my model until it simulates the observations. So far I have a model which allows me to vary emissivity, thermal conductivity, mass and area of the mirrors
According to physical laws, should the temperature-vs-environment behavior of the mirror change once some condensation does form on it? Is the formation of "some" condensation something that promotes even more condensation at the same temperature?
sian130
#7
Feb21-12, 11:12 AM
P: 9
Thank you for your response again. I am working through this at the moment...please bear with me!


Register to reply

Related Discussions
Plot 3D matrix as 2D plot and 3rd dimension as color in Mathematica Math & Science Software 8
How to plot multiple iterations of a while loop on one plot? [MATLAB] Math & Science Software 1
Rasterized density plot + countour plot in Mathematica (to solve EPS issues) Math & Science Software 0
Frequency Plot(or bode plot) in SIMULINK Engineering Systems & Design 1
Mathematica- Rotating a 2D plot into a 3D plot Math & Science Software 2