Comparing HADCRU, GISS and UAH Temperature Records

[joelupchurch]

I loaded the monthly temperature anomalies from HADCRU, GISS and UAH into a single Excel spreadsheet to make it easier to compare them. The result look like the attached image below:

I thought the spreadsheet might be handy for other people, so I've attached it as a zipped Excel file. It also includes the CO2 data from Mauna Loa. I've included links to where I got the data from. What you see in the image is a 12 month moving average and a linear trend for each data set. All three datasets start in December 1978, since that is when the UAH dataset starts.

It is interesting to see how much you can manipulate the linear trend by changing the start date for the graph. In Excel, right click on the graph and click on Select Data. In the Select Data Source dialog go to th Chart Data Range where it says ='Combine'!$A$1:$D$374. Change $1 to whatever row you want to start with.

 

[sylas]

I loaded the monthly temperature anomalies from HADCRU, GISS and UAH into a single Excel spreadsheet to make it easier to compare them. The result look like the attached image below:

That's a handy tool. I'd like to extend it a bit, to do a bit more automatic calculation and also some simple error bounds. I'd just use the normal regression errors without adjusting for autocorrelation, which would be too much work. I may have a shot and post the result.

It's worth noting that GISS and HADCRU are slightly different algorithms for measuring the same basic quantity, but that UAH is actually measuring a different quantity entirely. The former two are surface anomalies; the latter is an anomaly for the lower troposphere. The UAH estimate is often compared with a similar product from RSS (Remote Sensing Systems).

Having these in a spreadsheet is very handy for doing all kinds of things. If you just want a plot you can also try Wood for Trees for an online tool that allows all kinds of plot generation; and also the http://www.remss.com/msu/msu_data_validation.html#compare website for comparing RSS and UAH. But like you, I like having my own spreadsheets.

Cheers -- sylas

PS. Just so people new to this know... the plots are "anomalies", or a difference from some mean value. However, each series uses a different baseline for this difference. This is not a problem, because the absolute value of an anomaly is pretty irrelevant. What matters are the comparisons between one year and another within the same series, or the trends. Shifting a whole anomaly series up, or down, doesn't have any significance, so there's nothing unusual about one plot seeming to be above or below another. The difference in the trends is significant, however, and represents a real difference between the different datasets.

 

[sylas]

That's a handy tool. I'd like to extend it a bit, to do a bit more automatic calculation and also some simple error bounds. I'd just use the normal regression errors without adjusting for autocorrelation, which would be too much work. I may have a shot and post the result.

Done.

I have made a new speadsheet based on Joel's work. Unfortunately, I had to delete a lot of stuff to keep within the size limits for an attachment. So all the charts are gone, and there's only only sheet, now called "Regression", which compares the three datasets. I've updated the name with a "-v3", and removed the "-CO2" from the name as well.

All the calculations of expected trends are based on doing the regression in the worksheet, rather than relying on the numbers from the trend line in a graph. This means we can also calculate the standard errors on trend.

The sheet is protected. It has two green cells, which are the only ones where you can enter data without unprotecting the sheet. (Feel free to unprotect and modify some more!)

You can enter confidence limits (currently 95%) and a date in the future (currently 2100).

The sheet estimates the gain in temperature from the end of the data (2009.8333) up to the given date (2100) which is very close to what was given originally using the slopes transcribed from the graph. It also calculates the standard errors (uses Excel's builtin "linest" function to do linear regression).

It should be easy to combine this new functionality into the previous sheet.

There are two very important caveats with using these estimates.

We are extending a trend far beyond the end of data

This is not usually a useful thing to do, unless you have some very good reason to think there really is a linear trend that will be continued all the way to 2100. Of course, climate is not that simple.

An actual physics based estimate would need a "scenario" for climate forcings, and make estimates based on that. Exploring the scenarios, and the physics for applying them to climate, is another topic, and I don't propose to divert this thread into evaluating such projections.

But we should remember the limited relevance of projecting an estimate of linear trend.

It can be a useful thing to do, if you recognize the limits of our estimates here. For example, you could compare a linear projection against a physical model, and figure out whether the model is expecting trend to increase, or decrease, as the centuury continues.

We are assuming variation is random noise

The error bounds for simple regression analysis assume that the data is some unknown linear trend plus random noise above and below the trend. However, these time series obviously have strong "autocorrelation", which means that if one month is above the trend, then the next month is more likely to be above the trend as well. The months are not independent of each other, but follow other more complex short term cycles. Given this, the actual errors on trend are substantially larger than calculated from the simple regression model.

Even so, at least by giving some error bounds we get a bit more of an idea of how trend is only approximate.

With these caveats in mind:

The 95% confidence limits on trend in degrees per decade for the three datasets are:

• 0.158 +/- 0.014 (for Hadcrut)
• 0.179 +/- 0.020 (for GISS)
• 0.127 +/- 0.020 (for UAH)

As before, remember that UAH is actually measuring changes in the troposphere, where the other two are measuring changes at the surface, and so UAH is not directly comparable to the other two. Hadcrut trend estimates are a little bit smaller than GISS, because this dataset does not cover quite as much of the globe; and so misses the strong warming at present in the Arctic.

If we assume a simple underlying linear trend all the way up to 2100, we obtain the following 95% confidence limits for expected temperature gain from the end of the data up to 2100:

• 1.42 +/- 0.15 {+/- 0.29} (for Hadcrut)
• 1.61 +/- 0.21 {+/- 0.40} (for GISS)
• 1.15 +/- 0.22 {+/- 0.41} (for UAH)

Note that these are estimates for the climatology at that time; temperatures in a given month will tend to range above and below the climatology. This can be estimated also (although it is not in the sheet) and I have provided those bounds within the curly brackets.

The revised spreadsheet itself is an attachment.

A spreadsheet like this can be a useful tool to explore various ideas. There are all kinds of ways something like this can be extended to see other aspects of the data.

I am not entirely sure how our guidelines apply in a case like this; but no strong claims are being made, and I believe a lot can be learned by exploring data yourself; so I'm building on Joel's contribution to show some aspects of data analysis.

Cheers -- sylas

 

[joelupchurch]

That's a handy tool. I'd like to extend it a bit, to do a bit more automatic calculation and also some simple error bounds. I'd just use the normal regression errors without adjusting for autocorrelation, which would be too much work. I may have a shot and post the result.

It's worth noting that GISS and HADCRU are slightly different algorithms for measuring the same basic quantity, but that UAH is actually measuring a different quantity entirely. The former two are surface anomalies; the latter is an anomaly for the lower troposphere. The UAH estimate is often compared with a similar product from RSS (Remote Sensing Systems).

Having these in a spreadsheet is very handy for doing all kinds of things. If you just want a plot you can also try Wood for Trees for an online tool that allows all kinds of plot generation; and also the http://www.remss.com/msu/msu_data_validation.html#compare website for comparing RSS and UAH. But like you, I like having my own spreadsheets.

Cheers -- sylas

PS. Just so people new to this know... the plots are "anomalies", or a difference from some mean value. However, each series uses a different baseline for this difference. This is not a problem, because the absolute value of an anomaly is pretty irrelevant. What matters are the comparisons between one year and another within the same series, or the trends. Shifting a whole anomaly series up, or down, doesn't have any significance, so there's nothing unusual about one plot seeming to be above or below another. The difference in the trends is significant, however, and represents a real difference between the different datasets.

I've tried Wood For Trees. The graphs it generates are ugly, but the raw data function is useful since it has the data in a form easy to put in Excel. Converting the data in GISS to an Excel spreadsheet was a pain, because they have each month in a separate column.

You can produce nice readable graphs in Excel, but you need to override the defaults which seem to be intended to produce Powerpoint chartjunk. You should also go to the add-ins and install the analysis toolkit and the solver. They don't install by default.

 

[joelupchurch]

One tip. In Windows you can right click on a file and click on Send To and select Compress (zipped) folder. Excel workbooks compress quite nicely.

I think I will add the RSS data and put up an updated spreadsheet when the December numbers are available.