Science vs. Politics: Tipping Points in Climate Change Communication

[Sorry!]

This makes absolutely no sense. The CRU e-mails revealed unethical practices that have taken place. Why would you be expecting "new" science to appear a few weeks later? From what? Please explain what you are talking about because it makes no sense to us.

So you are going to try and tell me that if you were a skeptic of a particular field of science and you had scientific proof which conclusively proves that the data they use was being manipulated (I.e. another model a better method of dealing with data sets) that when it becomes released that a group of the science you are against has been 'manipulating' the data to 'mold scientific venue' that you wouldn't jump all over it with your OWN science. Instead most of what we have is politicians jumping all over it and scientists jumping all over it POLITICALLY.

Nothing scientific has come from it and nothing scientific will probably ever come from it.

 

[Evo]

So you are going to try and tell me that if you were a skeptic of a particular field of science and you had scientific proof which conclusively proves that the data they use was being manipulated (I.e. another model a better method of dealing with data sets) that when it becomes released that a group of the science you are against has been 'manipulating' the data to 'mold scientific venue' that you wouldn't jump all over it with your OWN science. Instead most of what we have is politicians jumping all over it and scientists jumping all over it POLITICALLY.

Nothing scientific has come from it and nothing scientific will probably ever come from it.

You are making absolutely no sense. What on Earth do you mean by "jump all over it with your own science"? Is that what you really think, that climate scientists that say the numbers are fudged by a certain group of other climate scientists that have refused to reveal the data points that they used so that their work can be verified have a "different science". What?

 

[Sorry!]

You are making absolutely no sense. What on Earth do you mean by "jump all over it with your own science"? Is that what you really think, that climate scientists that say the numbers are fudged by a certain group of other climate scientists that have refused to reveal the data points that they used so that their work can be verified have a "different science". What?

seycyrus's claim is hardly that they just are not allowing it to be verified. He is saying that they purposely did it to 'mold a scientific venue'. The point I'm making is that you can't refute that with e-mails you have to refute it with science. Yes the scientists do have a different view on the science... When I say 'with your own science' I think it's implied that I'm speaking of your VIEW of science. You can try to make what I'm saying seem as vague as you possibly can, it makes no difference. The skeptics have nothing new to add to their scientific claims even with this supposed 'knowledge' that data has been 'fudged' for their own 'scientific venue'.

EDIT: This is something that they wouldn't need CRU data to do. They would use their own data and their own models and they would present it as the correct version and show where CRU is wrong. Instead it's just now being assumed on a political level that because the released e-mails that AGW science is wrong. That's not science... This doesn't of course apply to every skeptic out there, there are some that understand this and are still going ahead with their own research.

 

[sylas]

You are making absolutely no sense. What on Earth do you mean by "jump all over it with your own science"? Is that what you really think, that climate scientists that say the numbers are fudged by a certain group of other climate scientists that have refused to reveal the data points that they used so that their work can be verified have a "different science". What?

Just a reminder. The vast majority of raw data is readily available and is more than sufficient to replicate the CRU analysis in the usual scientific sense of the word... with an independent analysis.

The small amount of raw data that cannot be made available is restricted not by the CRU, but by the owners of that data. It is not that scientists are "refusing" to reveal data. It is that some of the data is not theirs to reveal, and they are legally not able to just release it.

This kind of thing is not unusual in science. It's awkward, and the CRU is attempting to get the necessary permissions to make the full set available, but in the meantime the demands for access have been rather weird.

There's more than enough of the raw data readily available to replicate and confirm the basic results, and this has been done already. If anyone still really doesn't like using a dataset where they can't also get hold of every bit of raw data themselves, then I recommend you stick to the GISS data products, which also have the advantage of having slightly better global coverage, as well as full availability of all the raw data and code used to obtain the results.

Cheers -- sylas

 

[Evo]

Just a reminder. The vast majority of raw data is readily available and is more than sufficient to replicate the CRU analysis in the usual scientific sense of the word... with an independent analysis.

The small amount of raw data that cannot be made available is restricted not by the CRU, but by the owners of that data. It is not that scientists are "refusing" to reveal data. It is that some of the data is not theirs to reveal, and they are legally not able to just release it.

This kind of thing is not unusual in science. It's awkward, and the CRU is attempting to get the necessary permissions to make the full set available, but in the meantime the demands for access have been rather weird.

There's more than enough of the raw data readily available to replicate and confirm the basic results, and this has been done already. If anyone still really doesn't like using a dataset where they can't also get hold of every bit of raw data themselves, then I recommend you stick to the GISS data products, which also have the advantage of having slightly better global coverage, as well as full availability of all the raw data and code used to obtain the results.

Cheers -- sylas

Not from what I've read. There was a small set of data points used out of a ton of data. The points were then 'homogenized" to use their term and they claim to not have knowledge of precisely which points were used. If you have an updated statement from CRU stating that they have now found those specific original data points in question, please post it as I can't seem to find that update. Because the issue isn't if tons of data is available, as you know, it's which "specific" points were used in specific studies.

 

[sylas]

Not from what I've read. There was a small set of data points used out of a ton of data. The points were then 'homogenized" to use their term and they claim to not have knowledge of precisely which points were udes. If you have an updated statement from CRU stating that they have now found those specific data points in question, please post it as I can't seem to find that update. Because the issue isn't if tons of data is available, as you know, it's which "specific" points were used in certain studies.

I don't believe that is at all accurate. There was a ton of data and it was all used. The data is homogenized, to use the term used by everyone working on this and throughout the literature. The same terms are used by the groups for whom you have full access to all the data.

You have to back up your assertion about anyone claiming that they don't know exactly what points are used. I don't believe any such claim has been made.

What has been pointed out is that the full database of combined data cannot be released because a small proportion of it is proprietary. It would be too much work to identify and pull out those proprietary points -- the analysis make no distinction between the sources of data, of course. There would be nothing to gain from this, because pulling out the proprietary data leaves you with what you've already got available from other sources yourself. FOI requests are quite properly denied if the information you request is already freely available elsewhere.

To summarize.

• It's misleading to say that the problem is scientists refusing to release data. The data to which you refer is not theirs to release.
• There is plenty of data readily available to confirm the results in the usual scientific sense of the word; by independent replication. This has been done by several groups.
• I think your account of the "issue" is completely mistaken. The issue is NOT "which points were used" but rather a demand to get a open access to the full database. The reasons why you can't have that are not because of recalcitrant scientists.

Cheers -- sylas

 

[Pinu7]

Source? please.

The most trusted one, common sense. If more people are "convinced" of AGW, then there would be more pressure for scientists to provide data to support AGW to get funding. I am NOT saying that there is some super deep international fraud, but the circumstances are special enough to make it suspect.

With all the politically motivated nonscientific "eyes" on it, it is hard to consider climatology as trustworthy as the other sciences.

 

[Evo]

I don't believe that is at all accurate. There was a ton of data and it was all used. The data is homogenized, to use the term used by everyone working on this and throughout the literature. The same terms are used by the groups for whom you have full access to all the data.

You have to back up your assertion about anyone claiming that they don't know exactly what points are used. I don't believe any such claim has been made.

It's already been posted in the CRU hack thread, we discussed this there.

What has been pointed out is that the full database of combined data cannot be released because a small proportion of it is proprietary. It would be too much work to identify and pull out those proprietary points -- the analysis make no distinction between the sources of data, of course. There would be nothing to gain from this, because pulling out the proprietary data leaves you with what you've already got available from other sources yourself. FOI requests are quite properly denied if the information you request is already freely available elsewhere.

To summarize.

• It's misleading to say that the problem is scientists refusing to release data. The data to which you refer is not theirs to release.
• There is plenty of data readily available to confirm the results in the usual scientific sense of the word; by independent replication. This has been done by several groups.
• I think your account of the "issue" is completely mistaken. The issue is NOT "which points were used" but rather a demand to get a open access to the full database. The reasons why you can't have that are not because of recalcitrant scientists.

Cheers -- sylas

No, as I said, we've already discussed this. It's specific data points used in certain studies. That is what the issue is and it's all in the CRU Hack thread.

The specific data points requested were for this particular graph. http://www.cru.uea.ac.uk/cru/info/warming/

 

[Sorry!]

It's already been posted in the CRU hack thread, we discussed this there.

No, as I said, we've already discussed this. It's specific data points used in certain studies. That is what the issue is and it's all in the CRU Hack thread.

The specific data points requested were for this particular graph. http://www.cru.uea.ac.uk/cru/info/warming/

So all of them basically...? I must admit however, I do not recall people specifically only claiming that getting the data for that particular graph was their complaint. It's more about all the data being released.

Regardless if they wanted the 'small' dataset for that graph you posted it makes no difference. Sylas has already gone over why CRU will not release the data and why FOI requests have been, rightly, denied to people. It's not at all that hard to understand.

I guess once CRU obtains permission to release it's data it will all be clear, then how will people complain about CRU?

 

[Evo]

So all of them basically...? I must admit however, I do not recall people specifically only claiming that getting the data for that particular graph was their complaint. It's more about all the data being released.

Regardless if they wanted the 'small' dataset for that graph you posted it makes no difference. Sylas has already gone over why CRU will not release the data and why FOI requests have been, rightly, denied to people. It's not at all that hard to understand.

I guess once CRU obtains permission to release it's data it will all be clear, then how will people complain about CRU?

CRU has not said that they do not have permission to release this data. They said that they didn't keep it, they only kept the "homogenized data".

Also, are you familiar with WMO Resolution 40?

Members should provide to the research and education communities, for their non-commercial activities, free and unrestricted access to all data and products exchanged under the auspices of WMO . .

http://www.nws.noaa.gov/im/wmocovr.htm

Have you read the CRU's website on "availability of data"?

Since the early 1980s, some NMSs, other organizations and individual scientists have given or sold us (see Hulme, 1994, for a summary of European data collection efforts) additional data for inclusion in the gridded datasets, often on the understanding that the data are only used for academic purposes with the full permission of the NMSs, organizations and scientists and the original station data are not passed onto third parties

http://www.cru.uea.ac.uk/cru/data/availability/

They couldn't refuse to release the data to the climate scientist that requested it, so they told him they no longer had it.

 

[sylas]

It's already been posted in the CRU hack thread, we discussed this there.

The hack thread contains an incredible amount of nonsense. What gets "pointed" out often just is not so, or spun in ways that distort what is said. If you are going to make this accusation about what the CRU has stated in THIS thread, give reasonable reference that let's us confirm it here, or clarify precisely what was said.

I repeat... I am quite sure you are mistaken on this. You have said above "they claim to not have knowledge of precisely which points were used".

That's not true. It is true that is not an easy thing to go through the combined database and pick out which points come from the propriety national data, which is a different statement entirely.

The specific data points requested were for this particular graph. http://www.cru.uea.ac.uk/cru/info/warming/

Oh good grief! That's the final data product! In fact it is a simple diagnostic of the final product, showing a global average. You are demonstrating my point.

You are asking then for ALL the points that were used, and I refer you AGAIN to my account above of why you can't have that.

If they DID go through and pick out the 95% of data that is freely available, then you could use that... but you can go and do that NOW. And people HAVE done that, and replicated the same results for that curve to well within the measurement uncertainties.

Indeed, it turns out that you can get within the measurement errors of that curve using as little as 10% of the underlying data. That curve is, after all, merely a global average of the gridded data product. If you use a smaller underlying input dataset -- even a MUCH smaller dataset -- you get significant differences in the gridded data, but the errors tend to even themselves out when you take a mean over the whole globe.

This is reasonably technical. A lot of people are very confused about how it is done. It isn't actually that hard, but you honestly are getting it wrong.

Cheers -- sylas

And added in edit to correct another mistaken assertion:

CRU has not said that they do not have permission to release this data.

They've been explaining this now for years in relation to the FOI requests. The formal statement up at the East Anglia website explains it yet again. Here's an extract.

The University will make all the data accessible as soon as they are released from a range of non-publication agreements. Publication will be carried out in collaboration with the Met Office Hadley Centre.

The procedure for releasing these data, which are mainly owned by National Meteorological Services (NMSs) around the globe, is by direct contact between the permanent representatives of NMSs (in the UK the Met Office).

“We are grateful for the necessary support of the Met Office in requesting the permissions for releasing the information but understand that responses may take several months and that some countries may refuse permission due to the economic value of the data,” continued Professor Davies.

The remaining data, to be published when permissions are given, generally cover areas of the world where there are fewer data collection stations.​

You can find this all over the web, but alas the main UEA site seems to be down at present so I can't link direct to the original. But it's there. Evo is wrong from start to finish over this whole issue.

Cheers -- sylas

PPS. When the UEA website normal service is restored, you should be able to find the press statements at this link: http://www.uea.ac.uk/mac/comm/media/press/CRUstatements.

 

[Evo]

You can find this all over the web, but alas the main UEA site seems to be down at present so I can't link direct to the original. But it's there. Evo is wrong from start to finish over this whole issue.

Cheers -- sylas

I'm afraid it is you who are wrong Sylas, look up, I posted what their agreement for release of information is.

Sylas, you have taken what I said completely out of the context in which it was said and made up a meaningless reply that had nothing to do with what I said. My statement that

CRU has not said that they do not have permission to release this data.

Is obviously to do with the data set I posted, you know this, you even addressed the data set. I suggest you delete your error.

CRU has posted on their own website what I said that they claim to no longer have the data.

Data storage availability in the 1980s meant that we were not able to keep the multiple sources for some sites, only the station series after adjustment for homogeneity issues. We, therefore, do not hold the original raw data but only the value-added (i.e. quality controlled and homogenized) data.

Yet you continue to claim they never said this. As has been discussed, it is inconceivable that a scientist would not keep the data that his study was based on. And, as has been discussed, if they can not provide the original data so that their conclusions can be verified, their "conclusion" needs to be thrown out.

The UEA website is not down for me, I'm on it right now. Read what I posted from their website Sylas. They agree to share their data if it is with other academics, they just won't share with "third parties".

 

[sylas]

Sorry Evo, our posts crossed at the same time.

I have already demonstrated the CRU HAS in fact said that they do not have permission to release the data, so I have backed up my point that you were wrong to say they did not. Is that much clear yet?

Now you appear to be arguing, implicitly, that they were wrong to say that. You are still mistaken on this weaker claim.

CRU has not said that they do not have permission to release this data. They said that they didn't keep it, they only kept the "homogenized data".

Also, are you familiar with WMO Resolution 40?

Yes, I am. It would be nice if everyone followed it. The CRU does; but not all their sources do. It does not change the fact that the CRU does not have permission to release all the data, and they have said so.

Have you read the CRU's website on "availability of data"?

Yes, and it confirms what I am trying to tell you.

They couldn't refuse to release the data to the climate scientist that requested it, so they told him they no longer had it.

Your premise is false. They certainly could refuse to release the data; they are not legally permitted to release the data.

Your inference on motives is false.

The statement from the CRU is an attempt to explain to people how this kind of data accumulation works.

Let's try again.

• There is a lot of raw data collected at weather stations. This data is maintained and accessible from the various national meteorological bodies. Most of it is freely available, in line with WMO resolutions. Some of it is not freely available, however much the WMO, the CRU and the rest of us would like it to be.
  
• The CRU collected this data. This was a massive task, and involved negotiating arrangements with all kinds of bodies that have such data.
  
• All the data is then put together into a single combined data base. As a part of this step there is a standard homogenization procedure, which picks up issues at stations such as change of location, or instruments, and so on, which give small changes and steps in the record.
  
• The combined data base still contains the records from all the various individual stations, although there are sometimes small corrections in the homogenization step. These series for individual stations are not the property of the CRU, and they cannot be released, and they are still the property of the various national bodies. The fact that there are sometimes some small adjustments at this step does not change this.
  
• Consequently, the CRU cannot simply release this database. They ARE negotiating in order to do this. It won't make a scrap of difference to the people who have been asking for it, frankly; and real scientists are already using the freely available data just fine for legitimate independent replications.
  
• The CRU has not kept all the data that was given to it. This is not unusual. The data is not destroyed; it remains curated at the national meteorological bodies that loaned it in the first place. The disposal of data in this way is quite usual and indeed it is often a formal requirement. What they keep is the combined database of station data.
  

Summary.

The CRU has indeed said explicitly that they do not have permission to release the requested data.

The CRU is correct to say this.

The data that has been discarded is not lost; the originals remain curated at various other national meteorological bodies. All the weather station data at the CRU is kept in a single combined database after merging and homogenizing individual records. The homogenization process has been studied, and the small changes that apply do not impact trends; adjustments tend to have a gaussian distribution and are as often in one direction as in another.

The imputations of motives to the CRU above are effectively accusing them of lying, without any justification.

If you do not trust the CRU, that is your prerogative. Slander is not your prerogative. You can, if you prefer, use the GISS temperature record, where all the data and code is available to you.

Cheers -- sylas

Added in edit for a second post.

I'm afraid it is you who are wrong Sylas, look up, I posted what their agreement for release of information is.

No; you posted an explanation of the constraints under which they work.

Sylas, you have taken what I said completely out of the context in which it was said and made up a meaningless reply that had nothing to do with what I said. My statement that Is obviously to do with the data set I posted, you know this, you even addressed the data set. I suggest you delete your error.

I stand by my statement without any hesitation whatsoever, and deny that it is out of context.

I don't think you even understand how the data set you posted is formed. Your request doesn't even make good sense.

• Do you want the tabulated data from which the graph was plotted? It is available.
• Do you want the global gridded data that was averaged to make that table for plotting? Is is available.
• Do you want the combined database of weather station data that is used to construct the global data set? It is not available, because of the problems with sharing arrangements. The CRU is working towards making this combined dataset available, but they are not legally able to do so at present.
• Do you want the raw set of measurements recorded at the individual weather stations? That is not CRU data, and in many cases they are not legally able to give it, even if they have kept a copy. You should get it from the people who maintain it.

 

[Evo]

Once again Sylas, since this doesn't seem to be sinking in for you, CRU, in a letter in response to the request for the data for that graph SPECIFICALLY said that they do not have the original data related to that graph. Are you saying that they lied? I assume you have some information the rest of us are not privy to in this particular case? If you do, and you can prove they did not say that, then I will admit I am wrong. I have the feeling you've been arguing about a case for weeks that you are not even familiar with.

 

[turbo]

Also please consider that the people at the helm of CRU today were NOT in the top positions when raw data was discarded in the 1980s. Is there a single person here in this discussion that was involved in data-collection and analysis in the 1980s? I was. You didn't pass around massive data-sets on floppies. It was always done with tapes, and often the tapes were encoded in some pretty arcane and project-specific formatting.

The suggestion (even outright assertion) that failure to preserve all the source data is proof of collusion, deception, etc, is all over the blogosphere, and it can certainly convince the gullible, but anybody who was involved with massive surveys of historical data back in the 1980s would understand how lame the conspiracy theories are. There is no way that you could possibly share gigabytes (or even terabytes) of data without resorting to duplicating and shipping truckloads of tapes. Once the data on the tapes is re-formatted to fit the database that you are building, do you save all of the original tapes? Businesses and educational institutions do not have unlimited funds to preserve and store such bulky records.

 

[sylas]

Once again Sylas, since this doesn't seem to be sinking in for you, CRU, in a letter in response to the request for the data for that graph SPECIFICALLY said that they do not have the original data related to that graph. Are you saying that they lied? I assume you have some information the rest of us are not privy to in this particular case? If you do, and you can prove they did not say that, then I will admit I am wrong. I have the feeling you've been arguing about a case for weeks that you are not even familiar with.

Evo, I am very familiar with this indeed. I've been using this kind of data myself in an amateur way for a couple of years. This includes raw station data, homogenized station data, gridded global data, and diagnostic means.

The graph you showed is an example of a diagnostic mean for the HadCRUT3 dataset, which is actually put together by the Hadley center, using CRUTEM as one of the inputs.

So it is really very confusing when you say you want the data for that graph. I have a pretty good understanding of all the various steps involved in getting from raw station records to the plot you showed; basically because I have done those same calculations myself, for my own interest, in a different context... to measure the effects of homogenization and so on in the regional anomaly over the USA.

I have a pretty good understanding of what data the CRU can give and what it cannot, and why.

I stand by everything I have said in this recent discussion without hesitation. A lot of the confusion is with lack of clarity as to what is said and what it means. It does take a little bit of background to follow the context which let's it make better sense, but it takes an awful lot of time to explain it, especially if you have to cross reference every step of the way.

It is also off topic for this thread. But basically, the CRU really and truly cannot legally give all the raw station information away; neither can they give the homogenized database away. They are working towards doing the latter. The proper way to get the completely raw station data is from the bodies that own it.

I am not entirely sure whether a thread on this is better in the science forum or the politics forum. As usual, there is an awful lot of basic background to explain what the data is and how it is processed; plus the political issues of sharing; plus the matter of proper scientific conduct with respect to maintaining copies of data. All of these things have been subject to woeful misunderstandings, but trying to explain it all clearly and calmly is getting a bit difficult. It needs its own thread.

By the way, the website that is experiencing difficulty is the website at UEA, which contains the formal statements including what I quoted to demonstrate that yes, contrary to your assertion, the CRU really has said that they do not have permission to release all the data. Whether you believe them or not seems to be a least in part a matter of confusion on the various steps involved in getting from data to graph. But they have said this, and it is true.

The site currently has a message up about normal service being restored soon.

Cheers -- sylas

 

[Sorry!]

they only kept the "homogenized data".

Which is the data that they used to create the chart...

Also, are you familiar with WMO Resolution 40?

I think you need to re-read WMO Resolutions to get a better understanding of what is going on here.

It is not the job of CRU to give out 'raw station data' aside from that it would be illegal in some cases had they done that. The data they DO have they can not currently release due to the exact resolution you cite to try and say they HAVE to. I love how you quote mine the CRU statement:

Since the early 1980s, some NMSs, other organizations and individual scientists have given or sold us (see Hulme, 1994, for a summary of European data collection efforts) additional data for inclusion in the gridded datasets, often on the understanding that the data are only used for academic purposes with the full permission of the NMSs, organizations and scientists and the original station data are not passed onto third parties.

We receive numerous requests for these station data (not just monthly temperature averages, but precipitation totals and pressure averages as well). Requests come from a variety of sources, often for an individual station or all the stations in a region or a country. Sometimes these come because the data cannot be obtained locally or the requester does not have the resources to pay for what some NMSs charge for the data. These data are not ours to provide without the full permission of the relevant NMSs, organizations and scientists.

We are not in a position to supply data for a particular country not covered by the example agreements referred to earlier, as we have never had sufficient resources to keep track of the exact source of each individual monthly value. Since the 1980s, we have merged the data we have received into existing series or begun new ones, so it is impossible to say if all stations within a particular country or if all of an individual record should be freely available.

This is the main points of the CRU statements, not the polticially twisted segment you have picked out.

 

[Evo]

Evo, I am very familiar with this indeed.

Ok, so, who requested the information from CRU that they said they couldn't provide because they didn't have it? Which is what I've been citing, which you have been saying is wrong, that they do have it but can't give it out. If as you say, you are familiar with what I've been saying for the last month, then you also know that all I've said is correct.

 

[Sorry!]

Ok, so, who requested the information from CRU that they said they couldn't provide because they didn't have it? Which is what I've been citing, which you have been saying is wrong, that they do have it but can't give it out. If as you say, you are familiar with what I've been saying for the last month, then you also know that all I've said is correct.

Read my post it answers this.

 

[NeoDevin]

Ok, so, who requested the information from CRU that they said they couldn't provide because they didn't have it? Which is what I've been citing, which you have been saying is wrong, that they do have it but can't give it out.

All I've seen you do in this thread is make claims and fail to cite anything supporting them. You haven't even cited anything supporting the claim that someone requested the information, never-mind the claim that they can't provide it because they don't have it. I just re-read all of your posts in this thread to make sure I didn't miss any citations, and still can't find them. If you have actually cited something supporting your accusations, please quote which post it was in, because I can't find it.

 

[Proton Soup]

from what i remember on the CRU thread, CRU had a bunch of data sets in printed form. at some point, persons at CRU had to manually enter all this data into a computer. but what CRU did not have apparently was computing resources to store (and maybe more accurately, process) all this data. so, they produced a roughened, courser, "homogenized" data set from the original data with which to work. at some point, the printed data was deemed either burdensome or unnecessary and destroyed. presumably, either the computing resources to handle large data sets was still not available, or perhaps rough estimates were simply deemed "good enough", that seems unclear. waiting on simulations to complete is certainly time-consuming, so it is at least one factor to consider.

as for what i don't remember reading, what is this mysterious homogenization algorithm? has anyone ever reproduced the homogenized data set? one would assume data is shared between colleagues and that the basic tenet of science (reproducibility) is adhered to. if not, then we have no validation, a broken chain of custody, and no assurance that results produced from the data (and homogenized data) are valid.

another thing that's curious to me: how many cubic feet of printed hardcopies of original data are we talking about here? an entire room, a broom closet, or a bit of space on one of the researcher's bookshelf?

 

[Sorry!]

The most trusted one, common sense. If more people are "convinced" of AGW, then there would be more pressure for scientists to provide data to support AGW to get funding. I am NOT saying that there is some super deep international fraud, but the circumstances are special enough to make it suspect.

With all the politically motivated nonscientific "eyes" on it, it is hard to consider climatology as trustworthy as the other sciences.

This does not source anything for your comment, specifically:

I think this would explain why some climatologists "hide" data, to get funding.

It doesn't seem logical for common sense to indicate 'climatologists are hiding data to get funding'. I agree with everything you've said except for this and I'd like you to specifically cite where you know data has been 'hidden'.

It's important to note that what I'm asking for as well isn't a paper that has not included specific data then later on has explained why they have not used this data. You can hardly say that is hiding anything.

EDIT: Here's a pdf on quality control:
http://envgen.nox.ac.uk/courses/Microarray_data_quality/dataquality_control.pdf

 

[Proton Soup]

In this it says:

Inhomogeneities are introduced into the station temperature series by such things as changes in the station site, changes in measurement time, or changes in instrumentation. The station data that are used to make HadCRUT have been adjusted to remove these inhomogeneities, but such adjustments are not exact—there are uncertainties associated with them.

For some stations both the adjusted and unadjusted time-series are archived at CRU and so the adjustments that have been made are known [Jones et al., 1985, Jones et al., 1986, Vincent & Gullet, 1999], but for most stations only a single series is archived, so any adjustments that might have been made (e.g. by National Met. services or individual scientists) are unknown.​

ergo, adjustments for most stations are unknown. thanks.

 

[sylas]

ergo, adjustments for most stations are unknown. thanks.

That's right. The paper goes on to calculate the associated uncertainties, and this is part of the standard error estimated in the final data set. I refer you to the paper for details.

If you want to try your hand at homogenization yourself, try using the USHCN network, where you can get hold of all kinds of intermediate files plus the meta-data.

Cheers -- sylas

 

[Evo]

It's here Sylas, I gave the information about this back on 12-01. Russ, Astronuc, DH, we all dicussed the fact the CRU said they didn't have the requested information and you kept going off on a tangent saying that they had it, but weren't allowed to give it out, which had nothing to do with what the rest of us were discussing.

Roger Pielke had requested the data from CRU and had been told that they no longer held the data.

Obviously, the ability to do good research depends upon good data with known provenance. At the time WMO Resolution 40 was widely hailed in the atmospheric sciences community as a major step forward in data sharing and availability in support of both operations and research.

Thus it is with some surprise to observe CRU going through bizarre contortions to avoid releasing its climate data to Steve McIntyre. They first told him that he couldn't have it because he was not an academic. I found this to be a petty reason for keeping data out of the hands of someone who clearly wants to examine it for scholarly purposes. So, wanting to test this theory I asked CRU for the data myself, being a "real" academic. I received a letter back from CRU stating that I couldn't have the data because "we do not hold the requested information."

I found that odd. How can they not hold the data when they are showing graphs of global temperatures on their webpage? However, it turns out that CRU has in response to requests for its data put up a new webpage with the following remarkable admission (emphasis added)

(From CRU's website)
We are not in a position to supply data for a particular country not covered by the example agreements referred to earlier, as we have never had sufficient resources to keep track of the exact source of each individual monthly value. Since the 1980s, we have merged the data we have received into existing series or begun new ones, so it is impossible to say if all stations within a particular country or if all of an individual record should be freely available. Data storage availability in the 1980s meant that we were not able to keep the multiple sources for some sites, only the station series after adjustment for homogeneity issues. We, therefore, do not hold the original raw data but only the value-added (i.e. quality controlled and homogenized) data.

Say what?! CRU has lost track of the original data that it uses to create its global temperature record!? Can this be serious? So not only is it now impossible to replicate or reevaluate homogeneity adjustments made in the past -- which might be important to do as new information is learned about the spatial representativeness of siting, land use effects, and so on -- but it is now also impossible to create a new temperature index from scratch. CRU is basically saying, "trust us." So much for settling questions and resolving debates with empirical information (i.e., science).

http://rogerpielkejr.blogspot.com/2009/08/we-lost-original-data.html

 

[Integral]

Yep, Just like Sylas said? Now where's the problem?

 

[Evo]

Yep, Just like Sylas said? Now where's the problem?

Like Sylas said when? Sylas disputes that this ever happened. He disputes that CRU ever said that (that's from CRU's website, BTW). What on Earth are are you talking about Integral? Please explain.

 

[Evo]

I object to you telling me what I "know", and I continue to assert that I am doing my best to give you honest information in good faith. I continue to stand by what I have said in the thread, and am happy to explain it, but I would prefer you to assume my good faith in this. If you cannot assume my good faith, say so frankly.

We should be able to disagree and work towards whatever common ground is possible without this kind of projection about what the other person really thinks. It's disruptive of civil and robust debate. You do this much too often.

Not nearly as bad as your petty attempts at trying to discredit me. Let's agree to stop the personal attacks.

For my part, I do not claim that you "know" I am right. I think you are mistaken.

I've never said you were right, so I have no idea what you're talking about.

Evo, in [post=2502750]msg #66[/post] you said: "they claim to not have knowledge of precisely which points were udes". (I presume udes is a typo for used.)

A typo which was immediately corrected, if you look at my post. Are you really dragging the level of discussion that low Sylas? Was this typo of some great importance?

You don't give much context for that, but no matter. There's no context in which that makes any sense. It is false; they have made no such claim, and you have not shown any such claim. Indirect inferences are not claims, and appear to be founded on various misunderstandings of the data and the processing steps.

My post is quite clear Sylas, somehow, somewhere, a month ago, your car left the rest of the train and went down a different track. When people keep telling you that you're not talking about what they're talking about, you might want to stop and figure out why that is. We were discussing CRU claiming to have not kept their original data. Why you kept posting off topic saying basically "they have the data but can't give it out" had nothing to do with what the rest of us were discussing.

 

[Proton Soup]

Sorry!:

calculated results are not data.

homogenized "data" is not data.

 

[Integral]

Sorry!:

calculated results are not data.

homogenized "data" is not data.

Sure it is. It is the data that is used by the climate models.

 

[Evo]

(quote)Data sets must not be passed on to third parties under any circumstances. Any scientist requiring data which happens to have been supplied already to someone else, even within the same institute or programme of research, must first approach one of the NERC Data Centres, who have agreed to maintain records of data users for UKMO.

Once the project work using the data has been completed, copies of the datasets and software held by the end user should be deleted, unless permission has been obtained for them to be retained for some alternative use.(quote)

So yes Evo, where is the problem? I think you do not understand what is going on between skeptics request for information and the CRU at all.

You misread what you posted. It is saying that if the data is released to someone else to use, that the end user (that requested the data) delete their *copies* of the data, once their work is completed.

 

[Proton Soup]

Sure it is. It is the data that is used by the climate models.

So I wonder which data they use for their models and pretty charts... weird.

it's an intermediate calculation. and they've already admitted they can't reproduce the data requested.

 

[NeoDevin]

LOL. You misread what you posted. It is saying that if the data is released to someone else to use, that the end user (that requested the data) delete their *copies* of the data, once their work is completed.

The CRU got some of the raw data from various other agencies. Once they were finished with whatever particular work it was given for, some of the agreements stipulated that the raw data should not be retained by the CRU. Thus, the CRU is unable to provide this raw data, because they do not possesses it, because the agreements they made when acquiring the data stated that they weren't allowed to keep it (or in some cases, just that they are not allowed to pass it on to anyone else). If you really want the raw data, you need to approach the owners of that data, the same way that the CRU did.

The agreement referenced by sylas and Sorry! was between the CRU and the NERC. The CRU wanted to use the NERC data for their models, but they had to agree not to retain the data afterwards.

Are you intentionally trying to twist everything out of context?

 

[Evo]

If you really want the raw data, you need to approach the owners of that data, the same way that the CRU did.

Have you read CRU's webpage on "Availabilty"? I posted the link in a previous post. They state that they will share the data with other "academics" but not third parties. Read Pielke's blog. Look at what their respose was first to McIntyre, then what their response was to Pielke. Then address what CRU said.

 

[NeoDevin]

it's an intermediate calculation. and they've already admitted they can't reproduce the data requested.

Because the data requested is not theirs to reproduce.

If I want to do some work with someone else's data, then I write to them and request it. If someone wants to follow up and proceeds to request that data from me, I can't provide them the data, because (1) it's not mine to provide, and (2) my agreement may have stipulated that once my calculations are done, the data not be retained. (2) is often done if the group which owns the data intends to publish additional work with the data, they want control over who uses it for what purpose, so that they don't get scooped.

Put another way:

If I've done a bunch of work to produce some data, and I have a number of ideas for further projects to do with that data, then I want to maintain control of that data. If another researcher asks for the data to do something that I'm already intending to do, I would decline their request. If someone asks for the data to (a) reproduce my previous work or (b) do something that I have no intention of doing, then I would (probably) agree to give them the data. However, because I intend to get more publications from work done with the data, I wouldn't want them to be able to reuse it for any other projects they might come up with, or pass it on to other researchers to use for who knows what, because they might do some of the work that I was intending to do, and thus it would be detrimental to my career to provide unrestricted use of the data.

If you want a particular set of data, you must request it from the owner. In this case, that is not the CRU.