News Science vs. Politics: Tipping Points in Climate Change Communication

[mheslep]

Let's imagine for a second that the CRU did have the data and agreed to release it. What then? It's not like anything new would arise from the skeptic position no new papers showing why the CRU method is wrong that haven't already been written. ...

Absent the raw station data from CRU, how do you know that nothing new would arise?

 

[Proton Soup]

Can we try to keep the context clear?

My remarks you have quoted are specifically addressed to the point of the amount of flying time clocked up by Al Gore and Rajendra Pachauri. And I maintain, none of that makes any difference to the science. You have removed the remark from it's context here, and some people might thing it is now a remark about all the policitical brouhaha surround climate science generally. It is not.

We probably both agree that ideally the work of science in discovering answers to scientific questions should proceed without politically driven influence.

We agree, I suspect, that there IS in fact an influence of politics on this work; and that this is a bad thing. We might disagree on the nature and scope of that impact.

Cheers -- sylas

i don't think it's out of context at all. you imply that regardless of all the shenanigans displayed by Gore, Pachauri, et al. that the "science" they are triumphing is untainted.

 

[sylas]

For clarity, some of the original raw station data has apparently been lost or destroyed as Evo's link states. The information supposedly derivative of that raw data is what has been made available.

For more clarity --- NO.

The original raw data still exists and is maintained by the bodies that own it, and allowed the CRU to have use of it in their research. The CRU has merged all the raw data that was available to it into a single combined database, which has always existed and is being used all the time to get the final processed data products like CRUTEM.
The processing that was involved in the merge of raw data to a database is comparatively minor; for example it involves combining any duplicated records for a single station. Comments indicate that some of the original records may have been discarded by the CRU sometime after being merged into the combined database of underlying climate data.

The combined database cannot be released, because it contains all of that proprietary station data subject to binding non-disclosure arrangements, merged with all the data from other more open data providers. There is an ongoing work in progress to have this whole combined underlying record made available, but that requires permission from all the owners of the data that appears in the merged database.

You can think of it as a three step process.

Lots and lots of raw data --> combined database of merged raw data --> CRUTEM​

The vast majority of the raw data is available. You get it from the holders of that raw data. I don't know that the CRU itself puts up any of it on its own website; that kind of duplication achieves little. You should get the raw data from the owners and maintainers of the raw data, and about 95% of what has been merged is easily available.

The CRU does not release its combined database of merged raw data. It would be a handy thing to have, not so much for auditing, but for use by scientists in other independent calculations of all kinds of things. However, because there is a small amount of data in that merged database that is subject to non-disclosure, you can't simply put up the database for release.

The final processed CRUTEM product is released, of course, and always has been. Other research groups have replicated the entire process as an independent calculation entirely, and obtained the same result to well within measurement errors. This replication is not an audit; but a normal independent repeat as normally done in science to check someone else's results. Such replication takes nothing at all from the CRU, but gets their own data and uses their own calculations to obtain a final result that can be compared with CRUTEM.

Summary:

• It really is the original raw data that is mostly available, not only the processed result. What is available is sufficient for replication in the normal scientific sense of the word.
• The original raw data still exists. It is all held by the appropriate bodies which made it available to the CRU. Scientific replication means taking nothing from the CRU, but doing an independent collection and processing of data.
• The CRU also has a merged database of underlying climate data, which cannot be released because it includes proprietary information that the CRU does not own.

Cheers -- sylas

 

[Sorry!]

If you want to see exactly what sylas is talking about you could of course always head on over to CRUs ftp.

 

[Evo]

No longer available from THEM. That is very different. I don't think that any reasonable person thinks that CRU has the ability to destroy raw data held by NASA, NOAA, or any of the other international sources of their data-sets. CRU may no longer have THEIR copies of the huge data-sets that they obtained from other organizations, but that doesn't mean that the organizations that compiled those data-sets have destroyed them.

Let me post the collection of links that you deleted before (with NO link to the hosting organization) and everyone here will see that in fact the raw data, processed data, and processing code actually exists. The "destruction" of data is the biggest red herring ever in this particular subject. PF gets spidered and trawled over and over again every day, and subject to the storage capacity of the search engines, NOTHING is truly "lost" or deleted so that it cannot be recovered. I have made extensive used of NASA's NED web-site, CalTech's web resources, HyperLEDA, SDSS, and many other astronomical resources in the course of research. If I have limited space (I do) and deleted source data from my drives after populating spreadsheets, databases, etc, (I did), that in no way implies that the source data is "destroyed" or no longer available. I can't destroy that raw data, nor could CRU destroy the raw data that they relied on.

Last time. There is only similar data from GISS, etc... out of all of that data, the specific data that the CRU scientists picked out then changed is what is missing. Are you and sorry trying to troll?

 

[Sorry!]

For anyone interested to read the actually documentation on why CRU cannot release their raw data:

http://www.wmo.int/pages/prog/www/ois/Operational_Information/Publications/Congress/Cg_XII/res40_en.html

This is the reason.

 

[DanP]

I guess, it just gets annoying when people on here are complaining about how political AGW supporters are yet their only information comes from media sources. (Which is very clearly twisting the situation)

Why is that "twisting" the situation ? This is all political to the bone, I expect everyone involved will twist and use any leverage possible.

Besides, a lot of controversial info seem to come from CRU itself, through the e-mail hack. That is not "media".

 

[Sorry!]

Why is that "twisting" the situation ? This is all political to the bone, I expect everyone involved will twist and use any leverage possible.

Besides, a lot of controversial info seem to come from CRU itself, through the e-mail hack. That is not "media".

Not really. Most controversy from the e-mails occurred due to understanding the language being used. The continued controversy comes from certain media sources playing on these misunderstandings. As well if you do not think that the released e-mails over the internet is a media source than I do not know what else to say to you.

 

[Evo]

Now you are charging CRU scientists with cherry-picking data? Let me play PF-rule advocate here and suggest that you come up with some unbiased peer-reviewed references for that charge. That's not nice.

They have admitted it. How would a statement they made be peer reviewed? And we don't require peer review anyway.

Do you understand which data is being discussed?

 

[sylas]

i don't think it's out of context at all. you imply that regardless of all the shenanigans displayed by Gore, Pachauri, et al. that the "science" they are triumphing is untainted.

And that is precisely correct.

The shenanigans or otherwise of people who are publicizing science has nothing whatever to do with the quality of the science itself. The IPCC does not do research; it is a report on the state of science done by others.

I don't actually think the shenanigans are anything of the kind, but leave that aside. Certainly I think Gore in particular could do more to reduce his personal carbon footprint, but I don't think that is "shenanigans". It is a personal inconsistency. But this is all so clearly ad hominem for a comment on the science itself that we can presume they are baby eaters, if you like.

Gore has no input at all into the science, none. He is exclusively a popularizer.

Pachauri has almost no input into the science; when he became chair of IPCC he replaced a climate scientist as the previous chair, which some people consider to be a cause for concern. He himself is not a climate scientist, and was widely perceived as being more "industry friendly". His own PhD is in "Industrial Engineering and Economics", and he has a long history of direct industry involvement. Unlike the previous chair he has done no research at all into the science of climate, and has no mention or citations or references whatsoever in the WG1 report.

His alleged personal failings is a classic ad hominem fallacy.

Cheers -- sylas

 

[DanP]

Not really. Most controversy from the e-mails occurred due to understanding the language being used.

What, where those e-mails written in some sort of mystical secret society language and only those knowing the secret handshake can understand them ?

As well if you do not think that the released e-mails over the internet is a media source than I do not know what else to say to you.

It is a big difference. The source of that stolen data is CRU itself. The fact that it was stolen doesn't change it's origins.

 

[Proton Soup]

And that is precisely correct.

The shenanigans or otherwise of people who are publicizing science has nothing whatever to do with the quality of the science itself. The IPCC does not do research; it is a report on the state of science done by others.

I don't actually think the shenanigans are anything of the kind, but leave that aside. Certainly I think Gore in particular could do more to reduce his personal carbon footprint, but I don't think that is "shenanigans". It is a personal inconsistency. But this is all so clearly ad hominem for a comment on the science itself that we can presume they are baby eaters, if you like.

Gore has no input at all into the science, none. He is exclusively a popularizer.

Pachauri has almost no input into the science; when he became chair of IPCC he replaced a climate scientist as the previous chair, which some people consider to be a cause for concern. He himself is not a climate scientist, and was widely perceived as being more "industry friendly". His own PhD is in "Industrial Engineering and Economics", and he has a long history of direct industry involvement. Unlike the previous chair he has done no research at all into the science of climate, and has no mention or citations or references whatsoever in the WG1 report.

His alleged personal failings is a classic ad hominem fallacy.

Cheers -- sylas

so you think all this politicization doesn't influence who gets funding and who does not?

 

[mheslep]

Because the data has always been available.

Uh, where?

The station list has been posted for CRU I believe they use currently over 3000 stations and the list supplied gives all the stations used. (I think its something like 4000 stations). The CRU didn't 'get' their own information, they used other raw information and adjusted it (properly I might add). If you look at the first Station Jan Mayen you can look up the history of the station. It was moved I believe 4 times. So should all the raw data collected from this station be equally used? Of course not.

I didn't ask how or comment on how it should be used, I asked specifically for the raw temperature data.

You can also get the data from Jay Mayen. Knock yourself out?

You seem to be implying the temperature data since its founding in 1921 is somehow archived on this artic island? You know this how?

 

[Evo]

Evo the raw data is not CRUs data. So what this means (what turbo is saying to you) is that even if they DID delete ALL their raw data it's not gone. You can still go and get it for yourself.


IT'S THE SAME DATA.

From CRU's website

We are not in a position to supply data for a particular country not covered by the example agreements referred to earlier, as we have never had sufficient resources to keep track of the exact source of each individual monthly value. Since the 1980s, we have merged the data we have received into existing series or begun new ones, so it is impossible to say if all stations within a particular country or if all of an individual record should be freely available. Data storage availability in the 1980s meant that we were not able to keep the multiple sources for some sites, only the station series after adjustment for homogeneity issues. We, therefore, do not hold the original raw data but only the value-added (i.e. quality controlled and homogenized) data.

This is ridiculous, if you know what this is about then you know it is the data Roger Pielke requested and was denied. Odd that the authors of that paper (and you know which paper that is, right?) say the data is gone but you, turbo and sylas claim to have it. Perhaps you should send it to them, they need it.

And turbo, after I nominated you for the Community Spirit award for your Hot Stuff thread. tsk

 

[Sorry!]

You seem to be implying the temperature data since its founding in 1921 is somehow archived on this artic island? You know this how?

Well first of all I am implying that the data is still available. How do I know this? Well because I have seen the data for this location of course. Go look it up like I said instead of doubting me. And no I'm not going to post links to any data on here because they are not to 'CRUs raw data' and no-one cares to look at them.

 

[D H]

Not really. Most controversy from the e-mails occurred due to understanding the language being used.

I agree wholeheartedly.

I also had to capture your post before you "fixed" it.

 

[turbo]

Uh, where?

I can't show you where the data is available because there is a mentor here with an agenda that doesn't want me to post the links, unless I want to be banned. The data is there. The source data was never compiled by CRU, nor was it distributed by them. They are modelers, using data-sets compiled by others. A distinction that is apparently really convenient for some people to "overlook". I am NOT an AGW cheerleader, but I really detest the politicization of the issue and the mud-slinging against climate scientists who are trying to do their jobs. They might be good at it, or bad at it, or somewhere in between, but to portray them as crooks and frauds is pretty crass.

 

[Evo]

Well first of all I am implying that the data is still available. How do I know this? Well because I have seen the data for this location of course. Go look it up like I said instead of doubting me. And no I'm not going to post links to any data on here because they are not to 'CRUs raw data' and no-one cares to look at them.

Enough, you are not referring to the specific data that is being discussed. You are derailing the thread and it's going to stop now. We know there are tons of similar data, that's not the issue.

turbo, same goes for you, you're not addressing the specific requested data. Either neither of you understand or are intentionally trying to derail the thread.

 

[mheslep]

...

Pachauri has almost no input into the science; when he became chair of IPCC he replaced a climate scientist as the previous chair, which some people consider to be a cause for concern. He himself is not a climate scientist, and was widely perceived as being more "industry friendly". His own PhD is in "Industrial Engineering and Economics", and he has a long history of direct industry involvement. Unlike the previous chair he has done no research at all into the science of climate, and has no mention or citations or references whatsoever in the WG1 report.

His alleged personal failings is a classic ad hominem fallacy.

Cheers -- sylas

No, its hardly classic in this case. Ad hominem applies when a logical, supported by observation hypothesis is put forward, then instead of attacking the hypothesis on its merits the author is attacked. Here, on the subject of process, we have no scientific hypothesis from Pachauri, we have instead 'Pachauri says' the process is https://www.physicsforums.com/showpost.php?p=2467973&postcount=280", i.e. his opinion, in which case his personal credibility is entirely relevant.

 

[DanP]

Are you trying to say that raw data from Jan Mayen is not data that is being discussed?

It is not. It is a subset.

 

[mheslep]

Are you trying to say that raw data from Jan Mayen is not data that is being discussed?

from the file crustnsused.txt (CRU STATIONS USED):Gives ID number, position in the world the name of the station and the country. Jan Mayen data is very much part of the data being discussed here.

Yes everybody knows there's a meteorology station there. Where is the temperature data? I.e., May 15, 1965, 10:00 AM, 52 degs C.

 

[Evo]

Are you trying to say that raw data from Jan Mayen is not data that is being discussed?

from the file crustnsused.txt (CRU STATIONS USED):


Gives ID number, position in the world the name of the station and the country. Jan Mayen data is very much part of the data being discussed here.

What's the report we're discussing? Please provide the link.

 

[turbo]

turbo, same goes for you, you're not addressing the specific requested data. Either neither of you understand or are intentionally trying to derail the thread.

What data do you want? Will you ban me for trying my best to provide it from publicly-available sources? If CRU deleted the raw data in the 1980s after incorporating it into their models, that is understandable. These days, you can get a terabyte of storage for about $100 or so. 30 years ago, that was not possible - you'd need a budget that would scare a defense department. I have been trying to inject some balance into this thread regarding science vs politics, and have been disappointed again and again.

 

[Sorry!]

Yes everybody knows there's a meteorology station there. Where is the temperature data? I.e., May 15, 1965, 10:00 AM, 52 degs C.

Well the average temperature for the month of May 1965 was -2.0 Celcius?

 

[mheslep]

For anyone interested to read the actually documentation on why CRU cannot release their raw data:

http://www.wmo.int/pages/prog/www/ois/Operational_Information/Publications/Congress/Cg_XII/res40_en.html

This is the reason.

What, this?

Adopts the following policy on the international exchange of meteorological and related data and products:
As a fundamental principle of the World Meteorological Organization (WMO), and in consonance with the expanding requirements for its scientific and technical expertise, WMO commits itself to broadening and enhancing the free and unrestricted(1) international exchange of meteorological and related data and products;

Seriously, what portion of CRU temperature data is restricted by copyright?

 

[Evo]

What data do you want? Will you ban me for trying my best to provide it from publicly-available sources? If CRU deleted the raw data in the 1980s after incorporating it into their models, that is understandable. These days, you can get a terabyte of storage for about $100 or so. 30 years ago, that was not possible - you'd need a budget that would scare a defense department. I have been trying to inject some balance into this thread regarding science vs politics, and have been disappointed again and again.

If you know the specific data that was requested, then name that report. Post the link.

 

[Evo]

Sorry, I'm waiting for the answer. Link to the report please.

I'll even give you another ten minutes to come up with it.

 

[Sorry!]

Sorry, I'm waiting for the answer. Link to the report please.

It's not a report its data. It's already been posted, but this just goes back to when I said that the data being posted isn't being looked over.

@Your threat, please. Do whatever you want Evo.

 

[Evo]

It's not a report its data. It's already been posted, but this just goes back to when I said that the data being posted isn't being looked over.

@Your threat, please. Do whatever you want Evo.

That's what I thought. You have no idea what report the CRU claims they no longer have the data on.

 

[Sorry!]

Are you done deleting the post? Interesting to note that all the posts which reference directly to the data which is very much part of the discussion have been deleted. Also interesting to note that the post I made which clarifies your misrepresentation of the CRU statement has been deleted, as well as the post I made to point out that you had deleted it... Why?

 

[Galteeth]

This is a fascinating story. Unfortunately, it seems like the media has been misreporting it, as one would expect. This is primarily a story of ethical issues with regards to climate researchers working in an insulated atmosphere.
However, many are calling it the "smoking gun" of global warming fraud, which is not the case. My honest impression after reading the emails was "that's it?" Perhaps I am overly cynical, but I was actually expecting much worse. This is nothing compared to say, the fraud that goes in FDA drug trials.

I do think that some of these people, for example the folks over at the real climate blog, need a lesson in politics. Like it or not, these people are in a political position, and they should not have been so naive as to think that everything they did or said would not be used against them by the political opponents of climate change legislation. Furthermore, they come off as fairly childish in the way they put down those who disagree with them.

The real tragedy of this is that it adds just that much more noise to the discussion, and will further polarize people. This is a complex issue without easy answers, and the more people drift towards extremes (on both sides) the worse the ultimate outcome is likely to be.

 

[Evo]

Are you done deleting the post? Interesting to note that all the posts which reference directly to the data which is very much part of the discussion have been deleted. Also interesting to note that the post I made which clarifies your misrepresentation of the CRU statement has been deleted, as well as the post I made to point out that you had deleted it... Why?

Your posts on the deleted data is misinformation. You are continuing to post about something you don't have an understanding of, or are, at the very least, confused about. I haven't given you infractions yet, I've cut you a lot of slack, but I need to keep this thread about the facts. You proved you don't know which data we are talking about.

 

[turbo]

Sorry, I'm waiting for the answer. Link to the report please.

I'll even give you another ten minutes to come up with it.

You deleted the link that I gave to the raw data, the processed data, and the code. You also won't allow me to post the links without the links to the supposedly "biased" source hosting the links to the climate data. Yet you allow people to link freely to FOX and other highly-biased sources of information.

I will parse the links from the "biased" source, and post as many of them as pass your filter. I'm having a hard time with this because the restriction of raw scientific data for whatever reason you believe is not acceptable, and I'm seeing a strong right-wing bias on this forum I want to keep the discussion even and honest, but there is a strong bias that seems designed to prevent that.

 

[skypunter]

It's very convenient that CRU has incorporated copyrighted data within the full data set, and so cannot release the full data set.
Perhaps they could cut out the copyrighted data and release the balance.
Independent reserachers could then make up data to fill the gap and publish their research using this data as a basis.
Surely that would smoothe things over.

 

[Evo]

You deleted the link that I gave to the raw data, the processed data, and the code. You also won't allow me to post the links without the links to the supposedly "biased" source hosting the links to the climate data. Yet you allow people to link freely to FOX and other highly-biased sources of information.

I will parse the links from the "biased" source, and post as many of them as pass your filter. I'm having a hard time with this because the restriction of raw scientific data for whatever reason you believe is not acceptable, and I'm seeing a strong right-wing bias on this forum I want to keep the discussion even and honest, but there is a strong bias that seems designed to prevent that.

I gave all involved a chance to prove that you know what you are talking about. Greg now has the information I have on which data was requested. I have the source of the requested data, so I know what was requested. You shouldn't post repeatedly if you don't know what the issue is. It is disruptive to people trying to actually discuss the issues.

 

[Sorry!]

It's very convenient that CRU has incorporated copyrighted data within the full data set, and so cannot release the full data set.
Perhaps they could cut out the copyrighted data and release the balance.
Independent reserachers could then make up data to fill the gap and publish their research using this data as a basis.
Surely that would smoothe things over.

Well they've been doing this since 1980, prior to any of these events occurring by far. So I don't think its 'convenient' in the sly sense that they did it on purpose to hide the data.

 

[Choronzon]

Clearly it is not I who has no understanding of the topic. I'm also not spreading misinformation as I have no personal agenda.

Just because you 'one time dated a climate scientist' doesn't make you an expert on the subject. This is something I have studied with great interest for quite sometime I may not type it out as eloquently as sylas but I am definitely not spreading misinformation.

I think you need to take a deep breath and and read over your previous posts, because you are making some pretty far-fetched claims. It's the CRU's onus to provide the data, and once they've admitted to no longer having it I'm going to need more than some guy on a forum somewhere telling me that the data is readily available before I can accept that the CRU wasn't being dishonest or incompetent.

In all honesty, I don't know what report is being discussed here, but I do know that the CRU is admitting that they altered data to use in reports and then discarded the unaltered data. I can understand how these "quality adjustments" can be legitimate—but those adjustments should be transparent—which is impossible without the original, unaltered data. I'm definitely concerned and a bit disappointed that some of you want to defend the CRU's actions—maybe I had an idealistic view of science.

 

[skypunter]

What makes this data so valuable that it has to be copyrighted?

 

[Sorry!]

I think you need to take a deep breath and and read over your previous posts, because you are making some pretty far-fetched claims. It's the CRU's onus to provide the data, and once they've admitted to no longer having it I'm going to need more than some guy on a forum somewhere telling me that the data is readily available before I can accept that the CRU wasn't being dishonest or incompetent.

In all honesty, I don't know what report is being discussed here, but I do know that the CRU is admitting that they altered data to use in reports and then discarded the unaltered data. I can understand how these "quality adjustments" can be legitimate—but those adjustments should be transparent—which is impossible without the original, unaltered data. I'm definitely concerned and a bit disappointed that some of you want to defend the CRU's actions—maybe I had an idealistic view of science.

No I wasn't claiming that the data was still available 'as is' at CRU. I was saying that the data is readily available at various other sources. My point being (this is the politics forum) that the people who are finding something wrong with them merging the data and being unable to release it aren't actually wanting the data, since they have access to it anyways. They are just playing the politics game to a greater extent than even the AGW proponents.

To outline this I posted references to a specific observation station (which posts got deleted) where the data is still available. Actually someone requested temperature information for a particular day, I gave them the mean average for the entire month.

Their methods are transparent (I think atleast as they have multiple reports on them discussing the methods you can go through the posts to read them if you like).

EDIT: As well it is important to note that allegations of them 'deleting information' or 'throwing it away' are just that, allegations. The statement by CRU makes it very clear what they did with data, interpret it how you like, the fact remains that the files are still on the CRU ftp, which I can't seem to access any longer, lol, probably because McIntyre got a hold of some raw data a month ago from the Public FTP files.

 

[DanP]

What makes this data so valuable that it has to be copyrighted?

It doesn't really matter. If I , as in a private organization , collect some data I can choose to release it to public domain or not. I can sell the right to use the data to whoever I want, but I do not relinquish my property over it. The licensee can only use the data, but he can't disclose it to any other 3rd party.

Since it's the product of my work, it's mine to do with it whatever I want. Generally use it to generate profits for my organization.

 

[Choronzon]

No I wasn't claiming that the data was still available 'as is' at CRU. I was saying that the data is readily available at various other sources. My point being (this is the politics forum) that the people who are finding something wrong with them merging the data and being unable to release it aren't actually wanting the data, since they have access to it anyways. They are just playing the politics game to a greater extent than even the AGW proponents.

To outline this I posted references to a specific observation station (which posts got deleted) where the data is still available. Actually someone requested temperature information for a particular day, I gave them the mean average for the entire month.

Their methods are transparent (I think atleast as they have multiple reports on them discussing the methods you can go through the posts to read them if you like).

When the emails were first leaked I did run through RealClimate's posts on the topic to try and get their side of the story, and I did hear their justifications for the alterations, and like I said, I'll admit the plausibility of that particular argument, as long as that method is completely transparent.

I'm tempted to compare it my business's taxes. I have to turn in adjusted tax data to the Government periodically, and that usually is enough. If the government decides to audit me, which is their right, I then have to provide all of the original income information I used to create the adjusted claims. Without that information, my adjusted tax claims instantly become worthless. The same should be said of the report in question.

I'm going to disagree with the argument that AGW skeptics should have to collect their own data and perform studies to disprove AGW. The burden of proof should be on the scientific claim itself. I shouldn't have to disprove every theory that doesn't convince me.

 

[Choronzon]

It doesn't really matter. If I , as in a private organization , collect some data I can choose to release it to public domain or not. I can sell the right to use the data to whoever I want, but I do not relinquish my property over it. The licensee can only use the data, but he can't disclose it to any other 3rd party.

Since it's the product of my work, it's mine to do with it whatever I want. Generally use it to generate profits for my organization.

If the data isn't freely available for scientific review, than it should be completely inadmissible and any claims that require such supporting evidence should be ignored.

 

[sylas]

Yes everybody knows there's a meteorology station there. Where is the temperature data? I.e., May 15, 1965, 10:00 AM, 52 degs C.

Jan Mayer does not record data hourly.

On May 15, 1965, the minimum temperature was -4.0C, and the maximum was +0.4C. No precipitation was recorded on that day.

I obtained this data from the official GHCN repository at NCDC. It took me a while to sort through and get what I wanted, because there are over 1.7 GigaBytes of daily data there, and the format, though in ASCII, is not very human readable.

It is most unlikely that CRU would EVER release this exact original record; that is the responsibility of GHCN. I am hopeful that the CRU will eventually be able to release its entire database of daily climate data from all over the world; although it will almost inevitably be in a different format, for their own database. It would be a nice resource to have, though it will be enormous and rather unwieldy for amateur use.

Anyone want to suggest another station and another day?

Cheers -- sylas

 

[Sorry!]

If the data isn't freely available for scientific review, than it should be completely inadmissible and any claims that require such supporting evidence should be ignored.

Then ignore it? The difference of course between your taxes and this situation is that when the government audits you you are 'free' to give them the data or decline. In this situation they are not permitted to give the data, only to decline it. For now atleast.

 

[mheslep]

You deleted the link that I gave to the raw data, the processed data, and the code. You also won't allow me to post the links without the links to the supposedly "biased" source hosting the links to the climate data. ...

Turbo1, when you have a moment could you please PM me the link you have in mind?

 

[Choronzon]

Then ignore it? The difference of course between your taxes and this situation is that when the government audits you you are 'free' to give them the data or decline. In this situation they are not permitted to give the data, only to decline it. For now atleast.

Yes. Without the data you're left with nothing more than the researcher's word. That's most definitely not good enough for me. Are you saying that it's good enough for you?

 

[Sorry!]

Jan Mayer does not record data hourly.

On May 15, 1965, the minimum temperature was -4.0C, and the maximum was +0.4C. No precipitation was recorded on that day.

I obtained this data from the official GHCN repository at NCDC. It took me a while to sort through and get what I wanted, because there are over 1.7 GigaBytes of daily data there, and the format, though in ASCII, is not very human readable.

It is most unlikely that CRU would EVER release this exact original record; that is the responsibility of GHCN. I am hopeful that the CRU will eventually be able to release its entire database of daily climate data from all over the world; although it will almost inevitably be in a different format, for their own database. It would be a nice resource to have, though it will be enormous and rather unwieldy for amateur use.

Anyone want to suggest another station and another day?

Cheers -- sylas

Woah sylas sifting through the data. I just quickly found the monthly average to show that the data does exist. Nice job on that though

 

[Sorry!]

Yes. Without the data you're left with nothing more than the researcher's word. That's most definitely not good enough for me. Are you saying that it's good enough for you?

Yes, since I can replicate their results given the data and the source codes; which are all freely available.

The thing is Choronzon is that regardless of if the CRU makes the data public certain people will still find problems with various aspects climate research. I do think that the CRU should make its raw database public and in a format which is easily accessible to anyone who is interested however they are not doing that at this time. If you want you can head on over to I believe page 15 for the vast collection of climate research data (if it wasn't deleted) and look at everything that's available there. (Obviously not raw data from CRU) If this isn't good-enough for you merely because CRU isn't making their data available then there's nothing we can do is there?

Of course the CRU is the most highly cited organization on climate research however their results are still being compared to other organizations and accepted... Right?

EDIT: If you're interested there's a new report the link is posted in the Earth sciences forums. I would post it here but I think it may get deleted so why waste time. It explains a lot of the current beliefs and if you go to the main website you can find lots of references and resources.

 

[Choronzon]

Yes, since I can replicate their results given the data and the source codes; which are all freely available.

The thing is Choronzon is that regardless of if the CRU makes the data public certain people will still find problems with various aspects climate research. I do think that the CRU should make its raw database public and in a format which is easily accessible to anyone who is interested however they are not doing that at this time. If you want you can head on over to I believe page 15 for the vast collection of climate research data (if it wasn't deleted) and look at everything that's available there. (Obviously not raw data from CRU) If this isn't good-enough for you merely because CRU isn't making their data available then there's nothing we can do is there?

Of course the CRU is the most highly cited organization on climate research however their results are still being compared to other organizations and accepted... Right?

EDIT: If you're interested there's a new report the link is posted in the Earth sciences forums. I would post it here but I think it may get deleted so why waste time. It explains a lot of the current beliefs and if you go to the main website you can find lots of references and resources.

If the data is not freely available because some of it is "copyrighted", than no you can't. Also, as I said before, I shouldn't have to replicate their results by collecting my own data. They're the ones putting forth the claims—I should be able to look at their data and method and at least determine if I can accept their methodology.

Again, let's not forget—they have to prove their claims. I'm under absolutely no obligation to disprove them.

 

[sylas]

Woah sylas sifting through the data. I just quickly found the monthly average to show that the data does exist. Nice job on that though

I guess I probably know more about this particular data than anyone else at physicsforums at present. I can't be sure, but I would guess so. I don't tend to make any big deal about my own background, since I prefer to let my posts stand on their own merits, rather than have any personal authority. And I am indeed an amateur, with no professional training in climate science at all.

I have been using the monthly data myself for quite some time, for my own personal interest; and have had to work through all the usual difficulties of extracting data, decoding formats, picking out the bits I need, mapping between data from different sources, and so on. I write my own programs, and perform my own calculations.

The daily data is more than I have ever needed previously; but I am currently grabbing a copy to keep with all the other stuff I have lying around, as I can imagine several interesting things to try with it. The download tells me it has 9 hours and 47 minutes to go.

There is a file for Jan Mayer alone, which I used previously; that was only 756 K.

Some time ago (not here) I got into an argument over the quality of data in the USHCN network. Certain individuals believed that the data was hopelessly compromised by all kinds of flaws in the various US weather stations. I recognized that there were problems, but cited all the published accounts of how these problems were addressed. We reached an impasse; which is not new in this area -- as you can see in the thread!

That was when I started doing my own private analysis, just for fun. The skeptics had identified a very small subset of the network that they considered of reasonable quality. (On this matter, the term skeptic is appropriate and honourable.) So I obtained that data for myself; from USHCN and also from GCHN, and did my own calculation of an anomaly for the continental USA, to compare which what had been obtained using the full network. I downloaded all the source code (in fortran), which helped me figure out how to use the data, and then went ahead with my own programs (in C) supplemented with a few spreadsheets, and eventually got to the point of calculating an anomaly. Of course, as I was using only a small set of stations, I was not able to get the fine resolution of gridded data; I adopted a very coarse grid.

Eventually, I obtained and posted the results; and showed that the correlation with the data released by NASA for the continental USA was very high. No special tuning was necessary to get the result; it fell out pretty much straight away once I got my code debugged. That's skepticism at work as well. I was not willing to merely assume that everything was rosy. I saw the published papers that said the inaccuracies in stations had only a comparatively small effect; but calculating it for myself was a good confirmation. There was no audit involved; I used a different dataset (a subset) identified by the skeptics; and I used my own programs exclusively.

Now that I think about it, telling more about that little project might make a good thread in the independent research subforum. This was all about 18 months ago, I think.

Others have done something pretty similar, again using this limited set of USHCN stations that the skeptics had identified as reasonable. NOAA did it, for example, about a year after I did. They didn't bother to publish either. Frankly, there's not actually a lot of scientific interest or value in addressing the concerns of amateur skeptics on matters like this; it is more of a education issue, helping those interested learn more about the basis for the science.

That is my primary objective here also, believe it or not. I'm not actually a great social campaigner, and politics mostly bores me or depresses me. I have come to appreciate the need for action in response to the real threats, but I don't actually focus on that side of it myself much at all. I'm much more interested in the science for its own sake. Always have been.

Cheers -- sylas