Many forms of Government have been tried and will be tried in this world of sin and woe. No one pretends that democracy is perfect or all-wise. Indeed, it has been said that democracy is the worst form of government except all those other forms that have been tried from time to time.
-- Winston Churchill (1947)
This thread reminded me of this quote; and mentally I substitute "peer review" for "democracy"...
D H said:
New ES policy said:
Controversial claims must be supported by evidence that comes from a scientific, peer-reviewed journal or a similarly reliable source, i.e., unsubstantiated claims are not allowed.
The problem with this policy is that the reputable sources are not all that reputable. This latest UN report is but one example of that.
What do you think is the objective of this policy? Is there a better policy you have in mind?
When peer review fails
Peer review is only the first hurdle for a scientific idea. It doesn't ensure correctness. It is better thought of as a check that the work is worth looking at. The public at large often fails to grasp this; but hopefully most folks here will get it. Publication does not mean that scientific evaluation and criticism has been completed and the result is now confirmed. Publication is only the start of evaluation by the scientific community. The initial hurdle of review by a couple of often rather rushed journal reviewers is a check that it is worth putting the work up for consideration by the scientific community.
Sometimes the peer review process fails quite spectacularly, and let's through material that is unworthy of any consideration. It's not common, but it happens. I know a couple of breath taking examples. Here's one from a different field entirely. See the thread "[thread=322690]Understanding Einstein's Twin Paradox, past the usual level?[/thread]" This thread was prompted by a paper published in
International Journal of Theoretical Physics, a legitimate but low impact peer-reviewed physics journal (citation and link in [post=2343185]msg #7[/post] of that thread); and the paper is a sophomoric failure to understand special relativity. Complete and utter trash from start to finish. I know a few more such cases. The peer review requirement is not a panacea; but a very helpful heuristic.
By comparison with this relativity paper, the issues that have been raised here are trivial!
So why require peer review?
So why is the policy in place? I can't speak for Greg or the staff, but obviously there is a lot of material out there relating to climate in particular, which is not reviewed and is complete trash; but which still gets passed around certain circles as if it was credible science. The mentors here don't have standing as expert reviewers; and given the heat of discussion and the low quality of so much circulated material, it's sensible to have a readily manageable guideline to deal with this.
Sticking to material that has passed peer review strips out an awful lot of trash. It will also remove some perfectly good writing on the subject. But if the ideas expressed have any credible scientific basis, you should be able to find someone who has published about those ideas in the scientific literature. If it hasn't been published, that's a strong indication that the argument might not actually be as good as it appears to a general reader.
As I understand it, the requirement for peer-reviewed scientific sources is not a way to ensure that sources are beyond question. It's a hurdle which (almost always) means that the information in the source is worth considering. It's also a hurdle which is very easy to pass for any credible scientific argument. If the argument is any good, you should be able to find
someone who has managed to publish it.
Even bad papers are usually worth looking at. Peer review nearly always stops the trivial or nonsensical; but it still let's through a range of ideas and arguments, with many contrasting and mutually inconsistent ideas on various open research questions. Discussion in PF consistent with the guidelines still allows us to give a critical evaluation of published research, or examine the potential problems with new ideas.
What is the problem with the compendium cited here?
The problem that has been identified with the compendium cited in this thread is pretty trivial, by comparison with a real error. The main issue is very poor referencing; also the lack of error bounds, and a lack of discussion of the vertical scales.
However, no problem with the underlying science has been shown; and in fact the underlying science is legitimate peer reviewed science in good standing. The compendium is a secondary source.
Contrary to what has been said in preceding discussion, the image source is NOT "Wikipedia". It is "Wikimedia Commons", which is not the same thing at all; and the actual data sources for the image ARE peer reviewed scientific information in good standing. There's no prohibition on "original research" in wikimedia commons (see the
Project Scope policy), and no issue with the artist choosing to display the two curves together. The biggest problem is bad referencing in the compendium. The correct way to reference it (IMO) would be to give a credit to the artist (not a citation) combined with a citation to the data used by the artist.
The aim of the graphic was specifically to illustrate the correlation of a sharp rise in CO
2 with a sharp rise in temperature, as discussed in the main text of the compendium. This is a good graphic to illustrate that point. It would have been much better to reference it properly. It would have been even better to use images (with permission) from the primary sources directly.
The text of the compendium describes this correlation as prompting a question which is subsequently explored using more careful multidisciplinary analysis.
... But correlations between the curves of carbon dioxide in the atmosphere and the average global temperature appeared to be very strong: Over a 1000-year period, a fairly level trend seemed to start a sharp rise since the late 1800s. Was the climate changing—and, if so, how? And could human activity possible have anything to do with it? To answer such questions, researchers needed to use innovative approaches working with interdisciplinary teams that could incorporate data, information, and knowledge from many sources.
-- http://www.unep.org/compendium2009/PDF/compendium2009.pdf , McMullen and Jabbour.
[/color]
One minor criticism I have is that the compendium suggests the observed correlation was the impetus for the physical theory; in fact the theory came first.
Implications for PF policy
None. Wikipedia is still not a legitimate source for controversial claims in this forum.
Note that the original post got this wrong; there is no indication that wikipedia is used as a source. The actual source is simply a media repository, not the encyclopedia. The information behind the image is not wikipedia, but legitimate peer reviewed scientific papers.
But let's think about wikipedia anyway. If some
other reliable source uses an image from wikipedia, that does not discredit the reputation of the source or disqualify it according to the guidelines. Other sources DO have expert oversight which may not be easily accessible or recognized at physicsforums. A reliable source is free to evaluate images and decide if they are appropriate, subject to their own review requirements. If so, we then may use the reliable source, including the images it has deemed useful.
As to whether the Compendium considered here is actually a reliable source consistent with the guidelines quoted at the start of this post; that is for the mentors to decide. It's not technically peer reviewed scientific literature, I think; but (IMO) it is "similarly reliable". This does not mean perfect. I don't think any meaningful case has been given for saying that the compendium is not "reputable" or "reliable" in the same sense as a peer reviewed scientific article.
On the other hand, I am aware that some of the mentors and readers may see it differently. For my own part, I nearly always prefer to go back to the references in the scientific literature, unless there is some definite benefit in accessibility to use a source like this compendium.
Felicitations -- sylas