What is mainstream science?
Posted Nov18-09 at 09:47 PM by sylas
Updated Jan2-10 at 04:35 AM by sylas (add links to related blog entries)
Updated Jan2-10 at 04:35 AM by sylas (add links to related blog entries)
Physicsforums is a site for physics education. There's a lot of other stuff that goes on; social interactions, fun and games, discussions on other topics of various kinds, and so on; and that is all an important part of what makes it so much fun... but one of the main goals is education. This is explicit in the site guidelines:
This discussion also makes pretty clear what is intended by "mainstream" science; it is what you find within current professional scientific discussion. Professional peer reviewed journals are one of the most important ways in which mainstream discussion proceeds. There are also conferences, workshops, books, and private exchanges between researchers by email or over a cup of coffee.
"Mainstream" is not the same as "consensus". A vitally important part of mainstream science are the odd ideas by isolated individuals that get tossed into the mix. Mostly they don't work out and eventually get discarded; sometimes they become established and move into what is effectively a professional consensus. Sometimes contrasting ideas remain a focus for consideration for a long time before later developments eventually bring a resolution; and there are many open questions not yet resolved.
So what is NOT mainstream? There are many more of ideas that are not part of the current professional discussion, of course. Some are ideas that have been considered in the past, but which failed for various reasons -- such as having been falsified -- and are no longer in the current mainstream. Some individuals continue to push falsified ideas far beyond all reason, but not within the mainstream any more.
Peer review
The guideline singles out professional peer reviewed journals, and rightly so. This is one of the distinctive ways in which science discussion occurs in present times. However, it is neither necessary nor sufficient for recognizing what is mainstream.
Material not peer reviewed
There's a lot of useful information on mainstream science from sources other than peer reviewed journals. However, you can still use peer review as a useful indicator as follows:
If you have a claim or an idea in some other source, but you cannot find that idea or claim appearing also in current peer reviewed sources, or current established text books, then you've probably got something that isn't mainstream. It might be some new fledgling idea; but if it is at all useful then someone needs to publish it; and if they haven't, then that's a clear red flag warning.
Informal explanations and web essays are often excellent sources for describing mainstream science in ways that get past the often turgid pose and technical jargon of professional journals. But if the source of the essay is unknown or not themselves active in scientific publishing, then it is probably wise to double check against some other descriptions.
When peer review fails
Peer review is not an guarantee of credibility. Sometimes the peer review system can be bypassed by a journal editor and complete nonsense gets published, and sometimes review processes can fail spectacularly.
Examples of this can be very entertaining; they show up in many different fields of science from time to time. So although peer review is a useful guideline, it can sometimes go badly awry. Some examples are exposed in my blog at "When peer review fails".
Faux peer review
Another entertaining feature of non-mainstream science is that mavericks who can't get into credible journals often set up journals of their own, and attempt to pass them off as peer reviewed journals.
I have given some examples of this, across a range of fields, in my blog at "Journals to avoid".
----
In short, there's no simple identification of mainstream science. The concept is clear enough in principle, but there's no quick and easy method for a non-expert to sort it out.
For developing the capacity to distinguish sense and nonsense, nothing beats education; and that brings us full circle. In the process of learning, an interested amateur may often stray into side corners of the scientific world which look superficially plausible at first sight; and indeed exploring some of those corners can end up being very educational in its own right -- though sometimes embarrassing in retrospect. I've been there myself.
As far as physicsforums discussions are concerned, I generally try to be sympathetic to non-mainstream folks. Often people are genuinely uncertain of where the mainstream lies.
There are also folks who genuinely think that the whole mainstream of science is off the rails on some point; and are highly motivated to correct this defect. They want to show where science has gone wrong, and also show the validity of some idea which they know is not mainstream, and which they think is being unfairly rejected by most of the scientific community.
One of the things I like about physicsforums is that discussions along that line usually politely but firmly closed. There are other places in the web where they can push for the revolution. The special role of physicsforums, as I understand it, precludes threads in this line.
Quote:
Quote by Greg Bernhardt, on Sep 3 2003, in Physics Forums Global Guidelines
One of the main goals of PF is to help students learn the current status of physics as practiced by the scientific community; accordingly, Physicsforums.com strives to maintain high standards of academic integrity. There are many open questions in physics, and we welcome discussion on those subjects provided the discussion remains intellectually sound. It is against our Posting Guidelines to discuss, in most of the PF forums or in blogs, new or non-mainstream theories or ideas that have not been published in professional peer-reviewed journals or are not part of current professional mainstream scientific discussion.
"Mainstream" is not the same as "consensus". A vitally important part of mainstream science are the odd ideas by isolated individuals that get tossed into the mix. Mostly they don't work out and eventually get discarded; sometimes they become established and move into what is effectively a professional consensus. Sometimes contrasting ideas remain a focus for consideration for a long time before later developments eventually bring a resolution; and there are many open questions not yet resolved.
So what is NOT mainstream? There are many more of ideas that are not part of the current professional discussion, of course. Some are ideas that have been considered in the past, but which failed for various reasons -- such as having been falsified -- and are no longer in the current mainstream. Some individuals continue to push falsified ideas far beyond all reason, but not within the mainstream any more.
Peer review
The guideline singles out professional peer reviewed journals, and rightly so. This is one of the distinctive ways in which science discussion occurs in present times. However, it is neither necessary nor sufficient for recognizing what is mainstream.
Material not peer reviewed
There's a lot of useful information on mainstream science from sources other than peer reviewed journals. However, you can still use peer review as a useful indicator as follows:
If you have a claim or an idea in some other source, but you cannot find that idea or claim appearing also in current peer reviewed sources, or current established text books, then you've probably got something that isn't mainstream. It might be some new fledgling idea; but if it is at all useful then someone needs to publish it; and if they haven't, then that's a clear red flag warning.
Informal explanations and web essays are often excellent sources for describing mainstream science in ways that get past the often turgid pose and technical jargon of professional journals. But if the source of the essay is unknown or not themselves active in scientific publishing, then it is probably wise to double check against some other descriptions.
When peer review fails
Peer review is not an guarantee of credibility. Sometimes the peer review system can be bypassed by a journal editor and complete nonsense gets published, and sometimes review processes can fail spectacularly.
Examples of this can be very entertaining; they show up in many different fields of science from time to time. So although peer review is a useful guideline, it can sometimes go badly awry. Some examples are exposed in my blog at "When peer review fails".
Faux peer review
Another entertaining feature of non-mainstream science is that mavericks who can't get into credible journals often set up journals of their own, and attempt to pass them off as peer reviewed journals.
I have given some examples of this, across a range of fields, in my blog at "Journals to avoid".
----
In short, there's no simple identification of mainstream science. The concept is clear enough in principle, but there's no quick and easy method for a non-expert to sort it out.
For developing the capacity to distinguish sense and nonsense, nothing beats education; and that brings us full circle. In the process of learning, an interested amateur may often stray into side corners of the scientific world which look superficially plausible at first sight; and indeed exploring some of those corners can end up being very educational in its own right -- though sometimes embarrassing in retrospect. I've been there myself.
As far as physicsforums discussions are concerned, I generally try to be sympathetic to non-mainstream folks. Often people are genuinely uncertain of where the mainstream lies.
There are also folks who genuinely think that the whole mainstream of science is off the rails on some point; and are highly motivated to correct this defect. They want to show where science has gone wrong, and also show the validity of some idea which they know is not mainstream, and which they think is being unfairly rejected by most of the scientific community.
One of the things I like about physicsforums is that discussions along that line usually politely but firmly closed. There are other places in the web where they can push for the revolution. The special role of physicsforums, as I understand it, precludes threads in this line.
Total Comments 5
Comments
-
Sylas, I wrote the following to the CCNet some months ago. You might or might not find it worth a read.
===================
Dear Benny,
Mann's best friends. . .
I followed the link from Andrew Revkin to <http://dotearth.blogs.nytimes.com/2009/10/05/climate-auditor-challenged-to-do-climate-science/> and I am not sure that I did not read more into it than Andrew intended me to. The blog entry certainly was trenchant, and the attached threads if anything more so. All very entertaining if that is the sort of thing that entertains one. In all it vividly recalled Bierce's definition: "CONTROVERSY, n. A battle in which spittle or ink replaces the injurious cannon-ball and the inconsiderate bayonet." Spittle had the upper hand in that blog, I should say.
However, the point I wish to make here hardly surfaced in the blog though it is the most important theme of this entire matter. If AGW or any other form of climate change begins to bite, the question of Humanity's continued survival eventually will assume a fascination all its own, but for the present there are more immediate concerns, in particular the question of the survival of integrity and sense in science, and the role of peer review. Consider: "Michael Mann of Pennsylvania State University said that if Mr. McIntyre wants to be taken seriously he has to move more from blogging to publishing in the refereed literature. . ." and "'Skepticism is essential for the functioning of science,' Dr. Mann said. 'It yields an erratic path towards eventual truth. But legitimate scientific skepticism is exercised through formal scientific circles, in particular the peer review process. . . Those such as McIntyre who operate almost entirely outside of this system are not to be trusted."
Such stern sententiousness, irrespective of its source, should shame the most unregenerate cavillers to kennel, except those who pause slitty-eyed, to reflect on what he actually meant. "Peer review process", hmmm. . .? Those operating almost entirely *inside* of this system *are* to be trusted, are they? The same system that passed all sorts of publications of the most assorted standards during the last century or so, not to mention certain particularly embarrassing examples very recently? Publications that led to blushes inversely proportional to how effectively and for how long the parties concerned could distract attention from them? The same peer review process that has served as the most powerful tool for intimidating, quashing, and crippling the slightest dissent from the approved line? For punishing anyone who breaks the ranks of the favoured? For emasculating or deferring publication of the research of upstarts? The most powerful weapon for delaying outsiders' discoveries to the point of loss of priority of publication, or even to fatal obscurity?
Surely not! Which is fortunate, because that is not the point that I had referred to. Plenty of abler critics have raised similar objections more bitingly than ever I could.
No, the peer review that I write to praise and not to bury is the peer review that for generations of scientists has been the sentinel and shield against erosion of standards. It has been a sheet anchor both of the elite and the merely workmanlike journal, the means of assuring the editorial staff that the work they publish is sound, non-trivial, constructive, an advance on preceding work, a stone in the edifice of growing human knowledge. It has been an aid to efficiency, speeding the selection and augmenting the quality of the product of the researchers' labour and ingenuity; and of course (though perish the thought of any such sordid considerations crossing the mind of the authors) enhancing the kudos appertaining to the publication of the item.
Good stuff. Very good indeed. And yet I cannot rid my mind of a framed engineering degree on the wall of the office of an erstwhile young colleague of mine. It was in a large company, employing many graduates, and yet he was the only one that I remember nailing his colours to the er, wall in such a way. Any time the standard of his work or his good sense got challenged, he would point at his degree in rebuttal. Unanswerable of course.
And yet he did not last long, strangely.
Am I the only one to see this anecdote as relevant? Sorry. . .
Peer review as it should be used in a perfect world should not be a major concern of the author (except when a generous reviewer offers assistance or admonition, typically anonymous).
Peer review also should not be a major concern of the reader. If I read material dealing with a field I am so unfamiliar with that I cannot even follow the train of logic, then I act in bad faith and bad sense if I accept or condemn it on the grounds that it was or was not peer-reviewed. If however I can follow the logic, but without being able to challenge actual facts or observations, then I am able, with appropriate reservations, to accept, challenge, or reject the logic in good faith, but I still cannot justify my opinion by reliance on any peer review process. If I can claim to be fully conversant with the field, then I can accept, challenge, or reject any part, context or aspect of the work. If in doing so I need to defer to the dread dignity of peer reviewers, than how can I claim competence in the field at all? If I need to ask how it was reviewed before I consent to trust the work, then why am I reading such stuff, when there are plenty of Mills & Boone books to challenge my intellect?
Peer review or no review, it is for all readers to accept or reject research results according to what they find personally convincing. In good sense or good faith no research worker can justify a decision to accept or challenge work according to whether it had been peer reviewed.
*That is not what peer review is for.*
To criticise or praise a *journal* because of its eschewal or quality of peer review is reasonable in suitable contexts; even if one assumes that the editor is omniscient, it may be comforting to reflect that independent review guarantees lack of bias. However, to challenge the work of an *author* because it had not been favourably peer reviewed, is the most breathtakingly abject tactic I have seen, short of running to mummy because these nasty people had been disagreeing with ums. The more I contemplate it, the less it makes sense.
Consider what such justification for rejection amounts to: some third parties somewhere, who hadn't been asked to vet the work, but who might or might not have approved it if they had been asked, had not actually said anything about the work. Right? So because the work was not considered by those third parties, it thereby is errr. . . to be neglected without rebuttal by those in response to whose work it had been presented? Why should we respect authors who had been unable to assess the merits of criticism of their work or defend their work independently of peer reviewers? In the example under consideration, the criticism after all, did not involve novel work or novel techniques, but a critique of (peer reviewed) work. What role is peer review of the critique to play in such a case? What sort of peril would such peer review be intended to avert? Even in the top scientific journals, letters to the editor in response to peer reviewed articles are not in general peer reviewed. Right?
Never mind! Let's get back to the real world.
This much at least should be clear: science is passing through a most painful phase. (At least I hope that "passing" is not too optimistic a word!) As scientists we have a century or so of frequently (not invariably) inappropriate reliance on a cumbersome system. We have to deal with problems of ethics, politics, information explosion, population explosion, and technology explosion. In my opinion the peer review system *in its current form* has outlived its usefulness, in many respects even its viability. Whether the next generation is to rely on something totally new or on an amended review system, I cannot say, but what served for say the 1950s is hardly likely to serve for the 2050s. Some developments apparently in process within some Internet publications, in which pre-publications are exposed to public execration or appreciation before the final editing, may point the way to the future, but whatever form it takes, something new is needed.
Whether it turns out to be in the interest of the editorial staff, the author, or the reader, the fact remains that peer review as she currently is spoke, notionally is primarily for the benefit of the editorial staff, only contingently for the benefit of the author, and usually irrelevant to the reader, whether friend or enemy. But those who appeal to the process for shelter from unwelcome assessments of their work, or their duties to their readers; for some reason recall to me two lines of Burns written in a slightly different context:
From Envy and Hatred your corps is exempt,
But where is your shield from the darts of Contempt!
In case that strikes you as insulting, I invite you to consider it in the perspective of the insult to the reader at whom certain helpful remarks were directed -- remarks of the form: "Those such as McIntyre who operate almost entirely outside of this system [of the peer review process] are not to be trusted." We readers apparently are seen as stupid enough to swallow the hockey stick without choking on the mediaeval optimum or little ice age, but too stupid to gag at the implications of the physics of photon absorption, the history of volcanic influences on the climate, the principles of sample significance, or the implication of withheld data -- and far, far too stupid to read a statistical argument?
*Unless it is peer reviewed?*
No wonder Mann's best friends turned on him and bit him.
Jon
===================
Cheers,
Jon RichfieldPosted Apr3-10 at 02:35 PM by Jon Richfield 
-
Hi Jon, thanks for bumping my blog!Quote:Quote by Jon Richfield
Sylas, I wrote the following to the CCNet some months ago. You might or might not find it worth a read.
Bit of background first. I actually started this series of blog posts after being involved in the now closed climate science discussions here. I felt that we kept tripping over matters relating to what is science and when is it good or bad and so on, but that something about the specific climate subject distorted the matter.
So these posts are intended to be able to stand alone without reference to the whole climate thing, although they bear upon how one deals with any area where people get worked up about a scientific subject. Climate, evolution, relativity, cosmology, medicine, etc.
There's nothing particularly special about peer review, other than it seems to be the major method currently in use by the mainstream scientific community for sorting out what gets formally published. As certain wags have noted:No one pretends that peer review is perfect or all-wise. Indeed, it has been said that peer review is the worst form of journal oversight except all those others that have been tried from time to time.
As matters stand, if you want to know what scientists are actually doing on a particular subject (let's pick cosmology or black holes as the example, rather than climate) without being distracted by lots of poorly founded distractions with no scientific merit, a great rule of thumb is to insist on the use of peer review sources.-- From comments in my blog article on "When peer review fails".
Not because peer review is perfect. But because it is what is being used at present by scientists.
Now we come to this remark by Michael Mann."Those such as McIntyre who operate almost entirely outside of this system [of the peer review process] are not to be trusted."
It may be hard to separate this from the climate issue, since the author is a well known climate scientist, and McIntyre is a well known critic of conventional climate science. I suggest to you, concerns about the individuals aide, that the comment is perfectly correct. Remove the name McIntyre entirely, and just say that people who operate almost entirely outside the peer review system are probably not good sources on science; and their ideas are best treated with particular skepticism.
I certainly think that's true. Not because peer review is perfect, but because it is the system we've got at present. And people who operate outside it usually do so because they don't have the level of actual scientific merit that would allow them to have much impact through the admittedly imperfect systems we are using, in my experience.
If the statement had involved other names (say, Dembski vs Dobzhansky) perhaps it might have seemed more reasonable.
------
And so at last we come to it. Let's bite the bullet and take climate as the example.
Mann's statement is certainly true of McIntyre. He is mostly an irrelevant noise with respect to the many real debates involved in the open questions of science, contributing nothing useful and making life much harder for all the people who really are doing the science.
Say what? The truth is diametrically the reverse. He has pretty rock solid support from his friends and colleagues, and indeed pretty much the entire mainstream (that word again) scientific community. Given the peculiar position of climate science these days, there is a significant minority of scientists who have been critical of Mann, but not because of this statement. I don't think anyone has actually "turned" on him for making what should, in my view, have been a pretty unremarkable statement about a useful heuristic for seeking out where the science is being done.Quote:No wonder Mann's best friends turned on him and bit him.
I personally thought that the climate discussions which used to be hosted here were handled very well. To manage what is certainly a "hot" topic, the basic principle applied, as a simple heuristic to keep a focus on what is really going on in the world of climate science, was to keep the discussion focused on what is published. This allowed discussions to be more substantive and make real progress in learning about an area of physics where there is widespread public confusion.
I am, accordingly, soon to launch a new discussion venue for this subject, using precisely these guidelines.- Courtesy at all times
- Focus on what actually gets published in scientific literature
It won't satisfy everyone, of course, and there are some people who apparently think the scientific world is so corrupt that good criticisms can't get published. Such folks won't find the new forum much use. But these parameters still leave all kinds of scope for many open questions and divergent views -- as science has always done. So I think the forum has a niche. Would you be interested in such a forum?
Cheers -- sylasPosted Apr3-10 at 09:47 PM by sylas
Updated Apr4-10 at 11:59 PM by sylas -
Sylas, sorry that I never came back to you. I never spotted your reply and actually, I had totally forgotten submitting the foregoing. I regret this condignly, not only as a lost opportunity in polemics, but as a matter of simple courtesy.
I certainly agree with some things that you said, but breathlessly disagree with others...
Breathlessly...!
I am by now months behind your question, obviously, as well as any associated developments, but in principle I am definitely interested in such a forum. If anything has come of the exchanges so far, please let me know. I am uncertain whether you have the necessary email address, but the best probably is jonrichfield@gmail.com
JonPosted Jun12-10 at 08:11 AM by Jon Richfield
-
By the way, still wondering whether we are speaking of the same planet, did you by any chance write the above before reading the history of certain nameless climatologists and their standards of honesty and logic? Or (subsequently!) having a thoughtful trawl through certain leaked correspondence?
Irrespective of your reply to that question, if any, given the challenges to the logic and the reflections on the good faith of the participants and the truth of their respective statements, how would you go about using their publications before subjecting them to demands for retraction before re-issuing every argument in its entirety?
Try that question at two levels:
1) For a first-year undergraduate project
2) For a professor in his own speciality busting a gut to influence the Great and the Good world wide?
If your new forum addresses standards of integrity and scientific merit, I can't wait to see what has emerged!
Jon
PS: Do I SOUND shocked? I bloody-well should!Posted Jun12-10 at 08:23 AM by Jon Richfield
-
I am well aware of this matter, and I did make a careful trawl through the emails for myself, very early in the whole debacle.Quote:Quote by Jon Richfield
By the way, still wondering whether we are speaking of the same planet, did you by any chance write the above before reading the history of certain nameless climatologists and their standards of honesty and logic? Or (subsequently!) having a thoughtful trawl through certain leaked correspondence?
To be quite honest, I have little but contempt for the standards of logic and honesty of those who have tried to drum up the stolen emails into grossly inflated and absurd slurs on climate scientists who wrote them.
IMO the "challenges" you speak of here are without merit and without foundation. The emails do not bear out the spin. I have no problem whatsoever in using their publications as I have always done. There is nothing whatsoever in the emails to suggest any concerns on that score, at all.Quote:Irrespective of your reply to that question, if any, given the challenges to the logic and the reflections on the good faith of the participants and the truth of their respective statements, how would you go about using their publications before subjecting them to demands for retraction before re-issuing every argument in its entirety?
The only concerns might sensibly be raised are totally distinct from the legitimacy of publications, and go to handling of foi requests or questions of "openness". There too I think the whole beat up is pretty silly; but in any case it has no bearing on use of published work.
This whole side issue is a distraction from the topic of this blog post. Discussing whether or not my position on the standing of certain climate scientists in the light (or the darkness) of "challenges" is not something for this blog, or this forum.
-----
My main interest is not to defend people against slurs on their integrity. My main interest is simply to help explain the science, as best I can.
In no case does this depend on a few isolated publications from a few scientists. It is based on mainstream science of the whole scientific community, as set out in this blog post.
A bit late to a reply here, sorry, but I'd like to have this on record.
Cheers -- sylasPosted Oct23-10 at 11:14 PM by sylas
