Is Open Access Publishing a Threat to Academic Integrity?

[HuskyNamedNala]

Is peer review legitament anymore?
I was having a discussion with a friend of mine regarding academia. He was explaining to me why he left with a masters as opposed to a PhD (his original plan). One major reason he cited was how political and unscientific publishing in journals has become. I did some research for myself and read some disturbing articles (did you know China has a lucrative fake publication black market and many "peer reviewers" don't actually have a background in science?) that had made me seriously reconsider the value of higher education and university research. Among these articles I came across a blog by a professor where he cited a "sting" conducted against Science and open access journals. The results were depressing. What is really frustrating is that I can't find a legitament news source for this; there is some pretty clear bias in rebuttles of the claims in the blog by paid access journals. Anyway, how legitimate are the claims in the blog and will this have an affect on the future of education and academic research?

http://blogs.berkeley.edu/2013/10/04/open-access-is-not-the-problem/

 

[jedishrfu]

In any profession where there's money to be had there's going to be some sort of corruption. As the major journals started to control the articles published making it more difficult and more costly for researchers to publish then hucksters stepped in with fake journals with lower to no peer review process. From there it was easy, for anyone to publish junk science.

The only solution is deal with known well established journals where the peer review process is tough but fair.

http://www.npr.org/blogs/health/201...-journals-will-publish-fake-science-for-a-fee

http://en.m.wikipedia.org/wiki/Who's_Afraid_of_Peer_Review?

 

[Vanadium 50]

HuskyNaimedNala,

I don't entirely understand your point.
Is your position peer review is illegitimate because it can be done badly?
Is your position peer review is illegitimate because some shady journals don't do it?

 

[Choppy]

After reading the blog post I think the guy's basic point was that "peer review is a joke" because someone was able to get a manuscript accepted in Science (one of the most prestigious scientific journals) that was obviously flawed. He was presenting this in context of the open-access debate where one of the oft-cited cons of open-access is the potential for bad work to get published.

My first thought in response to this blog is that the author is asserting a general conclusion from a single data point. Okay, so a single bogus paper made it through into a prestigious journal. And for the record, the actual comments of the referees did not appear to have been published. Was this a rare outright acceptance, or was this a conditional acceptance where the reviewers asked for a number of those obvious flaws to be addressed prior to publication? Even if it was an outright acceptance, it's still a single case.

Peer review is flawed. It's subject to human errors. It's prone to subjective interpretations. It can be influenced by emotions, grudges, elitism, and other forms of prejudice.

But no one has ever claimed that it is more than that. Peer review is not a magical threshold that bestows "truth" upon a manuscript. All it means it that the manuscript was sent to a couple of other people who have also published similar material and they have said that the article: (i) is appropriate material for the journal, (ii) contains novel or important information, and (iii) that they didn't see any obvious errors in the methodology or conclusion. And all of those can have grey areas. Sometimes we like to think that everything in science should be black or white, but it's not.

Yet, I don't think we should abandon the peer-review process, because if you do the inevitable question becomes: what will you replace it with? We have a publishing free-for-all called the internet, and without presenting any specific data, I'm not too afraid to assert that the level of crack-pottery online is significantly greater than the level of crack-pottery ultimately published in peer-reviewed journals.

In short, peer review adds to the quality of the published material.

After publication the "scientific machine" will continue to churn. If someone were to publish an incorrect result, other groups would have trouble verifying it and may even publish conflicting data. Eventually, the conflict would be resolved because the probability of independent groups arriving at the same incorrect conclusion is generally smaller than the probability of independent groups arriving at the correct one. Then the original authors have to face the issue of how to handle their incorrect results.

 

[Andy Resnick]

Choppy makes excellent points. Even so, there are a few (addressable) problems with peer review; both journal articles and grant proposals:

1) Predatory journals (http://scholarlyoa.com/publishers/). Don't publish there or review for them.
2) Peer review takes a lot of (uncompensated) time and effort. Consequently, it's difficult to convince he best qualified people to perform this function. NIH allows peer reviewers to submit proposals when they are able (as opposed to regular deadlines), and this approach needs to be broadened to other agencies and (somehow) cover journal articles as well. The more professionals that serve as peer reviewers, the more efficient the process will be.
3) When I submit a manuscript, I am often required to provide 'suggested' names of reviewers. This practice should be stopped as it undermines the process.
4) Peer reviewers need to do more than simply poke holes in manuscripts/proposals. They also need to make *constructive* comments- indicate what they like, suggest ways to improve the manuscript/proposal, etc. This will ultimately strengthen the quality of published work.

 

[TomServo]

Why worry about what gets published when it is only the worthwhile stuff that will continue to be cited and serve as a foundation for future research, while stuff that isn't worthwhile will be forgotten?

EDIT:

IOW, why not let citations serve as a de facto "stamp of approval" rather than formal peer review?

 

[Vanadium 50]

When I submit a manuscript, I am often required to provide 'suggested' names of reviewers. This practice should be stopped as it undermines the process.

Really? I don't think it's so bad. I don't expect the editor will select my suggestions - I expect he will select someone like my suggestions.

 

[Choppy]

Why worry about what gets published when it is only the worthwhile stuff that will continue to be cited and serve as a foundation for future research, while stuff that isn't worthwhile will be forgotten?

EDIT:

IOW, why not let citations serve as a de facto "stamp of approval" rather than formal peer review?

There is a movement towards this. For example, Google scholar now has a number of indicies such as the h-index or i10-index that can be tracked for scholars rather than just their publication list. Such indicies aren't without criticisms either. For one, if you write an article that happens to become controversial, it will get cited regardless of its quality. Secondly, a high-quality paper might ultimately get a high number of citations five years down the road when it enables something else, but that doesn't help you if you're up for tenure this year.

Ultimately, I suspect there will always be a need for critical evaluation by peers and this will be subject to the flaws that ensue.

EDIT:
Also - in response to the first point - it's worth worrying a little bit about what gets published because researchers will have to spend time and money trying to verify or refute bogus results. And, in worse case scenarios consider situations where people will often base things like health policy on these results.

 

[Arsenic&Lace]

Peer review is like democracy, in that while it has serious short comings there is no known alternative which is actually less awful.

 

[Andy Resnick]

Really? I don't think it's so bad. I don't expect the editor will select my suggestions - I expect he will select someone like my suggestions.

I don't have a problem with suggesting reviewers- I agree, it can help 'orient' the editor. I am opposed to this being *required* because it can create a 'default' choice and thus potentially create a conflict of interest. Besides, the editor/associate editor/etc. should have the requisite knowledge- otherwise, why are they editor?

 

[f95toli]

Besides, the editor/associate editor/etc. should have the requisite knowledge- otherwise, why are they editor?

I think that is a bit harsh. It might work for very specialized journals, but journals like PRL, APL, NJP (not to mention Nature and Science) publish research from a huger number of fields, they will -at best- have a few editors for each sub-field but even then you haven't really narrowed it down much: PRL's "condensed matter:electronic properties" covers semiconductor devices, superconductivity, condensed matter theory (of the bulk), graphene etc; these are not only very different areas, but also different communities.

 

[cgk]

I personally think that there is little reason to discuss peer review in journals---it is here and it will stay. But saying "it is essential for good science" is misleading. Peer review was not generally adopted in physics and chemistry until the early-mid of the 20th century. Yes: Most of the papers laying out the fundamentals of condensed matter, quantum mechanics, statistical mechanics, relativity, and so on we are building our world on today have never been reviewed by anyone except for the author and the journal editor. Einstein was confronted with modern-style peer review exactly once(!) in his career, in 1936 (and when he noticed that his paper was sent to an external reviewer without his approval, he angrily revoked the paper submission).

So there is good reason to believe that peer review is not all that essential. I'd also like to note that while peer review can improve manuscripts and prevent obvious mistakes from being published, it does not have to. It can also make papers worse (e.g. by insisting on lengthy discussions of de-facto unrelated work, which detracts from the main point) or prevent useful ideas from ever being written down in the first place (by taking time away from academics, or putting an unreasonably high burden on publishing certain types of things, making it simply not worth it). On the other hand, many wrong or bad papers still get through, because in peer review, normally only very obvious mistakes get found.

So, is peer review, on average, good or bad for science? No idea. Since we cannot change it in either case, it might not even be worth discussing.

 

[Choppy]

But the peer review process is changing.

We're seeing a paradigm shift where there is pressure for the information in academic journals, previously restricted to those in the ivory towers of academia, to be made accessible to the public. In response to that pressure, publishers are opening up access at the expense of the authors and this has an effect where the publishing bottom line varies in proportion to the number of articles published. Therefore peer review is moving from an independent context to one that can be influenced by financial incentives.

I think this is entirely worth discussing.

 

[Andy Resnick]

I think that is a bit harsh. <snip>

So let me put it another way- if I am unable to communicate the context using both a cover letter and suggested list of reviewers, why would the editor think my submission is appropriate for his/her journal? Remember, I am opposed to *requiring* a list of possible reviewers.

 

[ZapperZ]

I don't have a problem with suggesting reviewers- I agree, it can help 'orient' the editor. I am opposed to this being *required* because it can create a 'default' choice and thus potentially create a conflict of interest. Besides, the editor/associate editor/etc. should have the requisite knowledge- otherwise, why are they editor?

Is this really a real issue? I don't know of any journal that "requires" you to list recommended referees. They may allow you to make your suggestion, but it is never required.

The editors are usually themselves PhDs in the broad area that they are responsible for. And they do try to follow the "trend" in the fields, because when I was at BNL. I saw a couple of Phys. Rev. editors attending colloquiums and seminars (APS journal office isn't that far from that lab). And they are the first line to vet out the submissions. This is even more true for Nature and Science where the editors play an even bigger role where my guess is that more than half of the submission never get to the referees.

Zz.

 

[Andy Resnick]

Is this really a real issue? I don't know of any journal that "requires" you to list recommended referees. They may allow you to make your suggestion, but it is never required.

I am required to submit names and full contact information for Optical Society of America (OSA) journals.

 

[Vanadium 50]

I just got a paper back from a referee. In this paper we present our results in the conventional way. The conventional way is, granted, stupid, and it's only used for historical reasons. The referee insists that we present them in a new, perhaps less stupid, and perhaps just differently stupid way.

Is this annoying? Yes.
Will this help the paper? Not really.
Is this one of peer review's best moments? Not hardly.

Even so, I think that on the whole, peer review has helped my papers.