Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Featured Double blind peer review winners and losers

  1. Aug 30, 2018 #1

    Demystifier

    User Avatar
    Science Advisor

    Double blind peer review is a process in which not only the author does not know the name of the reviewer (which is the standard practice), but also the reviewer does not know the name of the author (which is not a standard practice). What would happen if the scientific community suddenly decided to shift to the double blind review process? Who would be the winners? Who would be the losers?
     
    Last edited by a moderator: Sep 2, 2018
  2. jcsd
  3. Aug 30, 2018 #2

    TeethWhitener

    User Avatar
    Science Advisor
    Gold Member

    This paper:
    http://www.pnas.org/content/114/48/12708
    from last year insinuates that famous authors, authors from top universities, and authors from top companies are significantly more likely to get their papers accepted using single-blind peer review (reviewer is anonymous, authors aren't) than double-blind.
     
  4. Aug 30, 2018 #3

    DrClaude

    User Avatar

    Staff: Mentor

    Making it actually double blind would be difficult. Only looking at the list of references, it is often easy to figure out who wrote the paper :smile:. In my field, when there is an experimental component, there are often very few groups in the world that have the proper equipment to carry out the experiment. Also, experiments and numerical methods tend not to be described completely in each paper, but referenced to previously published work.

    As a referee, I find it helpful to know who wrote the paper. As you get to know other researchers, you get a feeling about who is careful about some things and who is sloppy, so you get a feeling about how much to trust what is written. It can be useful to know about the particular slant of certain researchers (think about how it is on PF: posts that may seem innocuous to a passerby are seen in another light when you know that the poster is always trying to peddle their pet theory).

    I have never felt intimidated when reviewing papers by famous researchers. But I remember being disappointed by papers written by researchers close to me of whom I expected more than some of the low-quality stuff they've had their name attached to.
     
  5. Aug 30, 2018 #4

    Vanadium 50

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor
    2017 Award

    Exactly - if one says they looked at 13 TeV collisions using a detector with a 3.8 T solenoidal magnet, I am pretty sure who wrote the paper.
     
  6. Aug 30, 2018 #5

    f95toli

    User Avatar
    Science Advisor
    Gold Member

    I also don't understand how double-blind reviewing would work.
    In my field in would be almost impossible to cover all the details about a given experiment in a 4-page article. In a typical paper we will refer to perhaps 3-4 previous papers which describe device designs or experimental methods.
    Also, my community is small enough that I could probably guess which group the paper came from in 90% of cased even if I didn't see the list of authors.

    Double blind reviewing (or even open reviewing where the reviewers name is stated) might be possible in e.g. some life sciences where the community is much.much larger but I can't see how it would work in a typical field in physics.
     
  7. Aug 30, 2018 #6

    fresh_42

    User Avatar
    2017 Award

    Staff: Mentor

    The name contains information which might help to increase productivity (= use of time). Is it unfair? Probably, but I like the comparison with PF, which is at small what papers are at large. Don't we use information given by name beforehand? To be honest, I don't even open some threads when I see the OP, for various reasons, not necessarily that I don't expect any insights from them. And how likely is it that you get an interesting paper on the desk you haven't heard of before? And usually a famous author implies a famous reviewer, which also means that they are not afraid of honest answers. The review process is a time consuming one and you don't want to waste your time with a tri-sectionist. Yes, this is the extreme end in the set of examples, the other end being breakthrough papers, but anything in between is along a linear function f(paper)=time. I learnt that reviews aren't among scientists' favorite duties, a double-blind process would at least double the time required. Unfair? Maybe, however, it's a compromise that works - usually.
     
  8. Aug 30, 2018 #7

    TeethWhitener

    User Avatar
    Science Advisor
    Gold Member

    One question: if the main objection is "you could easily figure out who wrote the paper," then what's the harm in leaving the authors' names off the paper?

    Edit: during review, that is. Obviously.
     
  9. Aug 30, 2018 #8

    DrClaude

    User Avatar

    Staff: Mentor

    I agree that it may appear contradictory, but it is actually addressing two different points:
    • from the authors' point of view, anonymity is far from being assured, since it is possible to figure out who wrote the paper
    • from the reviewer's point of view, while it is possible to find out who wrote the paper, one cannot be sure so it can make reviewing less efficient
    An additional point to what I wrote above is that the reviewing process does not tae place in a vacuum. While in some ideal vision one could think that a paper should stand on its own merit, in reality it is one part of a bigger jigsaw puzzle and evaluating if the piece fits depends in some measure on who produced it.
     
  10. Aug 30, 2018 #9

    TeethWhitener

    User Avatar
    Science Advisor
    Gold Member

    1) At least according to the PNAS paper above, the authors who would be most affected by double-blind peer review are the ones who are already highly decorated, since it is those authors who stand to benefit the most from having their names known to reviewers. Unknown authors essentially see no benefit/detriment.
    2) Why should effort be placed into finding out who the author is? I would rather think that effort should be redirected into making sure the science is good. If the science is good, it will stand on its own merits, independent of its author.
     
  11. Aug 30, 2018 #10

    f95toli

    User Avatar
    Science Advisor
    Gold Member

    Again, many -in my field probably most- papers will contain "for details about the device design see reference [x]" making this irrelevant.
    Papers that are completely "stand-alone" and do not build on previous work by the same authors are very, very rare.

    Also, an important part of my job as a reviewer is to check for "novelty" and that what has been submitted in not just some minor change to what he authors have published elsewhere. I''d guess that about 50% of the times when I recommend rejection it is because the authors have previously published very similar results.
     
  12. Aug 30, 2018 #11

    Vanadium 50

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor
    2017 Award

    The names? Nothing but the extra work for no benefit. Citations that "out" the authors? That would make understanding of what's going on just go slower.

    As far as I can tell, the biggest impact this would have would be to kill the arXiv.
     
  13. Aug 30, 2018 #12

    TeethWhitener

    User Avatar
    Science Advisor
    Gold Member

    So then there's no harm if their names are left off the paper.

    If novelty is important, why does it matter who the authors are? Sure, if they're just extending a minor result onto results published elsewhere, it's a little crummy of them, but I don't see how a different author making a minor extension to their paper should fare any differently in the review process.
     
  14. Aug 30, 2018 #13

    TeethWhitener

    User Avatar
    Science Advisor
    Gold Member

    What extra work? Why do you need to know who wrote the paper? Why doesn't the science stand on its own?
    I'm not sure I understand this comment. If you can't understand what's going on in the paper, then you should decline to review it.
    Why? Because you could go and compare the manuscript you're reviewing with a preprint on arxiv? Again, why do you need to put that much effort into knowing the author?

    I understand that double blind peer review might be irrelevant in large collaborative efforts like particle physics or astronomy, but the vast majority of science isn't large collaborative efforts, and a lot of big names have published unimpressive results in impressive journals, to the extent that one openly wonders: did this get published solely by virtue of their name? One really easy way to make that cloud go away is via double blind peer review.
     
  15. Aug 30, 2018 #14

    f95toli

    User Avatar
    Science Advisor
    Gold Member

    But that would mean that some papers were reviewed double blind and some not. It would also mean that the author would -potentially- have the alter the way they write since they would then have to consider if they were happy with the reviewers being able to figure out who they were or not.
    Also, well-known people could effectively decide against a double blind review simply by including a reference to their own work.


    Because there is no way any review can be familiar with ALL the literature in the field. If I did not know who the author were there it would be impossible for me to know if the same group had published pretty much the same thing in some journal that I don't read.
    Or, as was the case in a paper I reviewed a couple of weeks ago, if the "novel" bit of the manuscript had already been included as part of a very long article the same group had written a couple of years ago (but where the main topic at first glance was very different).

    Novelty is pretty much everything for high impact journals. If your work is not novel you won't get published irrespective of the quality of the work.
     
  16. Aug 30, 2018 #15

    TeethWhitener

    User Avatar
    Science Advisor
    Gold Member

    I don't think this is true. For one, it misses the point of my objection: what if a different group had published pretty much the same thing in some journal that you don't read? Then knowing the authors' names would mean precisely diddly squat. For another, the scenario in the preceding sentence in fact did happen to me (twice, and rather recently) while I was reviewing a paper. I did what I normally do: google a few key words and phrases, and guess what...a very similar paper from a different group popped up within the first few searches, and I was able to recommend the manuscript for rejection based on that.
     
  17. Aug 30, 2018 #16

    TeethWhitener

    User Avatar
    Science Advisor
    Gold Member

    Another thing: This is also (trivially) true in single-blind peer review, so I don't see how this counts as a drawback for double-blind peer review.
     
  18. Aug 30, 2018 #17

    Vanadium 50

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor
    2017 Award

    The point I am making is that I need to know the equipment used to evaluate the paper, and knowing the details of the equipment tells me who did the work. Furthermore, I will likely have already read the preprint. This is far more common than not: if I am asked to referee, it's because I am up-to-date in the field, and if I am keeping up with the field, I am reading the arXiv. You can't have both - either you go double-blind, or you stop publishing preprints.
     
  19. Aug 31, 2018 #18

    Demystifier

    User Avatar
    Science Advisor

    Not so famous authors would then make more citations of a certain famous author to create a false impression that the paper was written by that famous author and hence to increase the chances for publication. :woot:
     
  20. Aug 31, 2018 #19

    Demystifier

    User Avatar
    Science Advisor

    No. The arXiv could be modified so that the author is revealed only after the publication or acceptance.
     
  21. Aug 31, 2018 #20

    f95toli

    User Avatar
    Science Advisor
    Gold Member

    But that would mean that there would be no way of keeping track of what the various groups in your field are doing(!).
     
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted