How can scientists effectively review papers outside of their expertise?

[andresB]

A week ago I accepted to review a paper for a top journal. I was contacted because I've published several papers on the same semi-obscure topic in the last two years. I accepted because the abstract was interesting and thought provoking.

However, after having access to the full paper I discovered the content to be far away from my expertise. I had to study a lot of new things just to have an idea of what the authors are even saying.

Being a reviewer seems to be a very hard job, three weeks to understand and give a solid opinion on a work that condensates months or even years of investigation.

So, how people even do this thing called peer review? any advice?

 

[Andy Resnick]

I know how you feel- it can be disorienting to 'have to' comment on work that is quite foreign. The good news is that you probably don't need to verify/certify the authors' results. I try to review papers/proposals generally with a view towards improving the presentation. There are a few 'levels' that I review material:

1) grammar/spelling/etc.- the super low-level copyediting stuff. I try not to spend too much time on this unless it martially impacts my ability to understand the writing.

2) Clarity of presentation: do I understand the graphs/plots/images? Are the annotations clear? Do the captions provide sufficient information to understand the graphics/tables? I also scan the reference list for (obvious) errors and also to get an idea of the context the authors are working in.

3) Do I understand what are the authors trying to say? Do the methods used make sense? Does the evidence provided by the authors sufficiently support their conclusions, or are there alternative explanations they have not identified? And yes- this is the most time-consuming part. It's ok not to understand, and in fact it may be useful to just say that- if you can't understand the paper, most likely other readers won't understand it either and just ignore the paper entirely. Don't forget- you are but one reviewer among several.

I try not to suggest additional work- I consider that bad form. If the manuscript is of such substandard quality that the authors clearly need to do a lot of additional work, I just note that and reject the manuscript.

Last comment- try and write a supportive report. Be specific with suggestions. It's fine to point out errors/mistakes, but I assume errors and the like are honest mistakes, not evidence of fraud.