Advice for First Time Reviewers: Physics Journal Paper

[G01]

I've been asked to review a paper for a physics journal, and this is my first experience with being a reviewer.

Does anyone have any advice on being a good reviewer? Types of things to focus on, etc.?

Should I focus my critiques around the science alone, or is it ok to point out that the paper is poorly written (to the point that it obscures the science and impedes understanding.)

 

[jesse73]

My understanding is that the science is supposed to be leaps and bounds the focus of your feedback. Isnt the point of getting an expert as the reviewer supposed to be for judging the science because they could get anyone to judge the language?

 

[ZapperZ]

I've been asked to review a paper for a physics journal, and this is my first experience with being a reviewer.

Does anyone have any advice on being a good reviewer? Types of things to focus on, etc.?

Should I focus my critiques around the science alone, or is it ok to point out that the paper is poorly written (to the point that it obscures the science and impedes understanding.)

The journal should have some guidelines on what you need to look for when they ask you to review the manuscript. And yes, if it is poorly written, journals such as the Physical Review publications will consider that to not suitable for publication. Remember, the authors will have the opportunity to look at your comments and make the necessary changes. If it is poorly written, they should make an effort to rewrite it.

You must, however, make specific comments. Don't just say "the manuscript is poorly written". Cite the offending passages one at a time. This will provide the authors the exact section where they need to tackle.

Zz.

 

[G01]

My understanding is that the science is supposed to be leaps and bounds the focus of your feedback. Isnt the point of getting an expert as the reviewer supposed to be for judging the science because they could get anyone to judge the language?

The journal should have some guidelines on what you need to look for when they ask you to review the manuscript. And yes, if it is poorly written, journals such as the Physical Review publications will consider that to not suitable for publication. Remember, the authors will have the opportunity to look at your comments and make the necessary changes. If it is poorly written, they should make an effort to rewrite it.

You must, however, make specific comments. Don't just say "the manuscript is poorly written". Cite the offending passages one at a time. This will provide the authors the exact section where they need to tackle.

Zz.

Thank you both! My main concern is what I am supposed to focus my critique on. I didn't want to overstep my bounds. I am, of course, not going to critique writing style or get into arguments over phrasing.

However, the purpose of publishing results is to teach the community something new. If a specific paragraph is incoherent and thus makes the whole section a challenge to understand, I'd consider that something that needs to be fixed before the paper should be published. It works against the entire purpose of publishing a paper.

I will be sure to keep my critiques specific and to the point.

 

[AlephZero]

If you have been asked by the journal editors, presumably you have published some papers, so you already have some examples of other reviewer's work to re-read for "style" rather than "content".

But you might have been asked by a colleague who wants to "subcontract" the work, of course...

I think the basic test is "does the document in front of me make logical sense". That covers everything from the conclusions of an argument not following from the premises, badly designed experiments (e.g. uncontrolled variables), abuse of statistical methods, etc, down to typos in math formulas or references to the wrong figure or equation number.

If you know of published papers that are not referenced but relevant (whether they support or contradict the paper), that is worth mentioning.

Provided you justify your comments, don't worry about getting into "arguments", though the authors are unlikely to just roll over and accept everything you say, and sometimes the process takes several iterations to either converge or abort. In the end, the journal's editor makes the decision what to publish, not you - and in any case, the author shouldn't be given your identity. (But in specialized fields, sometimes you can make a pretty good guess who the reviewers are!)

 

[Vanadium 50]

I've reviewed a lot of papers, both internally and externally. I have never said "poorly written". I think it is better to be specific "lines xxx and yyy appear to be in contradiction"; "lines xxx to yyy have more detail than the rest of the paper, and I do not understand what point the authors are trying to bring out with this" and in one case "the authors purport to measure x, but in fact are describing the measurement of the closely related y".

That way, the author knows what the problem is.

The line between helpful and annoying is crossed when the referee says "this isn't the way I would have done it".

 

[Vanadium 50]

But in specialized fields, sometimes you can make a pretty good guess who the reviewers are!

True that.

 

[Choppy]

The first thing I try to do during a review is summarize succinctly in my own words what the authors have done, identify what is of interested to the readership of the journal, and what original work has been done. If I can't tell the associate editor what these are, then the manuscript needs work.

From there I go into details. Obviously the main focus is on the science, but it's fine to tell the authors about sentences or paragraphs that are unclear or ambiguous. I try not to assume that I'm an all-knowing authority, rather, an intelligent and critical member of the readership of the journal. If I find something confusing or ambiguous, likely other readers will to, and it's fair to point that out.

I divide the review up into general and specific comments (most of the journals I have reviewed for request this format). The general comments outline any major issues with the paper. Are there fundamental problems with the methodology? Have the authors interpreted their results correctly? Are important discussion points that warrant attention? Things like that.

In the specifics I go line by line and point out as much as I can in a first draft of my review. I later go back over these and decide whether I'm just nit-picking or whether there is an issue that actually warrants attention. I then identify which things are recommendations (i.e. if left as is I would still recommend the paper for publication) and which are necessary changes. I know I appreciate that when I receive reviews.

Most journals that I deal with have a copy editor as well. So grammar and spelling issues are often caught at that stage of the game.

Speaking as an author, one other point that helps is getting specific details about each issue that comes up. For example, if I haven't discussed an effect that could mitigate the results, it is sufficient to add mention of it in the discussion, provide a list of references, or should I run another experiment to fully quantify the magnitude of the mitigation?

 

[jesse73]

I've reviewed a lot of papers, both internally and externally. I have never said "poorly written". I think it is better to be specific "lines xxx and yyy appear to be in contradiction"; "lines xxx to yyy have more detail than the rest of the paper, and I do not understand what point the authors are trying to bring out with this" and in one case "the authors purport to measure x, but in fact are describing the measurement of the closely related y".

That way, the author knows what the problem is.

The line between helpful and annoying is crossed when the referee says "this isn't the way I would have done it".

A reviewer needs to be specific to stay on the safe side.

 

[Andy Resnick]

I've been asked to review a paper for a physics journal, and this is my first experience with being a reviewer.

Does anyone have any advice on being a good reviewer? Types of things to focus on, etc.?

Should I focus my critiques around the science alone, or is it ok to point out that the paper is poorly written (to the point that it obscures the science and impedes understanding.)

Some good comments here already. Reviewing a manuscript does not require you be to a subject expert, but it does require you to make an effort to understand what the authors are trying to say. I like to briefly summarize the goal(s) of the paper before making a few crucial comments:

1) What is new, and provide references if the results are already known.
2) Are the authors' claims validated by the results? Do the results agree with the discussion, or there other interpretations of the data that could lead to alternative outcomes?
3) Fact checks- is the data presented correctly, are there errors in derivations, do the figure captions match the data, was the experimental method faulty, etc.
4) Besides the authors, who else cares about this manuscript? Can the presented results be applied to other projects, or is it likely that nobody else cares?
5) Do the authors communicate clearly- are the graphs readable, the 'story' a clear logical progression, do they provide all needed background information, etc.? Note- some reviewers get hung up on this point and provide detailed lists of grammatical errors. Don't do that- you are not a copyeditor. A simple comment like "this manuscript would benefit from grammar and style editing" is sufficient.

Sometimes reviewers like to 'add on' extra things- suggesting additional experiments, for example. This is a gray area, and I would suggest refraining unless there is a compelling reason: unsubstantiated claims, lack of experimental controls, etc.

A reviewer has two goals. First, to ensure the science is accurately presented and discussed. Second, to help the authors make the paper better- pointing out strengths and weaknesses.

Other than that, try to be prompt with your review.

 

[G01]

I put together a review over the past several days and returned it to the colleague who "contracted out" the work to me. I'm told the review was quite well done.

Thank you all for your great advice!

 

[turbo]

Congrats, G01

 

[ZapperZ]

I put together a review over the past several days and returned it to the colleague who "contracted out" the work to me. I'm told the review was quite well done.

Thank you all for your great advice!

That is how many of us got into the "referee pool". I got in because my advisor at that time referred the journal editor to me to referee a manuscript.

Once you are in their database, you can expect more manuscripts being sent to you for refereeing. For the Physical Review journals, you will get your own referee profile that you can fill up, especially to indicate your area of specialty.

Zz.

 

[G01]

That is how many of us got into the "referee pool". I got in because my advisor at that time referred the journal editor to me to referee a manuscript.

Once you are in their database, you can expect more manuscripts being sent to you for refereeing. For the Physical Review journals, you will get your own referee profile that you can fill up, especially to indicate your area of specialty.

Zz.

So, what you're saying is that I just made a lot more work for myself?

Well, it's part of the job I guess!

 

[ZapperZ]

So, what you're saying is that I just made a lot more work for myself?

Well, it's part of the job I guess!

Yes, but there's an upside to this. Being a referee is part of being a professional physicist. When you are seeking a job, or doing your yearly evaluations, being asked to review things is something you want to include. It means that someone or some organization has recognized you as being an expert in a particular area, and value your opinion.

Many institutions look for such people.

Zz.