Improve ways to recognise good research papers

[kkt21]

Hi everyone, good day to you. A friend and I think that the process of writing a papers (especially in sciences and maths) are rather troublesome due to the following reason:

1. Long journal review process
2. Selection of papers to be published are not transparent, a few negative reviews can affect the decision
3. Abstracts are long winded, when a bullet form would do the job

So I have a few questions and would really appreciate your honest opinions on them :)
1. Would you be willing to let other readers review and summarise your paper? If that's okay, would you prefer only those who are qualified (with good research background) to write the review?
2. Would you help to review and summarise other authors' paper in return?
3. How would you suggest other (quantifiable) ways to recognise a good paper rather than number of citations?

Thanks for your time and I really appreciate it! It would be great if we can make the academic community more collaborative and fast-paced. Have a great day ! :D

 

[S.G. Janssens]

1. Would you be willing to let other readers review and summarise your paper? If that's okay, would you prefer only those who are qualified (with good research background) to write the review?
2. Would you help to review and summarise other authors' paper in return?

Isn't this essentially already the common practice? (Assuming that in (1) one would answer "yes" to the second question.)

3. How would you suggest other (quantifiable) ways to recognise a good paper rather than number of citations?

Read it.

It would be great if we can make the academic community more collaborative and fast-paced.

It is already pretty fast paced, in my opinion.

 

[Hornbein]

Hi everyone, good day to you. A friend and I think that the process of writing a papers (especially in sciences and maths) are rather troublesome due to the following reason:

1. Long journal review process
2. Selection of papers to be published are not transparent, a few negative reviews can affect the decision
3. Abstracts are long winded, when a bullet form would do the job

So I have a few questions and would really appreciate your honest opinions on them :)
1. Would you be willing to let other readers review and summarise your paper? If that's okay, would you prefer only those who are qualified (with good research background) to write the review?
2. Would you help to review and summarise other authors' paper in return?
3. How would you suggest other (quantifiable) ways to recognise a good paper rather than number of citations?

Thanks for your time and I really appreciate it! It would be great if we can make the academic community more collaborative and fast-paced. Have a great day ! :D

1. Sure. But people who know more than I about my area are quite few, and very likely won't want to do it.
2. Sure. But my area of expertise is so narrow I would almost certainly be of no use.
3. Read it and think about it. The "number of citations" system is OK, but it can be gamed. And its not much use for oddball papers.

Anything interdisciplinary always seems to have problems with the review process.

You may be hinting that it would be better for anonymous referees to write the abstracts. That seems like a good way to get rid of the irritating self-promoting hype that has crept into today's abstract.

 

[kkt21]

Hi Hornbein,

Thanks for your swift reply, really appreciate it! :D

One of the main issues which I notice is exactly what you've mentioned - the self-promoting hype in today's abstract and I find that it doesn't represent the true value of the paper itself. With regards to your last sentence, do you think that a review and summary from an anonymous referees would be less professional and trusted as his identity is not known? The aim is to allow other people to summarise and review this summary so that future readers can get a better idea of the papers rather than the self-promoting hype.

Thanks for your time Hornbein! :D

 

[Hornbein]

do you think that a review and summary from an anonymous referees would be less professional and trusted as his identity is not known?

I think it would be better than what we have now. It seems worth a try. See how it works out.

A good approach might be to keep the abstract as it is while having the reviewers write a summary. Then there would be a smoother transition. Otherwise you have the following problem:

Joe Grad wants to publish his awesome PhD along with an abstract letting the world in on how incredibly awesome his work is. If he loses control over the abstract then the world might overlook his weighty conclusions! He might publish somewhere else that isn't so killjoy. So let Joe have his fun, and let the referees and editors write an old-school bland sober summary. The reader can choose what to pay attention to.

 

[Vanadium 50]

I fail to see the problem here.

1. Long journal review process

It can be. But it doesn't have to be. I've had papers accepted for publication within hours after submission. On a holiday. (To be fair, the journal was warned)

The archival journals do take longer - but shouldn't they? If the point is to publish the definitive study, isn't it a good idea to make sure it's as good as can be? And shouldn't an outside reader be an important late-stage part of the process?

Besides, by the time the paper comes out, the results are already known, through conferences and preprints.

2. Selection of papers to be published are not transparent, a few negative reviews can affect the decision

But negative reviews are supposed to affect the decision.

If you are arguing the problem is unfair negative reviews, the author has ways to counter. He can complain to the editor that the review is unfair, and he can even send the paper to another journal. The only people who can't seem to get through this are the crackpots, and to that I say "Good!".

3. Abstracts are long winded, when a bullet form would do the job

They are a paragraph. If that's too much to read, I think the complainant is in the wrong line of work.

 

[e.bar.goum]

3. Abstracts are long winded, when a bullet form would do the job

Physical Review C has abstracts that are very close to being in bullet form. Here's the paper that comes up first on the PRC website as an example. http://journals.aps.org/prc/abstract/10.1103/PhysRevC.93.014311
Apparently, this form of abstract is easier to write and read. I'd tend to agree.

However, clicking around the editors suggestions for PRA-D, I don't see many abstracts I'd categorize as long winded. Most are fewer than 200 words!

 

[kkt21]

Hi Vanadium50,

Thanks for your honest reply, that's really helpful! For each paper, if there are people who help to summarise the papers (not reviewing), do you think that'll be more helpful as you don't have to spend time reading through the entire report? Would you think that a summary of the paper can attract the readers with important findings so that they look further into the paper?

Thanks for your time Vanadium50 :)

 

[kkt21]

Hi e.bar.goum,

Thanks for your suggestion. I had a look at PRC and it gives short summary which is quite a convenient read :) Thanks again!

 

[Vanadium 50]

, if there are people who help to summarise the papers (not reviewing), do you think that'll be more helpful as you don't have to spend time reading through the entire report? Would you think that a summary of the paper can attract the readers with important findings so that they look further into the paper?

No. The abstract does that already. You have given no evidence that non-authors can write better abstracts than authors.

 

[Hornbein]

If you are arguing the problem is unfair negative reviews, the author has ways to counter. He can complain to the editor that the review is unfair, and he can even send the paper to another journal. The only people who can't seem to get through this are the crackpots, and to that I say "Good!".

Here's what David Hestenes has to say about it. This is from a speech he gave to the American Association of Physics Teachers as they awarded him the Oersted medal for excellence in physics teaching. True, it is a grant proposal, not an article submission, but still...

I conducted a sociological experiment by submitting proposals to the NSF for support of my work on relativistic electron theory. After the first couple of tries I was convinced that it would never get funded, but I continued to submit 12 times in all as an experiment on how the system works. I found that there was always a split opinion on my proposal that typically fell into three groups. About one third dismissed me outright as a crank. About one third was intrigued and sometimes gave my proposal an Excellent rating. The other third was noncommittal, mainly because they were not sure they understood what I was talking about. The fate of the proposal was always decided by averaging the scores from the reviewers, despite the fact that the justifications for different scores were blatantly contradictory. Such is the logic of the funding process. I thought that my last proposal was particularly strong. But one reviewer torpedoed it with an exceptionally long and impressively documented negative opinion that overwhelmed the positive reviews. He was factually and logically wrong, however, and I could prove it. So I decided to appeal the decision. I wanted to challenge the NSF to resolve serious contradictions among reviews before making a funding decision. I learned that the first two stages of the appeal process were entirely procedural without considering merits of the case. In the third stage, the appeal had to be submitted through the Office of Research at my university. The functionaries in the Office worried that the appeal would irritate the NSF, so it was delayed until the deadline passed. Such is the politics of grant proposals.

​

 

[Choppy]

Thanks for your honest reply, that's really helpful! For each paper, if there are people who help to summarise the papers (not reviewing), do you think that'll be more helpful as you don't have to spend time reading through the entire report? Would you think that a summary of the paper can attract the readers with important findings so that they look further into the paper?

This doesn't make a lot of sense to me.

Why should other scientists spend time that could otherwise be spent on their own research summarizing the work of other authors that those authors have already summarized? It seems like a net loss of productive time to me.

 

[Vanadium 50]

True, it is a grant proposal, not an article submission, but still...

They are different. How can changing the way papers are published- what the OP proposes - change the grant process?