Badly written (published) articles?

[nonequilibrium]

Hello,

It's been twice now that I've been engaged in some kind of undergrad research and both of these times I've read a fair share of published papers and have had the feeling that a lot of these papers are poorly written (negligence) but more importantly contain blatant errors in their reasoning (and sometimes, resultingly, even wrong results). They're (theoretical) physics papers by the way.

Is this a normal sentiment? Do you personally also encounter this a lot? Do you just shrug your shoulders and look for a better paper? Or do you write a critique? I'm kind of surprised by this status of affairs...

 

[Fredrik]

I don't have as much experience as many others here reading published articles, but the experience I have tells me that you're right. Even articles that are 40-50 years old and are still being referenced are badly written and/or contain mistakes. This surprised me a lot too.

Now I think of this every time someone complains about Wikipedia not being reliable.

 

[homeomorphic]

Most journal articles are terribly-written, maybe by convention. I guess maybe the rational is that scientists shouldn't have to spend too much time doing the grunt work of writing. I do sympathize, since I hate writing. But it seems like a waste for papers to be so hard to read.

You have to look for the readable authors.

 

[nonequilibrium]

But homeomorphic, my full complaint also refers to blatant errors (see OP), not just bad style.

Fredrik, sad to hear you agree!

 

[Choppy]

You should see some of the papers that come in for review!

In my experience with the peer review process, it isn't perfect, but it is the best we have.

First there are issues with writing itself. For one, writing is often group work, but unless you have a single person who really assumes responsibility for the final polish, you can easily end up with incoherent statements or broken arguments, or blatent flaws in logic.

On top of that, many writers are not writing in their native language, which in many cases limits vocabulary and style and some ideas may get lost in translation. You get writers who are scrambling to meet deadlines, writers who simply don't put in the effort, or writers who are trying to salvage something coherent out of huge scientific efforts that were largely unsucessful.

And of course there are students... some like to write as if they are producing a magnum opus, others make you wonder if there were even any admission requirements to get into university in the first place. To be fair though, there are still others who write much better than people who've been publishing for 20 years.

Then there's the review process itself.

Referees will vary with respect to the amount of effort they put in. Some referees will go through a paper with a fine-tooth comb. Others wil come back with vague comments that make you wonder if they even read the paper in the first place. The review process will catch a lot of errors, but it won't catch everything. Remember referees are volunteering their time and are generally expected to complete the review in a period of weeks, so they don't all have the time to re-derive every equation, or look up every reference.

That being said, what do you do when you encounter a paper with an error? That depends on the error. Most people have better things to do with their time then correcting every grammatical error in their favourite journals. On the other hand, if the error is significant, you may choose to write a letter to the editor to point out what is wrong. In the journals that I publish in, the authors are usually granted the opportunity to post a response and/or publish an erratum.

 

[twofish-quant]

It's been twice now that I've been engaged in some kind of undergrad research and both of these times I've read a fair share of published papers and have had the feeling that a lot of these papers are poorly written (negligence) but more importantly contain blatant errors in their reasoning (and sometimes, resultingly, even wrong results). They're (theoretical) physics papers by the way.

Maybe. There is a bit of junk out there, but sometimes it takes some effort to figure out whether what you are reading is junk or brilliant. Physics papers are often writing assuming a lot of background knowledge so they skip over a lot of points. As far as errors in reasoning, papers will often omit assumptions that everyone is assumed to know, or the author may be engaged in what-if and hypotheticals. One thing that one of my mentors mentioned is that all papers have errors, the important question is whether the error is important to the outcome. Finally, it's important to read the papers in context. Often a paper is taking one side of an issue in which there is a real debate over what the correct reasoning is.

It helps a lot to get someone else to read the paper and find out what they think. In the case of peer reviewed papers in major journals, someone *has* already read the paper and found it publishable, so my inclination is to hold off a bit before saying "nonsense." Now if it's something that I download from the Los Alamos Preprint Server, then I'm not surprised if there is a major error.

Finally, professional courtesy starts becoming important when you start publishing papers. Essentially you want to treat people the way you want to be treated, so that even when you call attention it an issue in a paper, it's possible to be constructive and nice about it, if that's the way you want people to treat you.

Is this a normal sentiment? Do you personally also encounter this a lot? Do you just shrug your shoulders and look for a better paper? Or do you write a critique? I'm kind of surprised by this status of affairs...

I'd discuss this with other people in the department (and this makes an interesting conversation for journal clubs). If there is a consensus that there is in fact an error, then you try to involve the author in the discussion. The general way of approaching this is not "You stupid idiot, you made a mistake." but rather "I've read paper X, and I don't understand result Y." The reason that you avoid saying "you stupid idiot" is that a lot of the time, the author replies with something that addresses the issue (i.e. In order to get result Y, I assumed Z which is a standard assumption in this type of paper).

 

[twofish-quant]

I don't have as much experience as many others here reading published articles, but the experience I have tells me that you're right. Even articles that are 40-50 years old and are still being referenced are badly written and/or contain mistakes.

On the other hand, if you have a 40 year old article with a mistake, in my field, you'll find a 39 year old article that corrects the mistake. This may be field dependent, but in astrophysics it will be very hard to find a 40 year old article with some major assumption or calculation that we now know to be totally wrong. I expect that things are different in algebraic topology.

Now I think of this every time someone complains about Wikipedia not being reliable.

Wikipedia is pretty cool, and you get into some deep philosophical issues like "what is truth?" I think what people want is some "truth machine" that you just go through some rules and the truth comes out, but it's quite a bit harder to come up with something like this.

Also, this is where professional reputation comes in. There are some journals/authors which I expect more mistakes. One other thing is that there are authors that are considered to be geniuses because even when they are blatantly wrong, they are blatantly wrong in a creative productive way. Einstein for example.