Bad technical journal article indicators

[Frabjous]

I was about to post in a thread about a referenced article, when all references to it were deleted by the moderators because it was from a predatory journal.

As I have gotten older, I have learned to look at reference distributions.
The paper has 22 references distributed thusly
Papers by the author (9): 5, 7, 8, 9, 10, 16, 17, 18, 22
Textbooks(5): 1, 11, 12, 15, 19
Papers over 50 years old (5): 3, 6, 13, 14, 20
Remainder (3): 2, 4, 21
I have found distributions like this to be red flags.

I was wondering what other people found to be red flags.

 

[jedishrfu]

I found these to be good indicators:

Wrt to articles there used to be a list of predatory journals that people would reference as a quick means to skip those papers. I think it hasn't been updated due to some legal issues.

Beall's list:

https://beallslist.net/

and the Yale list:

https://guides.library.yale.edu/c.php?g=296124&p=1973764

I think Beall had flagged a prominent journal and got sued to have it removed. Beall also got pressure from his university who worried about the liabilities of the list.

https://en.wikipedia.org/wiki/Beall's_List

 

[Frabjous]

Predatory journal lists are a good start, but I am more interested in what raises red flags when reading a paper.

 

[Dale]

The biggest flag is an obvious anti-establishment bent. Anything that claims to overthrow relativity, QM, or Newtonian mechanics is suspect.

 

[Ygggdrasil]

Those familiar with a particular field should know which journals tend to publish good papers that get read and cited by others in the field, so a red flag to an experienced researcher in a field is a journal that they and others in the field have not heard of before or are unfamiliar with.

 

[Ibix]

I read the article you refer to. A huge red flag in relativity is an overcomplicated thought experiment. These come from a line of thinking that says that 1+1=2 and 2+2=4, but what if we combine these and write 1+1+1=4? Obviously this is contradictory, so there's a fundamental flaw in arithmetic! Actually, of course, I've simply stuck so many simple operations together that I've made a bookkeeping error. Relativity is fertile ground for that, since the necessary algebra is in reach of a good high school student, but the concepts and exactly what you need to actively track can be a little slipperier.

 

[Klystron]

Predatory journal lists are a good start, but I am more interested in what raises red flags when reading a paper.

After allowing for language and cultural differences; poor grammar, weak composition, insipid word choice and spelling errors 'raise red flags' particularly given prevalent software that mitigate many common errors. Translation mistakes can usually be distinguished from substantive errors.

Citing authoritative sources in opening paragraphs of a paper may indicate lack of substance, as does referring to a reliable source then ignoring or misinterpreting conclusions. Crackpots inevitably appeal to authority before diverging into nonsense.

Previous posters suggest evaluating references and bibliography prior to reading a paper. Include self-references in this initial survey depending on the author(s). Alan Guth, John Baez or Roger Penrose citing their prior publications carries more weight than self-references by Jo Blow.

Misunderstanding humor or colloquialisms as fact also raises flags. George Gamov was well known for including humor, often aimed at colleagues, in otherwise serious publications. A paper claiming 'Gamov attacks Einstein's theories' based on a quip might indicate bias as well as an undeveloped sense of humor.

 

[Ygggdrasil]

Citing authoritative sources in opening paragraphs of a paper may indicate lack of substance, as does referring to a reliable source then ignoring or misinterpreting conclusions. Crackpots inevitably appeal to authority before diverging into nonsense.

I would actually argue the opposite. Good academic scholarship should cite previous relevant work in the introduction and manuscripts that don't cite the appropriate papers are often indicative of researchers who lack a good understanding of the existing scientific literature (e.g. papers that just seem to haphazardly cite arbitrarily chosen recent papers on a subject versus those that dig up the original work(s) that established a certain finding).

I do agree that misinterpreting/ignoring/misrepresenting the conclusions of a paper they cite is also a sign of a manuscript that is not likely to be reliable.

For people familiar with a particular field (at least for experimental sciences), another good way to really figure out if the authors know what they are doing is to read the methods section of their manuscript.

 

[kuruman]

I was intrigued by the existence of such a list so I decided to look at a sample article. Of course it had to be something that would be within my realm of understanding. The International Journal of Novel Research in Physics Chemistry & Mathematics attracted my attention in which the article "THEORY OF UNIFORM CIRCULAR MOTION IN PORTFOLIO SELECTION" by one W.G.S. Konarasinghe attracted my attention. I downloaded it thinking to myself that it cannot be what it looks like it is. It was. Here are some choice morsels. I have uploaded the .pdf for anyone who wishes to experience (and be amused) by the full impact of the article.

First few lines of the Abstract:
Scientific forecasting depends on mathematical modelling and statistical modelling. Mathematical models are deterministic, therefore mathematical models are unable to capture the uncertainty in real life. Statistics is known as mathematics of uncertainty. Statistical models contain the randomness; therefore statistical models have become more prominent in forecasting. The theory of Uniform Circular Motion introduced several mathematical models to describe the motion of a particle in a horizontal or vertical circle. These models were extensively used in the fields of Physics and Engineering, but rarely in the fields of Economics, Finance, and Management etc. Yet, in recent past, the theory of Uniform Circular Motion was incorporated with Statistics in forecasting risk and return of Sri Lankan share market.
@kuruman's note: if you read the article, said incorporation was made by none other than the author.
: The author quotes his/her research to validate further research along the same lines.

From Theoretical Background
According to De Moivre's theorem; $e^{-k \theta}= \cos k \theta+i \sin k \theta~~$"
@kuruman's note: Clearly, this was not reviewed by anyone and if it was, well $~\dots~$ never mind.
: The article contains flagrant mathematical infelicities.

When the particle moves in a circle, it is constantly changing its direction. At all instances, the particle is moving tangent to the circle. Since the direction of the velocity vector is the same as the direction of the motion, the velocity vector is directed tangent to the circle; as such, the acceleration of the particle also tangent to the circle. Even though the particle is moving under the acceleration with a changing direction, it does not leave the circular path. Therefore, there should be force acting towards the centre of the circle which prevents particle leaving its locus. This force is named as the centripetal force (Hooker, Jennings, Littlewood, Moran and Pateman, 2009).
(@kuruman's note: After having said that the acceleration is tangent to the circle, the author ascertains in the next line that the force must be centripetal totally disregarding Newton's 2nd law. To add awe to disbelief, the author provides a modern (2009) reference to add credence to his/her incorrect statement. Unfortunately, the .pdf that I downloaded had only a partial list of references and did not include this one.
: The article contains self contradictions and patently incorrect statements.
: The article provides well established ideas (centripetal force) as novel.

I stop here. I don't know about you, gentle reader, but I wouldn't attempt to forecast my portfolio on the basis of uniform circular motion. Besides, it might apply to the Sri Lankan market only.

 

[berkeman]

After allowing for language and cultural differences; poor grammar, weak composition, insipid word choice and spelling errors 'raise red flags' particularly given prevalent software that mitigate many common errors. Translation mistakes can usually be distinguished from substantive errors.

And from a recent Mentor discussion about questionable journals...

Among many things, the OP needs to distinguish the American Journal of Physics (which is a respectable journal) from the American Journal of Modern Physics (which isn't).

So I would say that pretty much *any* English grammar or spelling errors in a journal with "American" in it's name would be a big red flag. And probably there are some to be found in the issues of the 2nd journal mentioned in the Mentor discussion quote...

Edit/Add -- Lordy. I figured that I'd Google the 2nd journal mentioned above in my post (the problematic one), just to see if it was obvious about the grammar issues. At their website, about the 2nd or 3rd article on their featured papers list was this one, with the following opening sentence in the Abstract:

Quantization and Structure of Electromagnetic and Gravitational Fields
Branko Novakovic

Abstract

Following Relativistic Alpha Field Theory (RAFT) here it is started with the solution of the field parameters α and α′ in the combined electromagnetic and gravitational fields.

 

[jedishrfu]

After allowing for language and cultural differences; poor grammar, weak composition, insipid word choice and spelling errors 'raise red flags' particularly given prevalent software that mitigate many common errors. Translation mistakes can usually be distinguished from substantive errors.

Citing authoritative sources in opening paragraphs of a paper may indicate lack of substance, as does referring to a reliable source then ignoring or misinterpreting conclusions. Crackpots inevitably appeal to authority before diverging into nonsense.

Previous posters suggest evaluating references and bibliography prior to reading a paper. Include self-references in this initial survey depending on the author(s). Alan Guth, John Baez or Roger Penrose citing their prior publications carries more weight than self-references by Jo Blow.

Misunderstanding humor or colloquialisms as fact also raises flags. George Gamov was well known for including humor, often aimed at colleagues, in otherwise serious publications. A paper claiming 'Gamov attacks Einstein's theories' based on a quip might indicate bias as well as an undeveloped sense of humor.

WRT to Gamov, there was a physics paper by Alpher and Gamov on the origin of the chemical elements that Gamov added Hans Bethe to the list so the paper became known as the Alpher, Bethe, Gamov paper.

https://en.wikipedia.org/wiki/Alpher–Bethe–Gamow_paper

 

[TeethWhitener]

Previous posters suggest evaluating references and bibliography prior to reading a paper. Include self-references in this initial survey depending on the author(s). Alan Guth, John Baez or Roger Penrose citing their prior publications carries more weight than self-references by Jo Blow.

Excessive self-reference is to be viewed with skepticism. If your research career has any sort of continuity, you’re bound to cite some of your earlier work in pretty much everything you publish. There have only been two or three instances in my career where I found myself working in a field far enough away from ‘home,’ so to speak, that the papers I published didn’t cite earlier work of mine. (And I’m no Alan Guth, but I don’t think I’m a crackpot either—but then again, what crackpot ever does? :D )