Most scientific papers are probably wrong?

[Astronuc]

Most scientific papers are probably wrong!?

from New Scientist.

Most published scientific research papers are wrong, according to a new analysis. Assuming that the new paper is itself correct, problems with experimental and statistical methods mean that there is less than a 50% chance that the results of any randomly chosen scientific paper are true.

John Ioannidis, an epidemiologist at the University of Ioannina School of Medicine in Greece, says that small sample sizes, poor study design, researcher bias, and selective reporting and other problems combine to make most research findings false. But even large, well-designed studies are not always right, meaning that scientists and the public have to be wary of reported findings.

http://www.newscientist.com/article.ns?id=dn7915

Well that would certainly undermine the credibility of the scientific community.

The rush to publish could be a factor, as well as publications from questionable work, e.g. cold fusion.

 

[russ_watters]

Well that would certainly undermine the credibility of the scientific community.

Yes, though I'm not so sure that it should. Scientists aren't robots and are as prone to mistakes as everyone else. As long as science remains open and honest, mistakes will be exposed and corrected, and its true integrity will be intact.

The rush to publish could be a factor, as well as publications from questionable work, e.g. cold fusion.

Rushing creates mistakes everywhere, but P&F are a bad example of that. IIRC, they first "published" their "theory" in the Wall Street Journal. They circumvented the scientific process.

 

[ZapperZ]

from New Scientist.

http://www.newscientist.com/article.ns?id=dn7915

Well that would certainly undermine the credibility of the scientific community.

The rush to publish could be a factor, as well as publications from questionable work, e.g. cold fusion.

I just wish New Scientist pays attention to this report itself, since it has the tendency to report on outlandish ideas that often have not been given any reasonable validity.

People tend to forget that publications in science journals is only the FIRST STEP in disseminating ideas and discoveries for others to study, disect, and verify. You need to get published first before this process can start. Publication does not automatically means validity. If it does, the BCS theory does not have to wait for 15 years for their authors to win the Nobel Prize. Even the dubious Fleishman and Pons cold fusion paper was published, and it was only after that can we clealy verify that what they reported didn't happen!

I do not know what is the rate of falsified work there are in physics journals. I would hazzard a guess that it is smaller in the most prestigous journals such as Nature, Science, and PRL. It is the lower-tier journals that have weaker requirements and refereeing that may be the culprit. But hey, the quacks need some source to bolster their theory.

Zz.

 

[Rach3]

edit2:

This study itself was not peer-reviewed, was it?

 

[Astronuc]

I just wish New Scientist pays attention to this report itself, since it has the tendency to report on outlandish ideas that often have not been given any reasonable validity.

I put New Scientist in the same category with OMNI magazine. Scientific American is so much better. I agree that NS publishes peoples' outlandish ideas, and do not put a lot of stock in their publications. I certainly would not publish with them.

People tend to forget that publications in science journals is only the FIRST STEP in disseminating ideas and discoveries for others to study, disect, and verify. You need to get published first before this process can start. Publication does not automatically means validity. If it does, the BCS theory does not have to wait for 15 years for their authors to win the Nobel Prize. Even the dubious Fleishman and Pons cold fusion paper was published, and it was only after that can we clealy verify that what they reported didn't happen!

I agree.

I do not know what is the rate of falsified work there are in physics journals. I would hazzard a guess that it is smaller in the most prestigous journals such as Nature, Science, and PRL.

I would hope that the rate of falsified work is very low, less than a few percent, and actually I would rather people did not falsify work - it will be discovered sooner or later, especially if someone tries to repeat the work, apply it, or test it.

I think there maybe a greater degree of misinterpretation, or faulty extrapolation. At least, that has been my experience. And I have seen highly detailed technical reports that contained faulty data and conclusions, e.g. from simple mistakes like a faulty calibration in the initial phase of the work, which was not discovered until after the final report was issued!

It is the lower-tier journals that have weaker requirements and refereeing that may be the culprit. But hey, the quacks need some source to bolster their theory.

Yeah, and I wish they would stop doing that!

 

[Astronuc]

edit2:

This study itself was not peer-reviewed, was it?

Who knows? I think perhaps maybe it's a self-critique of New Scientist.

 

[ZapperZ]

Do you notice something though? Herein lies the obvious contradictions for the claims of many quacks and people who attact science and scientific establishements.

One one hand, there is this attack that tries to equate religious dogma and scientific ideas and principles. How many times have you seen people saying that science is no different than organized religion? Yet, IF this report is anywhere even half true, there clearly is ample evidence that even when an idea or discovery is published in scientific journals, the scientific community still challenges its validity and can falsify something if it isn't valid! It doesn't matter how high of a stature the authors are (just ask Betram Batlogg and the Bell Labs debacle from a few years ago). Everyone is subjected to the same scrutiny and will have to go through the rigorous process of having a published paper accepted as being valid.

Now when was the last time you see a religious idea and practice having something similar, and done on a regular basis?

Zz.

 

[Astronuc]

Now when was the last time you see a religious idea and practice having something similar, and done on a regular basis?

Well, I don't want to get into a religious discussion, but in orthodox Judaism, there is considerable study of the Torah and Talmud, and part of that study is questioning just about every passage. That is one reason that Judaism has persisted for about 2700+ years with the Torah in written form as a basis.

 

[ZapperZ]

Well, I don't want to get into a religious discussion, but in orthodox Judaism, there is considerable study of the Torah and Talmud, and part of that study is questioning just about every passage. That is one reason that Judaism has persisted for about 2700+ years with the Torah in written form as a basis.

But (i) this is not a common practice in practically all of the religions of the world and (ii) even for judaism, you don't hear such rate of "falsification" and challenges going on on a weekly basis (see any issue of PRL).

The point I'm trying to get across is that when people accuse science of being nothing more than a religion, they're trying to emphasize a point that scientific dogma doesn't change and can't be challenged, just because their point of view was dismissed by science. They somehow do not consider the possibility that their point of view is faulty and that's why it was dismissed. The fact that science DOES change when we have a valid theory and discovery, and that established ideas do get challenged often, clearly falsify such claims of having any similarity to religious practices and beliefs.

Zz.

 

[Schrodinger's Dog]

I just wish New Scientist pays attention to this report itself, since it has the tendency to report on outlandish ideas that often have not been given any reasonable validity.

.

To use an analogy New Scientist is the Sun Newspaper , Nature the Daily mail and Physics world the Times.

I'd imagine it depends also on the field as to how accurrate a paper is? I mean if it's mostly maths whoever's reading it may well come up with a solid contention if someone forgot to divide by infinity or something. Where as an esoteric theory about human consciousness could well sound plausible but be total tripe.

 

[DaveC426913]

Now when was the last time you see a religious idea and practice having something similar, and done on a regular basis?

What are you talking about? Why, just last month in "The New Fanaticist" magazine, didn't they refute the Pope's claim that Jesus was the Son of God? Apparently, his findings have been irreproducible, and they are now questioning his research methods and scrutinizing his credentials.

 

[Astronuc]

But (i) this is not a common practice in practically all of the religions of the world and (ii) even for judaism, you don't hear such rate of "falsification" and challenges going on on a weekly basis (see any issue of PRL).

The point I'm trying to get across is that when people accuse science of being nothing more than a religion, they're trying to emphasize a point that scientific dogma doesn't change and can't be challenged, just because their point of view was dismissed by science. They somehow do not consider the possibility that their point of view is faulty and that's why it was dismissed. The fact that science DOES change when we have a valid theory and discovery, and that established ideas do get challenged often, clearly falsify such claims of having any similarity to religious practices and beliefs.

I agree. My comment on religion was too abbreviated.

The issue in any system of thinking, or more to the point, is the basis or foundation of that system, and whether or not that basis is purely dogmatic. Hopefully, scientists are objective, but certainly this is not always the case.

An important part of being a scientist is the ability to 'change' or 'revise' one's thinking or understanding upon the discovery of new information or evidence, which contradicts one's view at that point. One must be prepared to challenge one's own thinking and understanding.

Another important part is simple integrity of the person and the process.

 

[Rach3]

The bottom line is that religious scholars do not do experimental work, hoping for "repeatable experiments" and "falsifiable predictions" - for the simple reason that religion is not science! (Why would it be?) It would be fantastically absurd to see an issue of exactly how should people worship, being decided by a well-designed physical experiment using superconductors and lasers.

It's apples and oranges.

 

[Astronuc]

I heard someone tonight on the radio mention . . . are you ready for this . . . using science to prove reincarnation.

 

[heartless]

I heard someone tonight on the radio mention . . . are you ready for this . . . using science to prove reincarnation.

a little offtopic, but, why are you a laughing?