Brokenness of Scientific Method: Up to Half of Papers May Be Wrong

[Ackbach]

Excellent read on the brokenness of the scientific method: there are many, many issues, ranging from conflicts of interest, data falsification, lack of proper incentives, abysmal lack of statistical knowledge, etc. As many as HALF of scientific papers published today might simply be wrong!

In summary: the idea of the scientific method is good. Its implementation right now is fraught with egregious errors, many and varied. Do not just trust anyone in a white lab coat.

 

[topsquark]

Yes, there are any number of flaws in applications of Physics as well. (For example, the original data from Edwin Hubble was presented correctly but his best fit line for the redshift problem was completely guess work and should not have been presented as it was. More data was required to make any kind of argument. (And don't get me started on "cold fusion!") That's a nice article. Thanks for posting!

Now here's a thought. We assume that the scientific method can be used to investigate just about everything. The idea is that we should (if we have the technology) be able to set up the experiment such that a preschooler should be able to push a button and get the same results as anyone else. But is this true? Can the scientific method be used to verify the scientific method?

Do not just trust anyone in a white lab coat.

I'm a hard core theorist. I never believe them!

-Dan

 

[Ackbach]

Yes, there are any number of flaws in applications of Physics as well. (For example, the original data from Edwin Hubble was presented correctly but his best fit line for the redshift problem was completely guess work and should not have been presented as it was. More data was required to make any kind of argument. (And don't get me started on "cold fusion!") That's a nice article. Thanks for posting!

Very welcome!

Now here's a thought. We assume that the scientific method can be used to investigate just about everything.

Well, many things, at least. I haven't heard anyone claim that you can investigate ethics scientifically. And consciousness is a standard topic that has evaded all scientific inquiries for a long time - I personally don't think science is going to make much progress there.

The idea is that we should (if we have the technology) be able to set up the experiment such that a preschooler should be able to push a button and get the same results as anyone else. But is this true? Can the scientific method be used to verify the scientific method?

That's a great question! I would say, on the whole, not. All reasoning systems make assumptions, and typically those assumptions cannot themselves be verified within the reasoning system (other than, if possible, showing consistency). See Godel's work for why that is. Even so simple a logical system as first-order logic plus the axioms of arithmetic is incomplete: there are true statements within the system that cannot be proved from within the system. Godel went outside the system to show that the statement was true, and showed that the system itself could not prove it.

Surely if you add in all the complexities and uncertainties of an inductive process such as the scientific method, this problem would only get worse.

I'm a hard core theorist. I never believe them!

-Dan

Couldn't help but smile when I saw that. Same here! Just ask my wife how often I'm wrong. ;)

 

[MountEvariste]

 

[MountEvariste]

This case was really a blatant fabrication of data. But the causes of the replication crisis are usually more nuanced. Personally I've had my antennas up for fabricated studies since the start of the pandemic. There have been a few studies claiming that tobacco smoking or nicotine has protective effects against Covid-19. At least one of these studies have been shown to have serious flaws, but there have been more studies published and even nicotine trials in France. What I find suspicious is that this is happening at a time when the tobacco industry is struggling, and even tried to register tobacco as an "essential item" in many countries to bypass laws on buying essential items only during the pandemic.