Experiences with Cherry-Picked Data: Three "Three Wise Monkeys

[rigetFrog]

I saw this group many years back that I suspect was cherry picking their data.

The students would purposefully get as little data as possible. Enough to compare it to a model, but not enough to to negate the head researchers claims. When I would go and do the experiment more thoroughly, it was obvious the data and the claims were not aligned.

When I would discuss the research with the head of the group, they would say that any given student didn't know what they were doing. But they had no shortage of publications from any of their students.

There was one paper that had a systematic source of error in the experiment, which when accounted for, completely negates the original claims. The source of error eventually became main stream.

I have never confirmed my suspicions that the data was purposely cherry picked. I never confronted the head researcher , and will never do so because it would hurt my career too. By separating the data collection, analysis, and writing, I think they had a three "three wise monkeys" thing going on". Where no one can get accused of behaving unscrupulous. Each party can foist it off on a "mis communication" of the other.

http://en.wikipedia.org/wiki/Three_wise_monkeys

I still don't know if this was all in my head and I'm being paranoid, or if there was something unscrupulous going on. Does anyone have similar experiences?

 

[Andy Resnick]

What exactly is your (professional) relationship with "this group" and "the head researcher"?

 

[atyy]

I think you are being paranoid, it is more likely careless research. However, you do raise an ethical question. If you know the research is careless, should you hurt your career to correct it?

My initial thoughts are that
1) If you are not an author, then you don't have a responsibility, unless you know that it is grossly against the public interest (like Colin Powell announcing WMD). There's tons of wrong research that's published, even by well-meaning and excellent scientists such as Einstein.
2) If you are an author, I have seen two different conventions:
a) all authors are responsible for all parts of the paper
b) most authors are responsible only for parts of the paper, with only the head author(s) responsible for all parts of the paper
http://www.pnas.org/content/101/29/10495.full
http://www.nature.com/authors/policies/authorship.html

I believe I have seen (b) argued for because if (a) were enforced in a large collaboration, it would be almost impossible to publish the paper.

Just some quick and not necessarily correct thoughts, ethical questions are always important and worth thinking about.

 

[atyy]

In the erroneous measurement of superluminal neutrinos, it is interesting that not all members of the collaboration signed the paper, as reported eg. by Ouellette http://news.discovery.com/space/opera-leaders-resign-after-no-confidence-vote-120404.htm.

"Indeed, several OPERA members refused to add their names to the original paper because they felt the announcement and submission of the results for publication were premature. Extraordinary claims, as the saying goes, require extraordinary evidence. "

 

[Matrix0]

As a scientist, it is important to always uphold the highest standards of integrity and honesty in research. Cherry-picking data, or purposely selecting only the data that supports a desired outcome, goes against these principles and can lead to false or misleading conclusions. It is concerning to hear about experiences with cherry-picked data in this group's research.

It is important for scientists to be transparent and thorough in their data collection and analysis, and to be open to discussing and addressing any potential sources of error. It is also crucial for the head researcher to take responsibility for the work of their students and to ensure that all data is properly collected and analyzed.

If you have concerns about the validity of the data or claims in this group's research, it is important to address them in a professional manner. This could involve discussing your concerns with the head researcher, seeking guidance from a mentor or colleague, or even reporting your concerns to a higher authority if necessary.

In any case, it is always important to prioritize the pursuit of truth and accuracy in scientific research, even if it means potentially challenging the claims of a more established researcher. Ultimately, the integrity of the scientific community and the advancement of knowledge depend on it.