Experiences with Cherry-Picked Data: Three "Three Wise Monkeys

[rigetFrog]

I saw this group many years back that I suspect was cherry picking their data.

The students would purposefully get as little data as possible. Enough to compare it to a model, but not enough to to negate the head researchers claims. When I would go and do the experiment more thoroughly, it was obvious the data and the claims were not aligned.

When I would discuss the research with the head of the group, they would say that any given student didn't know what they were doing. But they had no shortage of publications from any of their students.

There was one paper that had a systematic source of error in the experiment, which when accounted for, completely negates the original claims. The source of error eventually became main stream.

I have never confirmed my suspicions that the data was purposely cherry picked. I never confronted the head researcher , and will never do so because it would hurt my career too. By separating the data collection, analysis, and writing, I think they had a three "three wise monkeys" thing going on". Where no one can get accused of behaving unscrupulous. Each party can foist it off on a "mis communication" of the other.

http://en.wikipedia.org/wiki/Three_wise_monkeys

I still don't know if this was all in my head and I'm being paranoid, or if there was something unscrupulous going on. Does anyone have similar experiences?

 

[Andy Resnick]

What exactly is your (professional) relationship with "this group" and "the head researcher"?

 

[atyy]

I think you are being paranoid, it is more likely careless research. However, you do raise an ethical question. If you know the research is careless, should you hurt your career to correct it?

My initial thoughts are that
1) If you are not an author, then you don't have a responsibility, unless you know that it is grossly against the public interest (like Colin Powell announcing WMD). There's tons of wrong research that's published, even by well-meaning and excellent scientists such as Einstein.
2) If you are an author, I have seen two different conventions:
a) all authors are responsible for all parts of the paper
b) most authors are responsible only for parts of the paper, with only the head author(s) responsible for all parts of the paper
http://www.pnas.org/content/101/29/10495.full
http://www.nature.com/authors/policies/authorship.html

I believe I have seen (b) argued for because if (a) were enforced in a large collaboration, it would be almost impossible to publish the paper.

Just some quick and not necessarily correct thoughts, ethical questions are always important and worth thinking about.

 

[atyy]

In the erroneous measurement of superluminal neutrinos, it is interesting that not all members of the collaboration signed the paper, as reported eg. by Ouellette http://news.discovery.com/space/opera-leaders-resign-after-no-confidence-vote-120404.htm.

"Indeed, several OPERA members refused to add their names to the original paper because they felt the announcement and submission of the results for publication were premature. Extraordinary claims, as the saying goes, require extraordinary evidence. "