Exploring Censorship in Science: A Discussion with Researchers

[brainpushups]

As part of my introductory physics class next year I'm planning on including more history of science into the curriculum. During one class I've decided to include excerpts of translations from Galileo's Starry Messenger and Dialogues Concerning the Two Chief World System (I'll also include parts of Two New Sciences, but this will be in a different class). Part of the discussion in this class will include a copy of Galileo's Papal condemnation and subsequent censorship. To bring the discussion back to contemporary issues I'd like to allow some time for a discussion about the role censorship plays in modern research.

Being an educator and not a researcher, I don't have any first hand experience with censorship. I have found some decent resources online (this article by Brian Martin, for example), but I am interested in hearing some opinions from individuals with research positions (academic or otherwise) about the role of censorship in current research.

 

[Andy Resnick]

From my perspective (US academic researcher), modern forms of 'censorship' typically stem from either corporate-funded projects or military-related projects, neither of which are good analogies to what the Catholic Church used to do. Certainly, there are still religion-backed censorship activities in US education (Intelligent Design, for example), and in foreign countries there are likely repressive pressures of which I am not directly aware.

 

[brainpushups]

Yes, sorry if I was unclear. I'm sure that the sort of blatant censorship that the Catholic Church did to Galileo and others is not experienced today, at least not in the context of religious belief. Certainly in the context of military and corporate research this can happen. What are your thoughts about censored corporate or military research? Is it ethical? Is it necessary? Should all scientific knowledge be open-source?

What about more subtle types of censorship like self-censorship, non-publication of null results, de facto censorship of lack of funding for certain areas of research, or the difficulty of challenging the current paradigm?

 

[tom aaron]

Yes, sorry if I was unclear. I'm sure that the sort of blatant censorship that the Catholic Church did to Galileo and others is not experienced today, at least not in the context of religious belief. Certainly in the context of military and corporate research this can happen. What are your thoughts about censored corporate or military research? Is it ethical? Is it necessary? Should all scientific knowledge be open-source?

What about more subtle types of censorship like self-censorship, non-publication of null results, de facto censorship of lack of funding for certain areas of research, or the difficulty of challenging the current paradigm?

I've never experienced any censorship in almost 4 decades of doing geologic research.

Censorship exists but is exaggerated in the sciences. Most censorship is in technology...as mentioned, in military and corporate environments.

Ethical? A million circumstances with all types of variables. Hard to generalize. Should we have shared nuclear bomb research with Nazi Germany? The Brits given up their early radar detection technology?

The problem with open source in more benign areas is that of incentive. In the sciences and technology, 99% of time is spent on incremental advances, ,fine tuning, repetitive testing, etc. I've spent months cutting and thin sectioning rock samples...samples that were collected over years of expeditions. All by our research centre. I certainly wouldn't have done it if someone could tap into the current findings and publish.

Would you work months and spend thousands of dollars customizing a car if some guy could just walk up, put his key in it and drive it off to a car show?

 

[brainpushups]

Censorship exists but is exaggerated in the sciences

That was my suspicion.

Ethical? A million circumstances with all types of variables. Hard to generalize. Should we have shared nuclear bomb research with Nazi Germany? The Brits given up their early radar detection technology?

Based on your phrasing of the question I take it that you would argue that bomb research shouldn't have been available to scientists in Nazi Germany. I'm not saying I disagree. Just playin' Socrates here:

Does that mean its tough luck for scientists living in an extremist regime if they work in fields of research for which the knowledge gained has the potential to be used to do harm? There was an http://scitation.aip.org/content/aip/magazine/physicstoday/article/63/8/10.1063/1.3480068 in PT several years ago about how sanctions against Iran were hurting Iranian scientists in their ability to get equipment or even access information on the web. Yes, different from censorship, but a similar effect. Should the flow of information to Iranian scientists be restricted? Is the demarcation criterion for censoring scientific research in this way be whether there is a possibility the information could be used to do harm?

What about censoring research funded by corporations? Should a corporation have carte blanche rights to perform a study and then censor the study if the findings could hurt profits?

The problem with open source in more benign areas is that of incentive. In the sciences and technology, 99% of time is spent on incremental advances, ,fine tuning, repetitive testing, etc. I've spent months cutting and thin sectioning rock samples...samples that were collected over years of expeditions. All by our research centre. I certainly wouldn't have done it if someone could tap into the current findings and publish.

Would you work months and spend thousands of dollars customizing a car if some guy could just walk up, put his key in it and drive it off to a car show?

Would a fellow scientist swoop in and 'steal' the work in this way just to publish a paper before you were able to? Is that revelatory of the pressures put on scientists to publish?

 

[Andy Resnick]

Yes, sorry if I was unclear. I'm sure that the sort of blatant censorship that the Catholic Church did to Galileo and others is not experienced today, at least not in the context of religious belief. Certainly in the context of military and corporate research this can happen. What are your thoughts about censored corporate or military research? Is it ethical? Is it necessary? Should all scientific knowledge be open-source?

What about more subtle types of censorship like self-censorship, non-publication of null results, de facto censorship of lack of funding for certain areas of research, or the difficulty of challenging the current paradigm?

Actually, you raise one important issue currently sweeping the sciences: replication of a study never results in publishable results. A direct consequence of this is that a fairly large fraction of results have yet to be duplicated, which is a cornerstone activity in establishing the validity of a result.

As for corporate/military sponsored research, you raise an interesting issue- if I perform research for (say) the Air Force that (eventually) results in a more accurate drone munition, am I somehow ethically responsible for the application and use of my results? To be sure, I am not required to accept funding from anyone, so I do have some autonomy here. Lots to discuss here...

 

[brainpushups]

replication of a study never results in publishable results. A direct consequence of this is that a fairly large fraction of results have yet to be duplicated, which is a cornerstone activity in establishing the validity of a result.

Indeed. And that is a shame. In the scramble for funding and time I wonder how that problem can be overcome... It's not a new one. From Feynman's famous '74 Caltech Commencement Speech:

"I was shocked to hear of an experiment being done at the big accelerator at the National Accelerator Laboratory, where a person used deuterium. In order to compare his heavy hydrogen results to what might happen with light hydrogen, he had to use data from someone else's experiment on light hydrogen, which was done on different apparatus. When asked why, he said it was because he couldn't get time on the program (because there's so little time and it's such expensive apparatus) to do the experiment with light hydrogen on this apparatus because there wouldn't be any new result. And so the men in charge of programs at NAL are so anxious for new results, in order to get more money to keep the thing going for public relations purposes, they are destroying--possibly--the value of the experiments themselves, which is the whole purpose of the thing. It is often hard for the experimenters there to complete their work as their scientific integrity demands."

As for corporate/military sponsored research, you raise an interesting issue- if I perform research for (say) the Air Force that (eventually) results in a more accurate drone munition, am I somehow ethically responsible for the application and use of my results? To be sure, I am not required to accept funding from anyone, so I do have some autonomy here. Lots to discuss here...

As to the ethical or moral responsibility for the use of one's research my initial reaction is that it depends on the intent of the researcher. If the work is being done for pure science (knowledge for knowledges sake) and then used by others for harm then I would say the researcher is not morally responsible for the application. Its a little stickier if you are doing research for the military. Perhaps your intent is pure knowledge, but the intent of your employer is not. I'd have a hard time absolving someone of moral responsibility for doing harm even if their intent was not so in such circumstances.

I wonder how often researchers working on military contracts contemplate the way their research fits into the global power dynamic. Feynman had some interesting words to say about that (in The Pleasure of Finding Things Out - used to be on youtube, but I didn't see it). He signed on to the Manhattan Project after he was convinced Germany was a threat to develop the bomb first. After Germany's defeat he says that he continued the project with the intent of getting success without reevaluating why he was doing it. After the initial elation of discovering that the bomb worked when dropped on Hiroshima he went into a significant depression; anticipating what world powers would do with such knowledge.

 

[ZapperZ]

Yes, sorry if I was unclear. I'm sure that the sort of blatant censorship that the Catholic Church did to Galileo and others is not experienced today, at least not in the context of religious belief. Certainly in the context of military and corporate research this can happen. What are your thoughts about censored corporate or military research? Is it ethical? Is it necessary? Should all scientific knowledge be open-source?

Please note that the restriction on the distribution of information under these circumstances are made perfectly clear when you either accept employment or funding from these sources. In other words, it should not be a surprised if the information that you acquire is restricted. If one cannot accept such restrictions, then one should be employed with such organization, or accept funding from those sources.

And no, all scientific knowledge should not be open sourced. We can fool ourselves into thinking that we are living in a Nirvana where no knowledge can be misused and used against someone, but we are not.

What about more subtle types of censorship like self-censorship, non-publication of null results, de facto censorship of lack of funding for certain areas of research, or the difficulty of challenging the current paradigm?

Those are not "censorship" in the common sense. No one is prohibiting the pursue of certain areas or publishing those ideas (i.e. one can try to find other avenues to pursue those). Lack of funding is a common issue from the very beginning of scientific endeavor. It is a reality that there is a finite amount of money available. Challenging the current paradigm has never been an issue. Science evolves because of that!

Unfortunately, the way this discussion is going, it is more of an issue suitable for a Social Science forum rather than Teaching and Education.

Zz.

 

[brainpushups]

Unfortunately, the way this discussion is going, it is more of an issue suitable for a Social Science forum rather than Teaching and Education.

Yes. It occurred to me after posting here that it would have probably been better elsewhere. Please move it if you wish.

Those are not "censorship" in the common sense.

I agree. The reason I listed such things is because of the article I linked at the beginning of the thread where the author cites such examples. He argues that censorship can be categorized in three ways: stopping the message, stopping the messenger, and establishing research priorities and proceeds to give examples.

 

[stedwards]

Concerning censorship outside of academia, there has been the UN 1995 IPCC report, politically modified. Also there "independent studies", where the data takers are pressured, through the chain of command, to skew data to satisfy the customers desired results.

Confirmation of the deflection of star light by the gravity of the sun, seems to have been a set of very tedious calculations in the time before computers and calculators. The calculations were repeated several times before agreement with Einstein's general relativity was obtained. Were the calculations repeated many more times after the (presumably) desired result of confirmation was obtained?

 

[OCR]

To bring the discussion back to contemporary issues I'd like to allow some time for a discussion about the role censorship plays in modern research.

This might be of interest... maybe?

 

[brainpushups]

This might be of interest... maybe?

Interesting. Thanks!

 

[OCR]

Thanks!

Sure... no problem.

 

[Andy Resnick]

This might be of interest... maybe?

A related story concerns my particular area of research- cell mechanosensation via the primary cilium. Back in 2001, a post-doc simply poked a cilium and saw a response; this was entirely novel and unheard of. She and her advisor could not find a journal to accept their manuscript, so the advisor (Ken Spring) simply accepted it into the journal he was Editor of:

http://www.ncbi.nlm.nih.gov/pubmed/11687880

This paper quite literally launched the entire field.

 

[256bits]

Does that mean its tough luck for scientists living in an extremist regime if they work in fields of research for which the knowledge gained has the potential to be used to do harm? There was an http://scitation.aip.org/content/aip/magazine/physicstoday/article/63/8/10.1063/1.3480068 in PT several years ago about how sanctions against Iran were hurting Iranian scientists in their ability to get equipment or even access information on the web. Yes, different from censorship, but a similar effect. Should the flow of information to Iranian scientists be restricted? Is the demarcation criterion for censoring scientific research in this way be whether there is a possibility the information could be used to do harm?

Regarding the history of censorship, at least somewhat more recent.
The old Soviet regime thought so ie the harm thing to its ideology.
https://en.wikipedia.org/wiki/Suppressed_research_in_the_Soviet_Union

Specific cases - ??

 

[tom aaron]

Actually, you raise one important issue currently sweeping the sciences: replication of a study never results in publishable results. A direct consequence of this is that a fairly large fraction of results have yet to be duplicated, which is a cornerstone activity in establishing the validity of a result.

As for corporate/military sponsored research, you raise an interesting issue- if I perform research for (say) the Air Force that (eventually) results in a more accurate drone munition, am I somehow ethically responsible for the application and use of my results? To be sure, I am not required to accept funding from anyone, so I do have some autonomy here. Lots to discuss here...

Excellent point on replication.

However, in my field I have zero desire to replicate anyone's results. I would think it the same for most scientists. No enthusiasm.

I have a dozen 'I-want-to-do' projects of my own...dozens of others I could potentially take on if so desired. There are only so many research scientists in my specific field (5 or 6). None of us are going to step out of our own research for 3 months, 6, or more and use our personal energy, funding. This is why we have publications that are peer reviewed. When I review someone's research before publication I am primarily looking at : Are these results by a qualified individual who used accepted scientific methods? It is about adding to the body of knowledge...is it credible? I don't need to replicate their work. The odds are their work will never be replicated...nor mine.

The vast, vast majority of science is not all that blockbuster in its impact. Not about a wonder drug or some breakthrough in Quantum theory. I can barely understand most of the abstracts in geology publications and I've been a geologist for almost 40 years. 99 plus percent of science is incremental steps that 99.9% of the public has no clue even exists.

 

[brainpushups]

However, in my field I have zero desire to replicate anyone's results. I would think it the same for most scientists. No enthusiasm.

Point taken. I can imagine there are instances where replication of a study is impractical or perhaps even impossible. But I also think Feynman's point is an important one (quote in post #7).

No doubt that the peer review process is one of the most important aspects of scientific practice. I try to make class time each year to discuss this with students and even for them to review each others' laboratory reports (which are modeled after an article rather than a typical 'lab report') and analysis results for the more open-ended experiments. I can imagine though that some less-than-credible results are sometimes published. I know I was surprised when I saw an article about gibberish papers that had to be removed even though they were supposedly peer reviewed. See this article.

 

[tom aaron]

Point taken. I can imagine there are instances where replication of a study is impractical or perhaps even impossible. But I also think Feynman's point is an important one (quote in post #7).

No doubt that the peer review process is one of the most important aspects of scientific practice. I try to make class time each year to discuss this with students and even for them to review each others' laboratory reports (which are modeled after an article rather than a typical 'lab report') and analysis results for the more open-ended experiments. I can imagine though that some less-than-credible results are sometimes published. I know I was surprised when I saw an article about gibberish papers that had to be removed even though they were supposedly peer reviewed. See this article.

Thanks. That's an interesting article. Bookmarked it. Will google more about it.

In my field anything blatantly bogus would stand out like an elephant at a mouse convention. Also, papers are published in geology journals.

Part of the issue is that there may be more and more a blurry line between what is accepted in a discipline as research and what is good quality but more educated opinion. I have recently been confused a few times about whether an on line piece is published research or not.

Also, physical access to some journals is becoming harder. After decades I know what is out there but a real quagmire for anyone starting out. I have originals or photo copies of thousands of articles but...when I hang up the rock hammer? Yikes. I've seen some disturbing trends, especially in the US Geological Survey. When cut backs and a top researcher retires...his files go out into the dumpster! Files that may have been built up over a century!

Not to rattle on forever but there are also positives. My area has zero value technically, minimal effect on resources, etc. Because of this in my discipline we Canadians, the Americans, Russians, Chinese, etc. have always been like a big fraternity. Everything exchanged freely, no politics. To be honest, we just ignored and bypassed any restrictions, rules. Best to have never asked permission...just interact and fein ignorance if anything was ever questioned. It never was. We were probably among the first to send bootleg copies of early Filemaker and word processing programs to 'the Commies' so we could all exchange files easier.

 

[Dr. Courtney]

Please note that the restriction on the distribution of information under these circumstances [military and corporate] are made perfectly clear when you either accept employment or funding from these sources.

Not always. When I was on the faculty of the US Air Force Academy, we were presumed to operate under the same guidelines of academic freedom as faculty at any other college or university. However, before submitting material for publication, it was required that the material be approved by the research office even in cases where there were no classified, confidential, or non-disclosure issues at the funding or design stages. In this review, material could be censored for a variety of reasons. Obstensibly, there were reasonable criteria (compliance with privacy laws, not revealing secrets, etc.), but there was also the possibility of material being censored for other reasons such as political correctness.

When my wife was on the West Point faculty (USMA), she had the same review and approval process.

 

[Dr. Courtney]

Another common censorship issue concerns the unwillingness of many publishers to publish animal research that was not approved beforehand by an institutional animal care and use committee. There are often funding and legal requirements for live animal research, but there are broad exceptions for things like wildlife studies and research in agricultural settings. Many of the publishers are much stricter and thus end up censoring research results that were in complete compliance with legal and funding requirements, because the exceptions in the funding and legal requirements did not require prior IACUC approval.

The IACUC rules were intended to apply to laboratory animals not to a study on which feed rations help produce the most milk, what water temperatures produce the fastest growth in hatchery trout, or how an oil spill effected wildlife populations.

 

[Andy Resnick]

Another common censorship issue concerns the unwillingness of many publishers to publish animal research that was not approved beforehand by an institutional animal care and use committee. <snip>

I'm not sure I'd call that 'censorship', but rather a check on the ethical conduct of science. IACUC etc. are important safeguards.

 

[Dr. Courtney]

I'm not sure I'd call that 'censorship', but rather a check on the ethical conduct of science. IACUC etc. are important safeguards.

The definition of censorship is:

"the practice of officially examining books, movies, etc., and suppressing unacceptable parts."

Censorship is acceptable if the "unacceptable parts" occur because of ethical violations such as not having IACUC approvals where they are required.

However, many journal publishers have gone much further, refusing to publish papers without IACUC approvals in cases where IACUC approvals were not required in the first place.

http://fishbull.noaa.gov/1094/1094oconnell.pdf

Consider the linked paper. This study was conducted without an IACUC approval and published in the peer-reviewed journal Fishery Bulletin. The study was in complete compliance with all laws of the US, ethical, and institutional standards. Yet, many journals would not even consider this paper for publication, because if the lack of an IACUC approval. This is a kind of censorship.

One wonders where this precedent may lead. Will journals one day censor papers from institutions not holding to all their values in human rights and employment? Will journals censor papers from states that permit flying of the confederate flag? Lots of abuses are possible under the guise of ensuring ethical conduct.

 

[Andy Resnick]

<snip>

Consider the linked paper. This study was conducted without an IACUC approval and published in the peer-reviewed journal Fishery Bulletin. The study was in complete compliance with all laws of the US, ethical, and institutional standards. Yet, many journals would not even consider this paper for publication, because if the lack of an IACUC approval. This is a kind of censorship.

<snip>

IACUC regulations regarding wildlife are (AFAICT) at least in the discussion phase:

http://www.google.com/url?sa=t&rct=...BwGWYRKHQ&sig2=U8a3v8dl6h_598X14Xqz5w&cad=rja

http://www.google.com/url?sa=t&rct=...fwI8IizXaFeGhAD5Q&sig2=yfc9eQ9E97GfX76AFtNu3Q

On your linked paper, it's not clear (to me) if IACUC approval was sought and found to be immaterial or simply not sought. If the study is indeed in 'full compliance', as you claim, then the relevant IACUC committee should have reviewed the study plan- otherwise, the study was not "in full compliance" with all ethical standards.

 

[Andy Resnick]

Excellent point on replication.

However, in my field I have zero desire to replicate anyone's results. I would think it the same for most scientists. No enthusiasm.

<snip>

Exactly. However, this becomes relevant when considering, for example, pharmaceutical company-funded drug studies (which are, by and large, of high quality). And this includes small-scale projects, like validation of a primary antibody or gene target for laboratory diagnostic use.

 

[Dr. Courtney]

IACUC regulations regarding wildlife are (AFAICT) at least in the discussion phase:

http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&ved=0CCcQFjAB&url=http://sites.nationalacademies.org/cs/groups/pgasite/documents/webpage/pga_084510.pdf&ei=vouaVeDiD8Wo-QG5o4ngDg&usg=AFQjCNGmn44MMHHzV76N0yvr2BwGWYRKHQ&sig2=U8a3v8dl6h_598X14Xqz5w&cad=rja

http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=5&cad=rja&uact=8&ved=0CDsQFjAE&url=http://grants.nih.gov/grants/olaw/140320_Wildlife_slides.pdf&ei=vouaVeDiD8Wo-QG5o4ngDg&usg=AFQjCNGuFdy81VN7rfwI8IizXaFeGhAD5Q&sig2=yfc9eQ9E97GfX76AFtNu3Q

Most people recognize that expecting compliance with rules that have not been passed yet is an example of an ex post facto law, which is inherently unfair and evil.

It's the equivalent of punishing students for using calculators on events that occurred BEFORE a rule against use of calculators had been announced.

How can a scholar recognize that ex post facto rules are unfair and evil in some contexts, but suggest is isn't unreasonable censorship to apply them in others?

 

[tom aaron]

Just a note... Much of science isn't about internal US regulations.

Many issues of 'ethics' get debated at a domestic level. This is especially the case in biological sciences. Non scientists may debate the use of stem cells, cloning, genetic engineering, double blind drug studies, etc.

In the meantime, research just goes ahead elsewhere in the world...China, India, Indonesia, etc. So yes, there are often barriers to some science but the research often isn't restricted by boundaries.

I was in China for a small geology conference...at a city I had never heard of previously. It had 4 million people and the University, heavy on the sciences, was uncanny in its size. The Chinese government is now the largest regulator of science agenda in the world. I have no idea how it exercises that power or if it 'censors' science.

 

[Dr. Courtney]

Actually, you raise one important issue currently sweeping the sciences: replication of a study never results in publishable results. A direct consequence of this is that a fairly large fraction of results have yet to be duplicated, which is a cornerstone activity in establishing the validity of a result.

Venues like PLoS One where publication depends on quality rather than novelty should help with the publication of studies that are essentially replications of earlier results. Conference proceedings and technical reports are also venues where replication of earlier results are commonly published. There is also great value in repeating experiments in cases where the new experiment can be performed with greater accuracy. Many important scientific findings result from reducing the error bars. Repeating an important experiment with improved accuracy usually yields a paper that's publishable in a peer-reviewed venue. I can point to examples on my own CV. For example, this experiment (http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.74.1538) is a more accurate replication of this (http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.47.83 ) earlier experiment.

One thing that we learned at the Air Force Academy is that performing experiments that double check manufacturers' product specifications often yields results contradictory to the published specifications. Such work is usually appropriate for the grey literature rather than the peer-reviewed journals, but it is suprising how many manufacturers are exaggerating their specs. I worked with cadets on a number of projects relating to things like aerodynamic drag, breaking strength, and rocket motor thrust curves. One could make a fruitful enterprise in the area of laboratory equipment: thermometers, scales, electronics, etc.

 

[Dr. Courtney]

Another area in which censorship is expanding in science is the refusal of journals to publish papers unless the authors make available their raw data, source code, etc.

Now, I regard the sharing of data and source code as reasonable requirements of funding agencies in many cases, but I can understand why is might be beneficial for scientists to publish results in scholarly venues without publishing raw data and source code.

 

[brainpushups]

Another area in which censorship is expanding in science is the refusal of journals to publish papers unless the authors make available their raw data, source code, etc.

This is sort of off topic, but..

To what extent does the requirement for sharing raw data enter into the peer review process? I don't recall ever seeing a paper with raw data included. I would expect that reviewers probably don't spend their time sifting through data unless there is something questionable in the analysis. Is it typical for data to be submitted with the manuscript? Also, to what extent is data available? For example, if you were to perform a meta analysis I imagine that having the raw data from the studies would be important. Would you need to contact the authors of each study?

 

[Dr. Courtney]

To what extent does the requirement for sharing raw data enter into the peer review process?

Not at all, unless the journal requires it and asks the reviewer to confirm that the requirements are met. I've reviewed dozens of papers and since the journals did not ever require it, it has never come up. I've never felt the need to repeat the data analysis in the level of detail that would require raw data in my role as a reviewer.

I have co-authored several papers (and I am aware of others) commenting on data analysis errors in published papers that slipped through the peer-review process. So, it's not that I don't think raw data should ever be requested by a reviewer, it's just that I've never had actual occasion to make the request. I've also authored dozens (approaching 100) of peer-reviewed papers, and I do not recall ever being asked for the raw data as part of the review process.

On occasion, there are data sharing requests made after a conference or after a paper is published by colleagues in the same or related field. These requests are always worked out amicably, but usually include an understanding regarding how the data will be used and that the data will not be passed along to third parties. There is often some agreement that allows data and source code to flow both ways in a mutually beneficial exchange.

I don't recall ever seeing a paper with raw data included.

We've done it a time or two when we expected reader interest or that it would save time answering anticipated reader requests, but it is an unusual occasion.

I would expect that reviewers probably don't spend their time sifting through data unless there is something questionable in the analysis. Is it typical for data to be submitted with the manuscript?

Right. Raw data is not typically submitted with manuscripts.

Also, to what extent is data available? For example, if you were to perform a meta analysis I imagine that having the raw data from the studies would be important. Would you need to contact the authors of each study?

The NIH and a minority of other grant sources now require making raw data available, as does PLoS One and a minority of peer-reviewed journals. See:

http://grants.nih.gov/grants/policy/data_sharing/data_sharing_guidance.htm

and

http://blogs.plos.org/everyone/2014/02/24/plos-new-data-policy-public-access-data-2/

Many private funding and Dept of Defense funding sources actually prohibit the sharing of raw data under their non-disclosure and confidentiality terms.

But in most cases, meta-analysis of raw data will require contacting the authors of each study. We've had varying levels of success requesting raw data from different sources.

 

[tom aaron]

This is sort of off topic, but..

To what extent does the requirement for sharing raw data enter into the peer review process? I don't recall ever seeing a paper with raw data included. I would expect that reviewers probably don't spend their time sifting through data unless there is something questionable in the analysis. Is it typical for data to be submitted with the manuscript? Also, to what extent is data available? For example, if you were to perform a meta analysis I imagine that having the raw data from the studies would be important. Would you need to contact the authors of each study?

That's a good question

In my field we do not recognize a paper unless there is current or future access. This isn't in writing but the expectation in my discipline. In the past we have rejected findings from China because of issues with having original curated specimens sent to us in a reasonable time frame. This has changed for the positive in the last 20 years. I used to feel bad for Chinese researchers who did excellent work but were frustrated by bureaucratic incompetence.

 

[Andy Resnick]

This is sort of off topic, but..

To what extent does the requirement for sharing raw data enter into the peer review process? I don't recall ever seeing a paper with raw data included. I would expect that reviewers probably don't spend their time sifting through data unless there is something questionable in the analysis. Is it typical for data to be submitted with the manuscript? Also, to what extent is data available? For example, if you were to perform a meta analysis I imagine that having the raw data from the studies would be important. Would you need to contact the authors of each study?

Yeah, this thread is veering off-topic... To the extent that journals provide clear requirements for acceptance, if authors choose not to meet those requirements, one cannot claim censorship. There are many journals to publish in, and each journal has it's own set of requirements. Some journals allow for "supplemental results", often online-only, that can have a lot of the low-level detail (detailed procedures, raw data sets, etc.) that would obscure presentation of the main findings. To be sure, if a colleague requests experimental sample material, I am generally happy to provide it as freely as possible.

In terms of making raw data publicly available, there seems to be two broad approaches. One view is "This research was paid for by tax dollars and is therefore property of the citizenry- by having to purchase a journal article, I am paying twice to obtain results. And besides, wouldn't it be neato-fun to have all those interested citizen scientists look through the raw data and maybe find something that the authors missed?" (for example, the Sloan digital sky survey data releases)

The other side takes a position of "By letting the raw data roam freely, there are no controls on someone with ulterior motives from re-slicing the data to generate alternative results." Think of what would happen if lawyers got access to raw patient data and could identify those who were harmed either by being given a placebo or being that one-in-a-million with a bad reaction. Climate science is another prime example of what can go wrong, and even wrongerer...

The current middle ground is that raw data (lab notebooks and the like) does not have to be made public; the final product (peer-reviewed journal article) must be. To safeguard the journals themselves (publishing papers costs money), many journals are allowed to 'embargo' for a certain time before the articles become free. Similarly, there are data repositories (GenBank, etc.) which experimenters are obligated to use as needed.

None of this constitutes 'censorship', tho.