Register to reply

Precognition paper to be published in mainstream journal

Share this thread:
Evo
#73
Jan12-11, 12:12 PM
Mentor
Evo's Avatar
P: 26,541
Quote Quote by Ivan Seeking View Post
Until we see a published rebuttal, all arguments are anecdotal or unsupported. Unpublished papers count at most as anecdotal evidence, which never trumps a published paper.

We don't use one standard for claims we like, and another for claims we don't like. See the S&D forum guidelines.
The rebuttal is going to be published in the same Journal at the same time as the Berm paper, so they are on equal footing.

Dr. Wagenmakers is co-author of a rebuttal to the ESP paper that is scheduled to appear in the same issue of the journal.
http://www.nytimes.com/2011/01/06/science/06esp.html
Ivan Seeking
#74
Jan12-11, 12:13 PM
Emeritus
Sci Advisor
PF Gold
Ivan Seeking's Avatar
P: 12,500
Technically, I am making a special exception to allow an unpublished rebuttal to a published paper. If the tables were turned, it would never be allowed. That would be considered crackpot or pseudoscience.
Ivan Seeking
#75
Jan12-11, 12:14 PM
Emeritus
Sci Advisor
PF Gold
Ivan Seeking's Avatar
P: 12,500
Quote Quote by Evo View Post
The rebuttal is going to be published in the same Journal at the same time as the Berm paper, so they are on equal footing.



http://www.nytimes.com/2011/01/06/science/06esp.html
Sorry, okay. I knew there were objections to be published, but not a formal paper.
harrylin
#76
Jan12-11, 12:17 PM
P: 3,187
Quote Quote by Ivan Seeking View Post
In science, an unpublished paper counts for nothing. They are only allowed for discussion here as they do often constitute anecdotal evidence for the claim or argument.
Actually, publication is simply a means for dissemination, and peer review is merely a noise filter for quality control (which both papers discussed here already passed). The same is also used for quality control of Wikipedia and discussion topics on this site.

Dissemination filters must however not be confused with science or the scientific method! What matters in science are facts and theories, and the verification or disproof of those theories.

Entries for further reading can be found in:
http://en.wikipedia.org/wiki/Scientific_method

Harald
harrylin
#77
Jan12-11, 12:26 PM
P: 3,187
Quote Quote by Evo View Post
The rebuttal is going to be published in the same Journal at the same time as the Berm paper, so they are on equal footing.

http://www.nytimes.com/2011/01/06/science/06esp.html
Thanks, I already wrote twice that they are on equal footing because they are both peer reviewed... but I didn't know that they were to be published in the same journal.

Perhaps it's done on purpose, in order to push for a change in statistical methods.
nismaratwork
#78
Jan12-11, 12:41 PM
P: 2,284
Quote Quote by Ivan Seeking View Post
In science, an unpublished paper counts for nothing. They are only allowed for discussion here as they do often constitute anecdotal evidence for the claim or argument.
I agree... just not in this situation for reasons you already have accepted, and I don't need to restate.


harrylin: JUST a filter? You make that sound so small, but it's the primary mechanism that ensures what you linked to is being FOLLOWED.
Jack21222
#79
Jan12-11, 01:11 PM
P: 772
Quote Quote by Ivan Seeking View Post
In science, an unpublished paper counts for nothing. They are only allowed for discussion here as they do often constitute anecdotal evidence for the claim or argument.
So on this forum, nobody is allowed to argue against a paper unless they themselves have that argument in a published paper? I don't follow. The Bem paper has some very basic flaws that I could have easily pointed out without referencing the paper that I did. However, that paper put it much more eloquently than I could.

Valid arguments don't become invalid just because they're not published any more than invalid arguments become valid just because they're published.

In any case, using the same set of data to both come up with AND test a hypothesis is a horrible methodological flaw that I hope anybody here could see, with or without a published or unpublished paper as a reference.
Ygggdrasil
#80
Jan12-11, 01:13 PM
Other Sci
Sci Advisor
P: 1,388
Quote Quote by Ivan Seeking View Post
Technically, I am making a special exception to allow an unpublished rebuttal to a published paper. If the tables were turned, it would never be allowed. That would be considered crackpot or pseudoscience.
It seems a bit severe to discount non-peer-reviewed rebuttals when the Bem paper has not actually appeared in print yet. If the precognition paper were 5 years old, I would support trying to limit the discussion to rebuttals appearing in the published literature, but given that the findings are very new, it seems prudent to consider unpublished responses from experts in the field. As very few researchers have had time to come up with experiments to address Bem's claims, let alone get them peer reviewed, limiting discussion to peer-reviewed findings in essence invalidates any criticism of the Bem paper.

Should these unpublished rebuttals be taken with a grain of salt? Yes, just as any research findings, peer-reviewed or not, should be met with skepticism.

Edit: Also, here is a peer-reviewed paper that discusses many of the flaws in study design and bias discussed in this thread:
Ioannidis JPA (2005) Why Most Published Research Findings Are False. PLoS Med 2(8): e124. doi:10.1371/journal.pmed.0020124
Abstract

There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.
nismaratwork
#81
Jan12-11, 02:24 PM
P: 2,284
Ygggdrasil, Jack... He already accepted the points you're making!

Quote Quote by Ivan Seeking
Sorry, okay. I knew there were objections to be published, but not a formal paper.
Otherwise everyone seems to arguing for the same rigor to be applied, so what's the problem?
Jack21222
#82
Jan12-11, 02:47 PM
P: 772
Quote Quote by nismaratwork View Post
Ygggdrasil, Jack... He already accepted the points you're making!



Otherwise everyone seems to arguing for the same rigor to be applied, so what's the problem?
It would be absurd to apply the same rigor to comments on an internet forum as in a peer-reviewed journal. Ivan seemed to be implying that all comments made here had to be peer-reviewed before he'd consider them valid.
nismaratwork
#83
Jan12-11, 02:50 PM
P: 2,284
Quote Quote by Jack21222 View Post
It would be absurd to apply the same rigor to comments on an internet forum as in a peer-reviewed journal. Ivan seemed to be implying that all comments made here had to be peer-reviewed before he'd consider them valid.
Jack, we both have been here long enough to KNOW that's not what he was saying. Was he wrong, yeah, was he being absurdist?... no.
Jack21222
#84
Jan12-11, 02:56 PM
P: 772
Quote Quote by nismaratwork View Post
Jack, we both have been here long enough to KNOW that's not what he was saying. Was he wrong, yeah, was he being absurdist?... no.
He wouldn't comment on the content of the post because it wasn't peer-reviewed. You tell me what that means.
JaredJames
#85
Jan12-11, 02:59 PM
P: 3,387
Honestly people, this is going in circles.

We've dealt with the 'finer points' of the documents, how about discussion gets back on topic.
nismaratwork
#86
Jan12-11, 03:00 PM
P: 2,284
Quote Quote by Jack21222 View Post
He wouldn't comment on the content of the post because it wasn't peer-reviewed. You tell me what that means.
I admit, that goes beyond my ability to explain; I can only say that I don't believe that's what Ivan intended, but obviously he speaks for himself.
nismaratwork
#87
Jan12-11, 03:00 PM
P: 2,284
Quote Quote by jarednjames View Post
Honestly people, this is going in circles.

We've dealt with the 'finer points' of the documents, how about discussion gets back on topic.
That would be nice!!!
harrylin
#88
Jan12-11, 04:28 PM
P: 3,187
Quote Quote by Ygggdrasil View Post
[..]

Edit: Also, here is a peer-reviewed paper that discusses many of the flaws in study design and bias discussed in this thread:
Ioannidis JPA (2005) Why Most Published Research Findings Are False. PLoS Med 2(8): e124. doi:10.1371/journal.pmed.0020124
Wow that's an amazing paper! But yes, it looks like Bem's paper and the criticism on it is being published to provide a case example of just that problem...
pftest
#89
Jan13-11, 04:12 AM
P: 271
Ivan was right that a published peer reviewed paper has more credibility than an non-published non-peer reviewed one. He just didnt know that the criticism paper (which was linked to in the NYT article that i posted) was also to be published. We cant just go "hey someone criticised that scientific peer reviewed paper that i dont like, that means its false", especially not in a skepticism and debunking forum. It will take time for science to show whether Bem has actually found ESP or not.

Quote Quote by harrylin View Post
Wow that's an amazing paper! But yes, it looks like Bem's paper and the criticism on it is being published to provide a case example of just that problem...
Where'd you get that from? That paper is 5 years old and it applies to the majority of published research, not just a single ESP paper. It specifically refers to the area of biomedical research.
nismaratwork
#90
Jan13-11, 04:57 AM
P: 2,284
Quote Quote by pftest View Post
Ivan was right that a published peer reviewed paper has more credibility than an non-published non-peer reviewed one. He just didnt know that the criticism paper (which was linked to in the NYT article that i posted) was also to be published. We cant just go "hey someone criticised that scientific peer reviewed paper that i dont like, that means its false", especially not in a skepticism and debunking forum. It will take time for science to show whether Bem has actually found ESP or not.

Where'd you get that from? That paper is 5 years old and it applies to the majority of published research, not just a single ESP paper. It specifically refers to the area of biomedical research.
The paper applies to statistics.

Hey... omg...

Einstein's paper on SR is what, 96 years old just from the date of PUBLISHING?! Quick, everyone... is 'c' increasing?!!? No? Hmmm, well GR is old, anyone suddenly find falsification for that?

The irony of course, is that you could have made the same argument logically (if poorly) from the OPPOSITE perspective, and been right: the longer a theory or paper is being peer-reviewed, attacked, worked on... the more credible it is. Science seeks to tear something down, in the hopes that it CAN'T and will be left with something valid... it's the destructive element of the process (not in a bad way), and a means of quality control in results AND methodology!


Register to reply

Related Discussions
Has anybody here been published in a scientifc journal ? Academic Guidance 85
Published paper is needed General Physics 10
New published issue of Journal of Physics Students (JPS) General Physics 0
New published issue of Journal of Physics Students (JPS) General Physics 0
Getting Published in a Journal General Discussion 2