Is the Scientific Community Covering Up Mistakes?

[theCandyman]

Here's an amusing (if depressing) tale of a physicist's attempt to publish a comment on another's work.

http://www.scribd.com/doc/18773744/How-to-Publish-a-Scientific-Comment-in-1-2-3-Easy-Steps

I have to say, his ending comment that when journals publish a paper with a mistake, both the editors and authors try to cover it up astonishes me. Others are going to build upon a broken foundation just because the authors of the original work don't have the guts to come out and say "I am wrong."

 

[Sorry!]

Here's an amusing (if depressing) tale of a physicist's attempt to publish a comment on another's work.

http://www.scribd.com/doc/18773744/How-to-Publish-a-Scientific-Comment-in-1-2-3-Easy-Steps

I have to say, his ending comment that when journals publish a paper with a mistake, both the editors and authors try to cover it up astonishes me. Others are going to build upon a broken foundation just because the authors of the original work don't have the guts to come out and say "I am wrong."

I found it pretty funny. It is a shame though.

 

[physics girl phd]

Rick Trebino's a fabulous character... huzzah to him for his persistence (and for sharing)!

 

[Moonbear]

Sad and funny!

How does a journal and its editorial staff become THAT dysfunctional?

I'm glad he also added a list of things to consider to fix the system, although I don't agree with all of them, I do think most make sense.

One I disagree with are requiring someone to write to the author of the work they are contradicting. That seems like the same sort of potential conflict loop as he wound up in trying to get replies to his comment from the original authors of the article. Of course, and editor COULD choose that person as one of the reviewers.

The other I disagree with is paying reviewers. It's part of the job description of being an academic, so in essence, we already ARE paid to be a reviewer. It's also part of the accepted responsibility of being an author, that if you'd like a speedy review on your own submissions, you'll return the favor and help be a reviewer for other submissions. I think there might even be more of a conflict if reviewers were paid...there'd be an awful lot of motivation to crank out reviews quickly to get paid sooner, but not necessarily as much motivation to be careful about the process.

I do think some of the problem is the rapid review process that happens now. It's one of those cases where in the old system, you lost a lot of time, and new publications weren't really all that new by the time they came out, because it could take a year from submission to publication. But, now they've cut back on that time, mostly by pressuring reviewers to return their reviews in 2 weeks instead of 2 months. You used to have time to read a manuscript, mull it over, send a grad student to get some of the cited references you were unfamiliar with, take time to read them and decide if you agreed they were being represented accurately, consider the results and conclusions, etc. Now you get notices that your review is overdue before you've even gotten that obscure reference sent by inter-library loan (it's always the obscure one that's suspicious).

On a side note about electronic review submissions (it sounds like some of the issues in this case were also related to a journal's electronic submission site)...has anyone else ever gotten an automatic review overdue notice telling you your review is -1 days late on the day before it is due? That always aggravates me, and I've gotten it with at least two journals I've reviewed for. Instead of sending a gentle reminder that the review is due tomorrow, please remember to submit it, you get a notice that it's overdue by negative days. GRRRRR! Then I usually end up waiting an extra day to submit it, just so I can get a justified overdue notice for positive days. I don't like getting yelled at by a computer that I'm late when it's still a day early.

 

[FredGarvin]

I applaud him for his patience. I would have been throwing dead bodies out the door well before a year later. Especially if everyone really was talking trash about my life's work.

I wish he would have listed the journal and the article.

 

[Proton Soup]

if this is how things go in physics, i wonder how much more so in the exciting world of climate science?

edit: actually, i see now he mentioned that exact thing in addendum.addendum.10

 

[Phrak]

Who is Rick Trebino, and upon which scientific disipline was he commenting? I mean, it's hardly fair to put them all in the same bag.

There's something about global warming --or global tilting in 'sequence of events' #59.

 

[Proton Soup]

http://scienceblogs.com/ethicsandscience/2009/08/the_saga_of_the_journal_commen.php

look in the comments

edit: very end, not Stemwedel's comments

 

[rootX]

From #1 to #2

Nonsense IMO. How would such a paper end up in a prestigious journal (like IEEE)?

I stopped at #5 because it all looked BS to me.

 

[junglebeast]

I'm surprised he didn't take this to court.

 

[Phrak]

http://scienceblogs.com/ethicsandscience/2009/08/the_saga_of_the_journal_commen.php

look in the comments

edit: very end, not Stemwedel's comments

Which response number?

Instead of dancing around this issue, I've long questioned the veracity if the global warming group as representing a scientific disipline.

Is it science because they advertise themselves to have membership in this category? How about Scientology. Is it a science?

The long and eloquent prose in their meandering publications and commentary are not a comfort, but a source of suspicion. If the author has something to say, spit it out! Don't decorate it with obfuscatory prose If it's not the intent to obscure the deductive chain, don' obscure it.

Having said all that, I want to know what journal and what article and what data he took offense to.

 

[Proton Soup]

Which response number?

it's not like there are many to consider. alternatively, you could try the google.

 

[junglebeast]

Well, there it is

http://www.opticsinfobase.org/abstract.cfm?uri=ol-34-17-2602

 

[Vanadium 50]

How does a journal and its editorial staff become THAT dysfunctional?

I'd place the blame on their parentage.

 

[Cyrus]

Which response number?

Instead of dancing around this issue, I've long questioned the veracity if the global warming group as representing a scientific disipline.

... I'm not sure what you mean by 'global warming group'. Climatologists?

 

[TheStatutoryApe]

The authors of an earlier paper [Opt. Lett.32, 3558 (2007)] reported two “ambiguities” in second-harmonic-generation frequency-resolved optical gating (FROG). One ambiguity is simply wrong--a miscalculation. The other is well known and easily avoided in simple well-known FROG variations. Finally, the authors' main conclusion--that autocorrelation can be more sensitive to pulse variations than FROG--is also wrong.

Wow... that's really short.

 

[Borg]

How does a journal and its editorial staff become THAT dysfunctional?

I'd place the blame on their parentage.

I think that the Verizon customer service reps are moonlighting.

 

[Phrak]

... I'm not sure what you mean by 'global warming group'. Climatologists?

Well.. It was a misplaced rant, anyway. This was over stuff in an optics journal.

And thanks, Proton. Found it.

 

[Phrak]

These are in chronological order.

1) This is a link to the initiating article by YKT, Optics Letters, Vol. 32, Issue 24, pp. 3558-3560 (2007),

http://www.opticsinfobase.org/abstract.cfm?uri=ol-32-24-3558"

Amplitude ambiguities in second-harmonic generation frequency-resolved optical gating

Abstract

"We construct field shapes with distinct amplitude profiles that have nearly identical second-harmonic generation frequency-resolved optical gating (SHG FROG) traces. Although such fields are not true mathematical ambiguities, they result in experimentally indistinguishable FROG traces. These fields are neither time-reversed copies nor pulselets with a mere relative phase difference, which are well known nontrivial ambiguities for SHG FROG. We also show that for certain example fields, second-order interferometric autocorrelation is more sensitive to the pulse shape than is SHG FROG."

"History
Original Manuscript: July 11, 2007
Manuscript Accepted: October 1, 2007
Published: December 14, 2007"


2) YTK correct some errors in the origin letter, Optics Letters, Vol. 33, Issue 23, pp. 2854-2854

http://www.opticsinfobase.org/ol/abstract.cfm?URI=ol-33-23-2854"

Amplitude ambiguities in second-harmonic generation frequency-resolved optical gating: erratum

Abstract
"In our recent Letter [Opt. Lett.32, 3558 (2007)], the values of frog errors (Δe) reported for the examples in Fig. 1 and 3 are not consistent with the trace-normalized frog errors conventionally reported in literature."

"History
Original Manuscript: November 6, 2008
Published: November 25, 2008"


3) Trebino et al, published comment, Optics Letters, Vol. 34, Issue 17, pp. 2602-2602

Amplitude ambiguities in second-harmonic-generation frequency-resolved optical gating: comment

http://www.opticsinfobase.org/abstract.cfm?uri=ol-34-17-2602"

Abstract

"The authors of an earlier paper [Opt. Lett.32, 3558 (2007)] reported two “ambiguities” in second-harmonic-generation frequency-resolved optical gating (FROG). One ambiguity is simply wrong--a miscalculation. The other is well known and easily avoided in simple well-known FROG variations. Finally, the authors' main conclusion--that autocorrelation can be more sensitive to pulse variations than FROG--is also wrong."

"History
Original Manuscript: July 30, 2008
Manuscript Accepted: February 17, 2009
Revised Manuscript: February 16, 2009
Published: August 20, 2009"


4) YTK's response to Trebino et al, Optics Letters, Vol. 34, Issue 17, pp. 2603-2603

Amplitude ambiguities in second-harmonic-generation frequency-resolved optical gating: reply to comment

http://www.opticsinfobase.org/ol/abstract.cfm?URI=ol-34-17-2603

Abstract

"The error calculations in our Erratum [Opt. Lett.33, 2854 (2008)] are correct and consistent with the numbers presented in the Comment [Opt. Lett.34, 2602 (2009)] on our Letter [Opt. Lett.32, 3558 (2007)]. However, we still find that the pulses in Fig. 3 pose a problem in correctly reconstructing the electric field shape."

"History
Original Manuscript: March 24, 2009
Manuscript Accepted: July 16, 2009
Revised Manuscript: July 15, 2009
Published: August 20, 2009"

Well, data with dates, by various claimants, supposedly. I've just now put this together, and haven't yet tried to discover who did what to whom, and when they done it to whos-it.

 

[junglebeast]

Well, data with dates, by various claimants, supposedly. I've just now put this together, and haven't yet tried to discover who did what to whom, and when they done it to whos-it.

yes, this history is consistent with that described in the "how to 123" guide...

the first Errata was published by the authors in order to refute the comment which was LATER published, because they obtained (dishonestly) information of the impending comment.

The comment was published, and they replied and continued to deny that they were ever wrong.

 

[Dragonfall]

Does this happen with math journals? I've always imagined math journals would be much "cleaner" in this respect.

 

[Cyrus]

This is the kind of BS that should end careers.

 

[Vanadium 50]

This illustrates a problem with the society journals. Toni Taylor (the senior author on the paper Rick Trebino is commenting on) is fairly high up in the organization of the OSA, which publishes Optics Letters. As it happens, the editorial decisions of society journals are quite independent of the society that publishes them, but to someone who doesn't know that, this looks very bad.