Sci-Fi Dream: 120+ Computer-Generated Papers Removed by Springer & IEEE

[Pythagorean]

Every now and then, that kind of naive hopeful interest that got me interested in science gets a little more diminished. Once I get my PhD, I think I'll just be a McDonald's Manager so I can tell people with B.S.'s what to do.

The publishers Springer and IEEE are removing more than 120 papers from their subscription services after a French researcher discovered that the works were computer-generated nonsense.

http://www.nature.com/news/publishers-withdraw-more-than-120-gibberish-papers-1.14763

 

[HomogenousCow]

"Labbé developed a way to automatically detect manuscripts composed by a piece of software called SCIgen, which randomly combines strings of words to produce fake computer-science papers. SCIgen was invented in 2005 by researchers at the Massachusetts Institute of Technology (MIT) in Cambridge to prove that conferences would accept meaningless papers — and, as they put it, “to maximize amusement” "

Lol

 

[DrClaude]

Note that these were all conference proceedings. They do not carry much weight in most fields, except maybe computer science from what I've read.

 

[Pythagorean]

Note that these were all conference proceedings. They do not carry much weight in most fields, except maybe computer science from what I've read.

True, but they're still (supposed to be) peer-reviewed!

 

[SteamKing]

True, but they're still (supposed to be) peer-reviewed!

Here's a paranoid thought: Unless the peer reviewers are dodgy, too!

 

[Ryan_m_b]

I think this phenomenon points out three possibilities, none of which are mutually exclusive:

1) Peer reviewers in many journals don't work to high enough standards
2) The technology for computer generated manuscripts is becoming a lot more convincing
3) There's something wrong with how we write scientific papers that makes them difficult to distinguish from crap.

 

[DrClaude]

True, but they're still (supposed to be) peer-reviewed!

It probably depends on how you define "peer."

Was each paper really sent to a specialist of the topic, or just reviewed by a commitee to see if it was acceptable for the conference? My guess is that it was the latter.

 

[ZapperZ]

True, but they're still (supposed to be) peer-reviewed!

The "peer-review" of conference proceedings is often very weak in most cases. Often, the manuscripts are sent to people with a peripheral knowledge of the subject matter. I've been given conference proceedings manuscript to review that was totally out of my field, and the only reason I was given it was because they happened to use photoemission spectroscopy to study the material.

Most of us who read conference proceedings tend to know the level of "validity" that we can expect. We also tend to know which group of authors to put more credibility on, and which type of conference proceedings have better quality. Unfortunately, someone from outside the field would not be able to tell the difference.

Zz.

 

[Pythagorean]

It probably depends on how you define "peer."

Was each paper really sent to a specialist of the topic, or just reviewed by a commitee to see if it was acceptable for the conference? My guess is that it was the latter.

No details are given on the implementation.

I think we should expect good quality control measures at conferences, especially if they claim to be peer-reviewed. Here we see an indication that it's not really peer-reviewed, at least not to any reasonable degree. If you pay money to go to a conference proceeding, you hope you're getting good science.

 

[ZapperZ]

No details are given on the implementation.

I think we should expect good quality control measures at conferences, especially if they claim to be peer-reviewed. Here we see an indication that it's not really peer-reviewed, at least not to any reasonable degree. If you pay money to go to a conference proceeding, you hope you're getting good science.

But it depends on the TYPE of conference. There are many types! The APS March Meetings is open to all. You can submit your talk to the appropriate session, and more often than not, unless it is total crackpottery, you get to present your talk as a contributed talk. But there are also INVITED talk in which the organizers invited you to present your work. So now, figure out which one of these two you would put more emphasis on as far as validity/credibility is concerned?

There are many conferences and workshops in which the talks are only based on publications and work that have already appeared in publications, while the "contributed" part are out of the poster session. All of them may appear in the conference proceedings. So the quality will be uneven.

I find this to be rather ironic. After having to defend complaints from people that science/physics is too selective, it is excluding others from participating, now we find ourselves in an area where that barrier has been lowered quite a bit to allow for a more open participation, and we STILL get criticized for it! It appears that it is a no-win situation with whatever we do!

Zz.

 

[Pythagorean]

Thanks for the perspective, ZapperZ and DrClaude.

 

[mesa]

I think this phenomenon points out three possibilities, none of which are mutually exclusive:

1) Peer reviewers in many journals don't work to high enough standards
2) The technology for computer generated manuscripts is becoming a lot more convincing
3) There's something wrong with how we write scientific papers that makes them difficult to distinguish from crap.

After reading through these papers and speaking with Professors within our campuses math department I am going with number 3.

But it depends on the TYPE of conference. There are many types! The APS March Meetings is open to all. You can submit your talk to the appropriate session, and more often than not, unless it is total crackpottery, you get to present your talk as a contributed talk. But there are also INVITED talk in which the organizers invited you to present your work. So now, figure out which one of these two you would put more emphasis on as far as validity/credibility is concerned?

There are many conferences and workshops in which the talks are only based on publications and work that have already appeared in publications, while the "contributed" part are out of the poster session. All of them may appear in the conference proceedings. So the quality will be uneven.

I find this to be rather ironic. After having to defend complaints from people that science/physics is too selective, it is excluding others from participating, now we find ourselves in an area where that barrier has been lowered quite a bit to allow for a more open participation, and we STILL get criticized for it! It appears that it is a no-win situation with whatever we do!

Zz.

No, it is the jumbled garbage that is being presented as 'Science' that seems to be at issue. Why is it that when I show a Professor a solution to an identity that is complex and abstract they get weak in the knees but when it is simple, elegant and beautiful I get laughed out of their office?

This culture has been brooding for decades and now it is biting back, the only people to blame are those who created it and any desperately clinging to the status quo, although I suppose we shouldn't be so surprised when people behave like humans.

 

[ZapperZ]

After reading through these papers and speaking with Professors within our campuses math department I am going with number 3.



No, it is the jumbled garbage that is being presented as 'Science' that seems to be at issue. Why is it that when I show a Professor a solution to an identity that is complex and abstract they get weak in the knees but when it is simple, elegant and beautiful I get laughed out of their office?

This culture has been brooding for decades and now it is biting back, the only people to blame are those who created it and any desperately clinging to the status quo, although I suppose we shouldn't be so surprised when people behave like humans.

You have other deep-seated issues here. You have an axe to grind against an individual, than focus on that individual. You look rather foolish when you try to generalize it to large and VARIED communities of those in science.

But then, I suppose that is a human tendency as well, no?

Zz.

 

[Nugatory]

No, it is the jumbled garbage that is being presented as 'Science' that seems to be at issue. Why is it that when I show a Professor a solution to an identity that is complex and abstract they get weak in the knees but when it is simple, elegant and beautiful I get laughed out of their office?

Can you show us an example of a "simple, elegant and beautiful" solution that is rejected in favor of one that is "complex and abstract"? I'm trying to get a more concrete sense of what you mean... in part because I have found that abstraction most often ends up supporting elegance, simplicity, and beauty more than complexity.

 

[mesa]

You have other deep-seated issues here. You have an axe to grind against an individual, than focus on that individual.

I wish it were that simple.

You look rather foolish when you try to generalize it to large and VARIED communities of those in science.

But then, I suppose that is a human tendency as well, no?

Zz.

I take no issue looking foolish to fools.

 

[mesa]

Can you show us an example of a "simple, elegant and beautiful" solution that is rejected in favor of one that is "complex and abstract"? I'm trying to get a more concrete sense of what you mean... in part because I have found that abstraction most often ends up supporting elegance, simplicity, and beauty more than complexity.

I would love nothing more although the moderators have made it clear that 'personal theories' are not allowed on PF as per our guidelines. Feel free to send an email, it should be listed in my profile (moderators I am assuming there is no issue with this?)

 

[AlephZero]

"Every complex problem has a solution that is simple, elegant, and wrong". (Possibly a slight misquote of H.L. Melcken)

 

[Borek]

True, but they're still (supposed to be) peer-reviewed!

Apparently you have not heard about later MIT invention, SCIpeerev.

 

[SteamKing]

"Every complex problem has a solution that is simple, elegant, and wrong". (Possibly a slight misquote of H.L. Melcken)

H.L. Mencken, the Sage of Baltimore.

 

[DrClaude]

Apparently you have not heard about later MIT invention, SCIpeerev.

 

[AlephZero]

True, but they're still (supposed to be) peer-reviewed!

What others said, plus, conference paper reviewing is often done under severe time constraints. Unlike publishing in a journal, you can't delay the conference start date. And if every would-be presenter who submitted their paper after the official deadline was automatically excluded, the conference might be embarrassingly short.

It's fairly nonsensical when somebody gets a pack of 10 conference papers to review in a time frame of 2 days, but it happens. And of course some of the reviewing may then get unofficially "sub-contracted" as a result. But if you want to remain part of "the system" yourself, just refusing to do the impossible might not be a good game plan.