Amazon Turk woes: BOTS the software we love to hate?

[jedishrfu]

https://www.wired.com/story/amazon-mechanical-turk-bot-panic/

FOR THE PAST week, psychologists all over America have been freaking out.

The cause of their agita was an observation by a psychology graduate student from the University of Minnesota named Max Hui Bai. Like many researchers, Bai uses Amazon’s Mechanical Turk platform, where individuals sign up to complete simple tasks, such as taking surveys for academics or marketers, and earn a low fee. On Tuesday, August 7, he posed a simple question in a Facebook group for psychology researchers: "Have anyone used Mturk in the last few weeks and notice any quality drop?"

As he would later elaborate in a blog post, Bai had found that the surveys he conducted with MTurk were full of nonsense answers to open-ended questions and respondents with duplicate GPS locations. He said he had to throw out nearly half of the data in his most recent survey, a sharp increase from what he was used to seeing. His Facebook post garnered 181 comments, with other researchers describing similar signs of low-quality data in their own recent work. A number of them wondered if the culprit was bots—automated programs mimicking human behavior, not the actual human labor MTurk is supposed to supply.

 

[Bystander]

Really? [sarcasm]Surprise, surprise. Social scientists.[/sarcasm]