Statistically identifying cheaters at pub trivia? Could it be done?

[enkrypt0r]

Forgive my ignorance; I'm a programmer and relatively math-minded, but I haven't taken statistics since high school.

From what I understand, there are many different ways to do this for things such as standardized tests, given to thousands of students. I'm wondering if it would be feasible to have a sort of algorithm which, given information about which questions each team got right and wrong, would send up a red flag if something didn't jive.

For the sake of example, "cheating" here just constitutes using a cell phone or some other method of getting the answer. Don't worry about teams collaborating, copying off of or listening in on one another, or anything like that. We'll assume that the questions expect free-form answers, i.e. no multiple choice. Either they wrote "Waterloo" or they didn't. It's binary.

The big hurdle here (for the place I play trivia, at least) would be statistical certainty. There are generally only 6-10 teams, and each team generally has roughly five people. There are 50-60 questions in a given night. This seems like so few points to draw data from, but (as a programmer) I'm used to much larger datasets and I don't really remember how correlation coefficients work. Would it be possible to draw any meaningful conclusions (or at least hints) given such a small data set?

I would think that if a team is consistently the only one to get an answer correct, that would be a big indicator. It would, however, also suggest that the team is just better at trivia. Is there a way to sort out when their knowledge is unlikely to be unassisted?

Would this be at all doable given the parameters I've outlined, or is there just no way to tell given *only* information about which teams answers which questions correctly and incorrectly? I realize that a better way to catch cheaters is probably instinct and observation.

I realize I'm relatively ignorant in this area, so please feel free to correct my thoughts and assumptions.

Thanks!

 

[FactChecker]

I don't think that there is a statistically valid way to distinguish between skill and cheating. Unless you can think of some symptom of cheating that would show up, how could it be done?

 

[Borek]

I think you could design trap questions aimed at finding those cheating, but otherwise I don't see how.

 

[Stephen Tashi]

We'll assume that the questions expect free-form answers, i.e. no multiple choice. Either they wrote "Waterloo" or they didn't. It's binary.

The scoring may be binary, but aren't there many possible wrong answers?

You'd need to collect data from teams known not to cheat and teams known to cheat. To collect such data, you could form a "simulated" teams that answered the questions but didn't actually enter the competition. You could have some simulated teams cheat and some not cheat.

 

[FactChecker]

The better a person is, the more his answers will agree with the cheaters. The only way to distinguish between the two is to assume that there is a limit to how good a person can be. Put in some questions that no one could answer without cheating. But that is a very debatable assumption.

The only way I know to catch the cheaters is to ask the same or similar questions in person, when you know they can not cheat and see if they are really that good. That is probably not what you had in mind.

 

[disregardthat]

I agree with the general idea here. A naturally good team would answer correct on more questions, but ought also to answer correct on each question with a probability which scales accordingly to the general proportion of teams who got it right. If a team consistently deviates from this trend, it is suspicious. There are some flaws to this though, suppose it is a team of mathematicians, then it is very likely they will get the questions of mathematical content correct no matter the general difficulty of the question.

 

[haruspex]

I think you could design trap questions aimed at finding those cheating, but otherwise I don't see how.

Perhaps you could arrange that some questions are much easier than others (of otherwise similar difficulty) if one has access to the net. You could then look for a correlation on that basis. Wouldn't stand up in a court of law, though.

 

[Mogarrr]

For the sake of example, "cheating" here just constitutes using a cell phone or some other method of getting the answer.

I don't know if the pub would go to these extreme lengths, but I have a solution!

In my calculus course, a professor claimed that he would use a device to detect whether cell phones were turned on during the final exam. I am ignornant of this technology, but assuming he wasn't bluffing, such a device could be implemented.

Of course, a participant could claim to take a bathroom break, and use their phone in the bathroom. The pub could suspend a player's participation in the game, for one question, whenever they leave the cellphone-free-zone.

 

[Sheppie]

I think you could design trap questions aimed at finding those cheating, but otherwise I don't see how.

I think this is the most truthful answer. You lack data for a statistical identification. But like Borek suggested - such question, could be a very good way to find cheaters.