Can a Nuclear Physicist Outsmart Watson on Jeopardy?

  • Thread starter Thread starter BobG
  • Start date Start date
Join the discussion
Registration is free. Ask a follow-up in this thread, or start your own.
Sign up Log in
26 replies · 6K views

Discussion

Physics news on Phys.org
There's a site that allows you to play against a simulated Watson. A machine of comparable answering prowess with significantly less computing power. So, instead of quick 3-second bouts, you are allowed to take you time, and both you and Watson can win together (if you both answer properly).

I smoked Watson.

Perhaps timing will be the deciding factor in this battle of awesomeness.

EDIT: http://www.nytimes.com/interactive/2010/06/16/magazine/watson-trivia-game.html

DOUBLE EDIT:
beatwatson.png
 
Last edited:
FlexGunship said:
There's a site that allows you to play against a simulated Watson. A machine of comparable answering prowess with significantly less computing power. So, instead of quick 3-second bouts, you are allowed to take you time, and both you and Watson can win together (if you both answer properly).

I smoked Watson.

Perhaps timing will be the deciding factor in this battle of awesomeness.

EDIT: http://www.nytimes.com/interactive/2010/06/16/magazine/watson-trivia-game.html

Timing would be a serious cheat if that's the advantage, although you're almost certainly brighter than the average Jeapordy contestant.
 
nismaratwork said:
Timing would be a serious cheat if that's the advantage, although you're almost certainly brighter than the average Jeapordy contestant.

Very late edit: Having seen round 1, it seems that buzzer timing needs to be eliminated from this to be meaningful. Watson gets this before the question is done, and it's... A machine! I'm dissapointed that this is pure gimmick from IBM, especially given the history of these kinds of challenges.
 
nismaratwork said:
Very late edit: Having seen round 1, it seems that buzzer timing needs to be eliminated from this to be meaningful. Watson gets this before the question is done, and it's... A machine! I'm dissapointed that this is pure gimmick from IBM, especially given the history of these kinds of challenges.

Didn't seem that unfair to me. Besides, you can't buzz in before the question ends. IN fact, a human being is responsible for starting the "buzz-in" timer after the question is fully read.
 
FlexGunship said:
Didn't seem that unfair to me. Besides, you can't buzz in before the question ends. IN fact, a human being is responsible for starting the "buzz-in" timer after the question is fully read.

Is that true? I thought that the window opened when the question was exposed. My impression was that the cut to the contestants pressing the button after the question is finished was time-delayed to make it look like they were clicking after the question was done...

(EDIT -- I mean in general on Jeopardy, not just for this Watson match)
 
berkeman said:
Is that true? I thought that the window opened when the question was exposed. My impression was that the cut to the contestants pressing the button after the question is finished was time-delayed to make it look like they were clicking after the question was done...

(EDIT -- I mean in general on Jeopardy, not just for this Watson match)

Well, I can't seem to re-find my source, but yes. It's a producer. So it is kind of a judgement call when the question is ready to be answered.
 
FlexGunship said:
Well, I can't seem to re-find my source, but yes. It's a producer. So it is kind of a judgement call when the question is ready to be answered.

Hmm, looks like you are right. I didn't realize that's how it worked. Found this at wikipedia:

http://en.wikipedia.org/wiki/Jeopardy!
Ringing in
Contestants must wait until the host finishes reading the clue before ringing in. Ringing in before this point locks the contestant out for one fourth of a second.[10] Lights mounted around the game board illuminate to indicate when contestants may ring in, and the contestant has five seconds to offer a response. Additionally, a tone sounds in conjunction with the illuminated lights on episodes that feature visually-impaired contestants.

Before Trebek's second season, contestants were able to ring in at any time after the clue had been revealed, and a buzzer would sound whenever someone rang in...
 
nismaratwork said:
Very late edit: Having seen round 1, it seems that buzzer timing needs to be eliminated from this to be meaningful. Watson gets this before the question is done, and it's... A machine! I'm dissapointed that this is pure gimmick from IBM, especially given the history of these kinds of challenges.

Yes this is a gimmick from IBM showing off their advanced computer programing skills. However unlike Deep Blue, Watson has great potential after the contest that it was designed for. Imagine an expert system in every hospital, ( and when the processing power gets sized down) and ambulance that is up to date on all medical studies, and treatment options. Or a research assistant that can read through millions of GBs of data and information and give you a reasonable and accurate amount of data. Or any other situation where an expert system would be useful.
 
Well, I didn't mean that Watson had the jump, but it has no hesitation either. If you match the reactions of Watson vs. a human, I'd still expect Watson to buzz in before the human more often than not. When the window is a matter of tenths of a second, even if the intiater is also human, just from my experience with bots playing games and other things... the computer often prevails.
 
Argentum Vulpes said:
Yes this is a gimmick from IBM showing off their advanced computer programing skills. However unlike Deep Blue, Watson has great potential after the contest that it was designed for. Imagine an expert system in every hospital, ( and when the processing power gets sized down) and ambulance that is up to date on all medical studies, and treatment options. Or a research assistant that can read through millions of GBs of data and information and give you a reasonable and accurate amount of data. Or any other situation where an expert system would be useful.

I like that, but centralize it, expand it, and then sell access to it through your phones and other gadgets.
 
nismaratwork said:
I like that, but centralize it, expand it, and then sell access to it through your phones and other gadgets.

[PLAIN]http://www.pamil-visions.net/wp-content/uploads/2009/04/skynet.jpg
 
Last edited by a moderator:
FlexGunship said:
[PLAIN]http://www.pamil-visions.net/wp-content/uploads/2009/04/skynet.jpg[/QUOTE]

True... so let's not give it access to our launch codes? :wink:
 
Last edited by a moderator:
berkeman said:
Hmm, looks like you are right. I didn't realize that's how it worked. Found this at wikipedia:

http://en.wikipedia.org/wiki/Jeopardy!
Ringing in
Contestants must wait until the host finishes reading the clue before ringing in. Ringing in before this point locks the contestant out for one fourth of a second.[10] Lights mounted around the game board illuminate to indicate when contestants may ring in, and the contestant has five seconds to offer a response. Additionally, a tone sounds in conjunction with the illuminated lights on episodes that feature visually-impaired contestants.

Before Trebek's second season, contestants were able to ring in at any time after the clue had been revealed, and a buzzer would sound whenever someone rang in...

I think this modification adds to the game on TV, but the Commodore 64 version without the delay was much more fun than the later computer versions of the game.

There was a huge advantage to controlling the board, since you knew whether you were going to select an easy question or a hard question and the other competitors didn't. You also knew when you were going to select a question, so the other competitors had to be really fast and focused on beating you in spite of the natural advantage you had.

You'd select an easy question and immediately "buzz in" knowing only that the question would be easy. Then, with your competitors intent on beating you to "buzz in", you'd suddenly pick a hard question in an obscure category and watch the expression on their faces when they "buzzed in" and then read the question they had to answer.

Combined with the fact that we usually played this while drinking beer, our games inevitably degenerated into "Full Contact Jeopardy". Fortunately, the Commodore 64 was very resistant to spilled beer.

Watson may be pretty clever, but I think we would have kicked his butt once the game entered the full contact phase.
 
BobG said:
I think this modification adds to the game on TV, but the Commodore 64 version without the delay was much more fun than the later computer versions of the game.

There was a huge advantage to controlling the board, since you knew whether you were going to select an easy question or a hard question and the other competitors didn't. You also knew when you were going to select a question, so the other competitors had to be really fast and focused on beating you in spite of the natural advantage you had.

You'd select an easy question and immediately "buzz in" knowing only that the question would be easy. Then, with your competitors intent on beating you to "buzz in", you'd suddenly pick a hard question in an obscure category and watch the expression on their faces when they "buzzed in" and then read the question they had to answer.

Combined with the fact that we usually played this while drinking beer, our games inevitably degenerated into "Full Contact Jeopardy". Fortunately, the Commodore 64 was very resistant to spilled beer.

Watson may be pretty clever, but I think we would have kicked his butt once the game entered the full contact phase.

bolding mine: Sounds like the process that ended most games of Monopoly I've played... has that game EVER ended with smiles and hugs?

Oh, and as for butt-kicking on the physical plane... um... see Flex's pic. I don't think they even have butts... :bugeye:
 
i find watson a little creepy.
 
Proton Soup said:
i find watson a little creepy.

Funny. He thinks the same thing about you.
 
http://venturebeat.com/2011/02/15/ibm-watson-jeopardy-2/

I didn't get to watch the game, I don't have television. But I love this part:
In an early Daily Double question, Watson wagered an oddly precise $6,435, which drew laughs from the crowd.

That's so typical of how I would expect a computer to operate. I guess it'll make it easy to spot the Cylons.
 
I don't know if the buzzer thing is in Watson's favour. Humans tend to buzz and then thinkk of the answer - we know we know it, so we answer - Watson (I gather at last) buzzes when it has an answer - so no recall time (5 seconds is a long time for a machine such as this - just look how fast Google is at sifting through indexed information).

I would like to see the full video of the day - including the silly answers given.

PS: I think they should make the voice a little more HAL9000 like - freak out the other contestants "What are you doing Dave?"
 
FlexGunship said:
http://venturebeat.com/2011/02/15/ibm-watson-jeopardy-2/

I didn't get to watch the game, I don't have television. But I love this part:


That's so typical of how I would expect a computer to operate. I guess it'll make it easy to spot the Cylons.

If you ever play poker online, its easy to spot the cheats using a poker-bot to play- they bet silly amounts like REaise "$217.48" - so easy to beat (just do what they expect you to do when you have them beat! and leave before they "learn" what you are doing)
 
I have watched the last two nights. I don't typically watch Jeopardy because of the timing and my kids, but I think this is a very interesting problem that IBM is working on.

I was very eager to see a couple of things about Watson. First, where are his blind spots. Since Jeopardy is a category based game, I was interested to see what categories Watson had trouble with. Some that I noticed were: decades and literature.

The other thing I was eager to see was how Watson handled Final Jeopardy. These questions tend to require more natural language inference. He failed miserably I thought. But it is just one question so far. Also his betting values for double and final Jeopardy were interesting. I would be interested to read on what type of algorithm he used for that.

Also, it was interesting to see some of the strategies he used. Typical Jeopardy players search for the daily double to start. Watson also did this. But after they are all gone, human players tend to work through a category. I think it helps constrain the possible questions and allows us to focus more. Watson did not. He worked by question value. Finish all the 100 point questions. Then do the 200 point questions, then 300, etc. I wonder how much that messed with the human players.
 
Wolf5370 said:
If you ever play poker online, its easy to spot the cheats using a poker-bot to play- they bet silly amounts like REaise "$217.48" - so easy to beat (just do what they expect you to do when you have them beat! and leave before they "learn" what you are doing)

I'm not sure I'd say using a computerized poker player in an on-line game was as much cheating as an interesting experiment. In many ways, poker is an even more interesting challenge than Jeopardy.

To play Jeopardy, the challenge is first to decipher the language in order to understand the question (and this is a huge challenge) and then to search on the words the computer 'felt' were important. The lower percentage nonselected answers are almost more interesting than the answer the computer selected.

Designing a computer program that can win at poker is an even more interesting challenge. The odds part for each hand is a breeze and can probably work in an on-line game against average players. Upping the level to playing against professional players would require a lot more. The program would have to start interpreting human behavior (based on time to bet, patterns in amounts, etc?). Definitely a different problem than interpreting human language.
 
BobG said:
I'm not sure I'd say using a computerized poker player in an on-line game was as much cheating as an interesting experiment. In many ways, poker is an even more interesting challenge than Jeopardy.

To play Jeopardy, the challenge is first to decipher the language in order to understand the question (and this is a huge challenge) and then to search on the words the computer 'felt' were important. The lower percentage nonselected answers are almost more interesting than the answer the computer selected.

Designing a computer program that can win at poker is an even more interesting challenge. The odds part for each hand is a breeze and can probably work in an on-line game against average players. Upping the level to playing against professional players would require a lot more. The program would have to start interpreting human behavior (based on time to bet, patterns in amounts, etc?). Definitely a different problem than interpreting human language.

It's interesting... who forgot to include "Detergent" in Watson's programming? :-p

I agree with you, but I know what Wolf is saying. Still, I remember when bots first arrived in PC games... specifically a variant of 'Quake' which gave people agita for a long time. I thought much as Wolf did... basically, given time you recognize machine-thinking and actions. I don't care how clever someone is when they script, it may be maddening, but at that time 'Lag' was a much bigger issue.

They haven't gone away, and in fact, they've become a plague in far more than just their original realms. Consider this... once you teach a computer to bluff, is it possible for a human to "read" the computer? No, never, unless that's a built-in fault. Can the computer read PEOPLE?... uh, yeah... add a decent thermal camera to the regular and suddenly you're REALLY in trouble.

That's what the problem really is with all of this from the "player's" view; a computer can be easy to spot, and they inevitably have blind-sides and faults. A human can do a lot to correct those faults, but a machine can be utterly specialized. You could conceivably make a poker computer with a face (on a screen) that actively BLUFFS... ouch.

So... clearly Watson is amazing to do what it did, period. As a contest however, it's just not meaningful to me given the information provided, and as a demonstration of language it's... impressive... BUT...

... It's still not good enough in a realm that needs to be at LEAST as good as a human. Given that, I'd like to see the buzzer times: how often did a contestant and Watson come down to a button press, not an issue of knowledge?
 
ZapperZ said:
US Representative Rush Holt, who happens to be a nuclear physicist as well, beat Watson last night.

http://blogs.wsj.com/washwire/2011/02/28/rep-holt-bests-watson/
http://washingtonscene.thehill.com/in-the-know/36-news/8433--rep-holt-beats-watson-the-computer-at-jeopardy

Zz.

:smile:

So maybe Jeopardy just needs a higher class of player... good to know the science at the core of Watson can't beat the scientist.
 
Last edited by a moderator: