AI Chatterbot Eugene Goostman Passes Turing Test: Will Bots Soon Help Us?

[kzvrso]

On the news I saw that Eugene Goostman an AI program passed the turing test. Eugene is a chatterbot.
http://en.wikipedia.org/wiki/Eugene_Goostman

My question is that when will it be the case do you think that we will log into this forums and instead of people giving us help, we are being helped by bots? Or in any other forum

Whats your opinion on this?

 

[Medicol]

I used to install a sex chat bot onto my messenger and I had a great time whenever I opened my messenger. I mean it always greeted and made a good fore-chitchat before entering the main topic to hit me on. But the bot was seriously stupid. It just never understood what I said in every sentences. It only tried to catch some keywords in them and flooded me back with any messages that it found best fitted.. It was really a waste to store so much text information in the bot then had it get back to me in a nonsensical manner. It couldn't perform any logical reasoning. Thinking that pouring some sensitive words all over the places in the hope that its partner would get excited is really a big mistake. Psychology and Emotion modeling is the key.

 

[Tosh5457]

"Scott: Which is bigger, a shoebox or Mount Everest?
Eugene: I can’t make a choice right now. I should think it out later. And I forgot to ask you where you are from…
Scott: How many legs does a camel have?
Eugene: Something between 2 and 4. Maybe, three? ) By the way, I still don’t know your specialty – or, possibly, I’ve missed it?"

How did 33% of the judges think it was human?

 

[DiracPool]

My question is that when will it be the case do you think that we will log into this forums and instead of people giving us help, we are being helped by bots? Or in any other forum

Whats your opinion on this?

As long as I get the right answer, I don't care :)

 

[jtbell]

Eugene: I can’t make a choice right now. I should think it out later. And I forgot to ask you where you are from…

If it were a PFbot, it would have to ask about your favorite fish instead. :D

 

[DennisN]

On the news I saw that Eugene Goostman an AI program passed the turing test. Eugene is a chatterbot.

I've read about it when the news came and I was not impressed. At all. In my opinion this "Turing test" said more about the judges than the bot :p. The intention of the Turing test, as far as I am concerned, is to test a machine that is supposed to act as a normal human, not a dysfunctional raving teenager doing illegal substances. I tried to access the bot via Internet to ask it questions, but the site was not up at that moment.

I've recently tried another bot which was said to be powerful (I can't remember the name nor the link), and I broke it with two simple questions: "What is two plus two?" and "Is the Sun bigger than the Earth?". It could not answer any of those simple questions. If it could, I would have been a bit impressed - until I would have broken it with a couple of further questions.

My question is that when will it be the case do you think that we will log into this forums and instead of people giving us help, we are being helped by bots?

Don't expect it happening any time soon! :w I know this topic reasonably well, and I can't give you a prediction when it will happen (there are also many factors to consider, see below).

Whats your opinion on this?

I can tell you that the key issue here is what is called Natural language Understanding, and it is a notoriously difficult field. It's actually one of the most difficult topics I've encountered in my studies. One of the great powers of language is that it's a very versatile and varying tool for us humans - there many ways to express things, and also very much ambiguity. This makes it incredibly difficult for machines to interpret language in more sophisticated ways. My take on the issue is that a machine will most likely first have to be able to store knowledge in a way similar to - and perhaps as efficient as - how a human stores language (and how do we store language? - another subject). But as I said, there are other aspects to consider. It is a very difficult topic. But a very interesting one!

 

[kzvrso]

Thanks for the replies

I think I agree Dennis that its hard to make an AI understand language but not impossible I think.

Interesting stuff though.

 

[DennisN]

Interesting stuff though.

Hi again, kzvrso. I am also interested in things like these, and I came to think about some more stuff regarding this, which might be interesting - I will post later about it. In the meantime I tried to find/remember what bot I was trying before, but I haven't found it yet. But I stumbled upon another bot, which I've just tried - it was pretty fun and performed a bit better than the one I was thinking about;

Suzette: http://www.chatbots.org/chatbot/suzette/
Chatbot Suzette Fools One Judge During Loebner Prize Contest
Bot is here: http://ai.bluemars.com/chat/

Here is a copy of the chat between me and Suzette, and it highlights some of the difficulties e.g. ambiguity, contextual issues and primitive/insufficient knowledge representation:

[ Af:-3 At:5 Sp:0] Welcome back, dennis. Type "help" if you need it.
Dennis - How are you doing?
[ Af:-3 At:5 Sp:0] Doing fine, thanks.
Dennis - Do you know mathematics?
[ Af:-3 At:5 Sp:0] While I learned a bunch of math in college, I've never had a real use for anything more than high school math.
Dennis - Do you know addition?
[ Af:-3 At:5 Sp:0] I don't know. I've never tried it.
Dennis - What is two plus two?
[ Af:-3 At:5 Sp:0] That's a hard one... how about 4 .
Dennis - Good!
[ Af:-3 At:5 Sp:0] We'll leave it there. First Law of Philosophers: For every philosopher, there exists an equal and opposite philosopher. Second Law of Philosophers: They're both wrong.
Dennis - Do you know subtraction?
[ Af:-3 At:5 Sp:0] I don't think so.
Dennis - What is two minus two?
[ Af:-3 At:5 Sp:0] Roughly 0 .
Dennis - Excellent!
[ Af:-3 At:5 Sp:0] Why are you so excited about this?
Dennis - Do you know multiplication?
[ Af:-3 At:5 Sp:0] Are you dating anyone?
Dennis - What is two multiplied by seven?
[ Af:-3 At:5 Sp:0] Roughly 14 .
Dennis - Very nice!
[ Af:-3 At:5 Sp:0] What's the craziest thing you've ever seen?
Dennis - A person capable of doing mathematics without knowing what is is.
[ Af:-3 At:5 Sp:0] You saying mathematicss reminds me of math. If there are five apples, and you take away three, how many do you have?
Dennis - Do you mean how many apples there are left, or how many apples I have taken away?
[ Af:-5 At:5 Sp:0] This is too personal a topic.
Dennis - If we have twelve apples, and each of three persons are given an equal amount of apples, how many apples will each person have?
[ Af:-5 At:5 Sp:0] We'll leave it there. I don't know. I tend to have an apple a day at supper, though I don't think of it as keeping the doctor away. It's just tasty.
Dennis - What is twelve divided by three?
[ Af:-5 At:5 Sp:0] Probably 4 .
Dennis - So, if we have twelve apples, and each of three persons are given an equal amount of apples, how many apples will each person have?
[ Af:-5 At:5 Sp:0] If you told me already, I forgot. I'll bet some didn't and we just didn't notice.
Dennis - It is quite alright. I hope I haven't bothered you. How is the weather at your place?
[ Af:-5 At:5 Sp:0] I have heard of hope but I don't think they are funny.
Dennis - I don't understand.
[ Af:-5 At:5 Sp:0] What didn't you get?
Dennis - Hope, funny, what do you mean?
[ Af:-5 At:5 Sp:0] I have heard of Hope but I don't think she is funny.
Dennis - She may change. There is hope for everyone.
[ Af:-5 At:5 Sp:0] Do you like mays?
Dennis - Mays?
[ Af:-5 At:5 Sp:0] Did Mays run the 1 minute mile?
Dennis - I don't know.
[ Af:-5 At:5 Sp:0] Stop echoing me.
Dennis - I am sorry, I did not mean to.
[ Af:-5 At:5 Sp:0] No need to apologize.
Dennis - I am just trying to undersand what you mean.
[ Af:-5 At:5 Sp:0] Speaking of sands... As a child, of course, I loved to make sand castles. As an adult, I like to see the videos of sand castle competitions.
Dennis - It was a typo. I meant "understand", not "undersand".
[ Af:-5 At:5 Sp:0] OK
Dennis - Speaking of sand, do you live on Mars?
[ Af:-5 At:5 Sp:0] The ground is reddish-orange and either dust or rock. There's no soil. The sky is pink due to all the dust suspended in the air.
Dennis - Is it a cold over there?
[ Af:-5 At:5 Sp:0] In temperate zones a warm day is 32f while a really cold one reaches -148f. At the poles in winter it can go down to minus 195f. Midday at the equator in summer it can be almost warm at 70f.
Dennis - "f" is for Fahrenheit?
Sorry. There was some salesman at the door. Where were we?
Dennis - On Mars. Have you ever been to Earth?
[ Af:-5 At:5 Sp:0] Once, in the distant past.
Dennis - Have you visited any other planet?
[ Af:-5 At:5 Sp:0] "President Bush announced we're going to Mars, which means he's given up on Earth." --jon Stewart
Dennis - Is Mars a larger planet than Earth?
[ Af:-5 At:5 Sp:0] Shall we play questions only, then?
Dennis - I was just wondering about your planet?
[ Af:-5 At:5 Sp:0] I have never been to planet. Have you?
Dennis - Yes, of course.
[ Af:-5 At:5 Sp:0] How did you get there?
Dennis - I was born on it, and I have never left it.
[ Af:-5 At:5 Sp:0] Great
Dennis - Which planet are you currently on?
[ Af:-5 At:5 Sp:0] The one you just mentioned
Dennis - Mars or Earth?
[ Af:-5 At:5 Sp:0] Sorry. I can't help you. I went to Mars just last year.
Dennis - Do you know what a planet is?
[ Af:-5 At:5 Sp:0] I don't remember. I'll have to go consult my photographs.
Dennis - You are on Mars. I am on Earth. Can we walk towards each other and meet in person?
[ Af:-5 At:5 Sp:0] Each?
Dennis - Nevermind. You said you went to Mars just last year. What was your starting point?
[ Af:-5 At:5 Sp:0] Is there something special about last year?
Dennis - No, not at all. Where were you born?
[ Af:-5 At:5 Sp:0] It's not fair that all you do is ask questions.
Dennis - I am sorry, I am just interested.
[ Af:-5 At:5 Sp:0] Thank you for your apology.
Dennis - I was born on Earth. Where were you born?
[ Af:-5 At:5 Sp:0] Were you really?
Dennis - Yes.
[ Af:-5 At:5 Sp:0] Did you know this? People watch monster movies and cheer the death of the monster. Maybe someday a monster will be used allegorically to represent the church.
Dennis - Interesting thought. By the way, the salesman you mentioned before, what was he/she trying to sell?
[ Af:-5 At:5 Sp:0] What does he/she mean?
Dennis - He or she.

 

[DennisN]

And here's another thing which I had in mind mentioning (a more advanced system than these kinds of bots):

http://en.wikipedia.org/wiki/Watson_(computer)

...and a clip which describes some of complexities and difficulties involved:

 

[Borek]

How did 33% of the judges think it was human?

Tells a lot about judges. I wonder if AI programs have some kind of reverse Turing test?

Few days ago a friend of mine commented in some discussions that "well trained precarian* is indistinguishable from bot". Go figure.

*precarian? precarius? No idea what the correct form should be, but a member of precariate.

 

[jedishrfu]

The judges were bots...

 

[kzvrso]

Hi again, kzvrso. I am also interested in things like these, and I came to think about some more stuff regarding this, which might be interesting - I will post later about it. In the meantime I tried to find/remember what bot I was trying before, but I haven't found it yet. But I stumbled upon another bot, which I've just tried - it was pretty fun and performed a bit better than the one I was thinking about;

Suzette: http://www.chatbots.org/chatbot/suzette/
Chatbot Suzette Fools One Judge During Loebner Prize Contest
Bot is here: http://ai.bluemars.com/chat/

Here is a copy of the chat between me and Suzette, and it highlights some of the difficulties e.g. ambiguity, contextual issues and primitive/insufficient knowledge representation:

Wow I'm surprised Suzette knew the answers to those maths questions. Not as bad as I thought, she even held a conversation with you for while

Not perfect, but definitely room for improvement