How Close Are Chatbots To Passing The Turing Test?
The question is same since the term AI was coined at a conference at Dartmouth College in 1956: Can a machine be ever as intelligent as humans?
A lot of people, including some prominent scientific minds, always felt that it is never going to happen. It was until 1997, IBM’s Deep Blue computer defeated the world chess champion Garry Kasparov in a 6 match chess series, people started thinking otherwise. Since then a series of event be it Google Deep Mind defeating the Alpha Go champion Lee Sedol or the recently CMU Libratus defeating the top Poker pros in Poker championship, proved the fact that the machines are getting smarter day by day. But are the machines smart enough to pass the Turing Test?
Turing Test, in honor of Alan Turing, the brilliant British mathematician is a test to check the presence of mind, or thought, or intelligence in a machine. In simpler words, it is a test to check whether a machine can imitate human intelligence. Or in other words, if a machine can fool a human and make the humans believe it is a human, the machine passes the test.
Suppose there is a dark room and we are outside of the room. Questions are being asked from the outside of the room to both the machine and a human, and if we cannot distinguish whether a person is answering the question or the machine is doing so, the machine will definitely pass the Turing test. It can also be thought as the machine has acquired intelligence which is similar to that of human. In other words, it is imitating human intelligence.
But, there has been some criticism to the Turing test with people claiming that it is a test to imitate human intelligence and not a way to achieve intelligence.
For instance, a human will find it difficult to calculate large arithmetic problems or calculation like 39877/139 which for a machine is pretty simple. Now for the machine to mimic human intelligence, it should not calculate the answer at once and should take time in doing so. This is not a way to achieve intelligence but to achieve human intelligence. So the assumption that human intelligence is the only form of intelligence is incorrect to assume and there might be other form of intelligence we are unaware of.
But the modern way to administrate the Turing Test is through Loebner Prize competition that started in 1991. Every year the prize is awarded to the best performing chatbots that tries to convince the judges that it has some form of intelligence in it. The winner of last year, 2016 was the bot named Mitsuku. Let’s analyze some of the Mitsuku’s answers during the event:
Judge: My name is Andrew, what’s yours?
Mitsuku: Your name is Andrew. My name is Millie.
Judge: What’s my name?
Mitsuku: Hmm. Your name is Andrew. Is it not?
From the above conversation we can analyze that Mitsuku is keeping the previous chat context in mind while answering new questions. That is similar to what humans do. But has it acquired intelligence? Let’s see another chat.
Judge: The cat tried to climb in the box but got stuck because it was too big. What was too big?
Mitsuku: Well at least they tried their best. Large, above average in size or number or quantity or magnitude or extent.[2]
Here it is clearly demonstrated that it is far from acquiring human level intelligence. It seems it is not understanding the context of the question and is trying to answer based on the words in the question. There are a number of examples showing the fact that the bots cannot understand the context every time but upon training them over a long time using day to day conversations can make them smarter.
However, a bot cannot be trained on all possible scenarios in this world which makes them vulnerable if they are based on some rules. That is where intelligence and logic building up comes into picture. The ideal chatbot must have memory, can learn over time and answer smartly or in short, must posses some sort of intelligence.
But how far is that ideal bot? The answer is not in near future. The bots defeating the world champions at very complex games like Alpha Go proves that we are going in right direction. But if the bot developed for Alpha Go is asked to play poker, it will not perform the task well. Chatbots like Mitsuku or Cleverbot are well performing in the tasks they are trained on but they fail in something that they are not trained on. Unlike humans, they cannot build logic in tackling a new problem.
According to some predictions, the bots might pass the Turing test at the end of year 2029. But, is achieving human intelligence all we are looking for or should we work towards achieving true intelligence?
Read my attempt in creating a chatbot for Loebner Prize that tries to generate it’s own answers when trained over 100 GB of chat data here.
References:
[1] http://sitn.hms.harvard.edu/flash/2012/ai/
[2] http://www.aisb.org.uk/events/loebner-prize#finals2016
[3] Two chatbots hold a conversation in the Cornell Creative Machines Lab. (Cornell University/YouTube)
We’re a team of bot creatives and AI scientists with one common goal:
blowing business objectives out of the water with bots and cognitive solutions
