This contest proved how far behind the times chatbots really are
The chatbots you’re hitting up on Facebook or Kik may seem as though they can really understand you, but in all actuality they’re pretty stupid, at least when it comes to common sense. That’s why a new Turing Test in the form of the Winograd Schema Challenge seeks to prove how lacking the bots really are in the grand scheme of things.
The challenge asks computers to make sense out of specific sentences with grammar that humans can understand, but that may be obtuse to machines. For instance, in the sentence “The city councilmen refused the demonstrators a permit because they feared violence,” computers aren’t able to parse who the word “they” is actually talking about. In contrast, human readers can understand it because of context clues. That’s exactly the type of thinking researchers are looking to improve, namely with deep learning.
The contest featured a grand prize of $25,000 for entrants who could achive 90 percent accuracy with similar sentences, and the best came from Quan Liu, a researcher from the University of Science and Technology of China as well as Nicos Issak, a researcher from the Open University of Cypress.
Facebook and Google, who have long since been championing the improvement of the bots’ understanding of complex human speech were not present at the contest, but they also have utilized “deep learning” neural networks to improve their chatbots and personal assistantes. Quan Liu’s group, from the challenge, did as well.
The contest came as a result of an improvement on the original Turing Test, which is a way for humans to evaluate a machine’s intelligence. Still, even with the accuracy the teams were able to achieve during the conference, machines clearly have a long way to go when it comes to matching human accuracy.
(12)
