ENGLAND - 1958: English Electric developed several notable pioneering computers during the 1950s. ... [+]
When computers were still in the nascent stages, Alan Turing published his legendary paper, “Computing Machinery And Intelligence,” in the Mind journal in 1950. In it, he set forth the intriguing question: Can machines think?
At the time, the notion of Artificial Intelligence (AI) did not exist (this would not come until about six years later at a conference at Dartmouth University). Yet Turing was already thinking about the implications of this category.
In his paper, he described a framework to determine if a machine had intelligence. This essentially involved a thought experiment. Assume there are three players in a game. Two are human and the other is a computer. An evaluator—who is a human—then asks open-ended questions to the players. If this person cannot determine who is the human, then the computer is considered to be intelligent.
The Turing Test was quite ingenious because there was no need to define “intelligence,” which is fraught with complexities. Even today this concept is far from clear-cut.
Keep in mind that Turing thought the test would ultimately be cracked by 2000 or so. But interestingly enough, this turned out to be way too optimistic. The Turing Test has remained elusive for AI systems.
“If Alan Turing was alive, he might be shocked that given 175 billion neurons from GPT-3 we are still unable to pass his test, but we will soon,” said Ben Taylor, who is the Chief AI Evangelist at DataRobot.
So why has it been so difficult to beat the test? A key reason is that it can be tricked. If you ask a nonsensical question, the results will often be non-human like. Let’s face it, people are very good at detecting when something is not quite right.
“When you ask a GPT-3 system how many eyes the sun has, it will respond that there is one and when asked who was the president of the U.S. in 1600, the answer will be Queen Elizabeth I,” said Noah Giansiracusa, who is an Assistant Professor of Mathematics and Data Science at Bentley University. “The basic problem seems to be that GPT-3 always tries in earnest to answer the question, rather than refusing and pointing out the absurdity and unanswerability of a question.”
But over time, it seems reasonable that these issues will be worked out. The fact is that AI technology is continuing to progress at a staggering pace.
There may also be a need for another test as well. “Since the Turing test, humans have actually discovered much more insight into our own minds through fMRI and what makes us superior in our own intelligence,” said Taylor. “This insight into our own brains justifies changing the goals of a test beyond mimicking behavior. Defining a new test might help us get out of the deep-learning rut, which is currently insufficient for achieving AGI or Artificial General Intelligence. The Turing test was our moonshot, so let's figure out our Mars-shot.”
Over the years, other tests have emerged. According Druhin Bala, who is the CEO and co-founder of getchefnow.com, there are:
- The Marcus Test: In which a program watches a television show and is asked meaningful questions about the show's content.
- The Lovelace Test 2.0: Where you detect AI through an ability to create art.
- Total Turing Test: Where the questioner can also test perceptual abilities as well as the ability to manipulate objects.
But my favorite is the Wozniak Test (yes, this is from the co-founder of Apple). This is where a robot can enter a stranger’s home and make a cup of coffee!
Now of course, all these tests have their own issues. The fact is that no test is fool-proof. But in the coming years, there will probably be new ones and this will help with the development of AI.
“The Turing Test is brilliant in its simplicity and elegance, which is why it's held up so well for 70 years,” said Zach Mayer, who is the Vice President of Data Science at DataRobot. “It's an important milestone for machine intelligence, and GPT-3 is very close to passing it. And yet, as we pass this milestone, I think it's also clear that GPT-3 is nowhere near human-level intelligence. I think discovering another ‘Turing Test’ for AI will illuminate the next step on our journey towards understanding human intelligence.”
Tom (@ttaulli) is an advisor/board member to startups and the author of Artificial Intelligence Basics: A Non-Technical Introduction and The Robotic Process Automation Handbook: A Guide to Implementing RPA Systems. He also has developed various online courses, such as for the COBOL and Python programming languages.
