Artificial stupidity, Part 2

Can chatterbots be as dumb as a box of hammers and still pass the Turing test? Go ask ALICE, she might know.

Editor's note: Part 2 of two parts. Read Part 1.

By John Sundman

Feb. 27, 2003 | If Hugh Loebner's contest is just hokum, and the Turing test has outlived its usefulness, why should we care about it or its various squabbling participants?

A vocal camp in the brainy "philosophy of mind" profession believes that the Turing test should be relegated to the history books, but I'm going to assert axiomatically that the test, as it is generally understood by ordinary humans like you and me, is interesting. The question of whether computers can successfully pose as human beings has obsessed writers, filmmakers and computer scientists for decades. Therefore, without getting sucked into a philosophical vortex about the nature of minds, machines, intelligence and so forth, all we need to find out -- if we want to know if the Loebner competition matters -- is whether there exists a more respectable variant of the Turing test. As far as I can determine, there doesn't. The Turing test is, as it were, state-of-the-art.

But instead of buckling down to meet the challenge that Loebner poses, the artificial intelligence community has made a consistent effort to change the rules -- to do away, even, with the very name of their own discipline.

Neil Bishop, the organizer of the 2002 Loebner competition, summed it up as follows:

"In the professional and academic circles the term Artificial Intelligence is passé. It is considered to be technically incorrect relative to the present day technology and the term has also picked up a strong Sci-Fi connotation. The new and improved term is Intelligent Systems. Under this general term there are two distinct categories: Decision Sciences (DS) and the human mimicry side called Mimetics Sciences (MS)."

Decision sciences, by the simplest possible definition, refers to computerized assistance in resource allocation. An example provided by a press release from MIT announcing the creation of a decision sciences program was "complex computer-based 'passenger yield management' systems and models that the airlines use to adjust pricing of each flight's seats in order to maximize revenue and profitability to the airline."

That's a far cry from the bold claims made by A.I. visionaries in decades past. But focusing on such systems has a signal advantage for scientists who have been failing miserably at the Turing test. It gets them off the hook. As James H. Moor, of Dartmouth College's department of philosophy and the organizer of the 2000 competition, wrote:

"The Turing test is not very useful for many A.I. scientists today because they work on projects that have nothing to do with human linguistic performance."

But Moor did concede that Alan Turing's challenge is still worth chasing: "Nevertheless, the Turing test will remain a philosophically interesting test and a long range challenge for A.I. If a computer could routinely converse with us as well as Deep Blue could play chess and we had no reason to believe some kind of trickery was involved, how could we deny it had at least some intelligence?"

Even as recently as November 2002, the influential IBM Systems Journal featured a technical forum on "Machine Intelligence and the Turing Test"; but its only mention of Loebner was in a footnote:

"A formal [Turing test] yearly contest, sponsored by Hugh Loebner and The Cambridge Center for Behavioral Studies, accords a $2000 prize and medal to the most human-like computer contestant. Among the most well-known critics of the contest is Marvin Minsky ... Minsky has wittily sponsored a "Minsky Loebner Prize Revocation Prize."

I don't know about you, but I find this sycophancy embarrassing. As to the antipathy to the Loebner competition from the A.I. establishment, Neil Bishop confirmed my impressions:

"The hard-core DS types like Professor Minsky firmly believe that the 'Holy Grail' (cognitive understanding and response) can only be realized through the DS approach. As a result they have very little respect for the mimetics (human mimicry) side of the equation. Also just by its nature the MS side embraces the general public's view of the old A.I. term. After all if you can talk to an artificial person and it responds in a human-like manner, who cares if it is actually 'thinking' or just doing a damn good job of fooling you? And if you look at Turing's original concept that is really all that is needed to win. As a result, the DS camp seems to think that mimetics are undermining their image. Particularly since there have been many bold projects and claims in the DS camp which have failed. This is where the rift spawns. Then you take the apparently fragile personalities of key players in both camps, well, to put it bluntly you end up with a childish display of emotions at the least, and at times a real 'bitch fight' will get started."

In other words, if you read between the lines what you come up with is that one reason "serious" A.I. scientists don't try to mimic human speech anymore is that they discovered they can't do it. Of course, they promised 30 years ago that they would be able to do so "real soon now," but it has turned out to be harder than expected, so now it's dismissed as mere mimicry.

Next page: What respectable A.I. scientist wanted to lose to some cheezo hobbyist?