Replacing the Turing Test 129
mikejuk writes A plan is afoot to replace the Turing test as a measure of a computer's ability to think. The idea is for an annual or bi-annual Turing Championship consisting of three to five different challenging tasks. A recent workshop at the 2015 AAAI Conference of Artificial Intelligence was chaired by Gary Marcus, a professor of psychology at New York University. His opinion is that the Turing Test had reached its expiry date and has become "an exercise in deception and evasion." Marcus points out: the real value of the Turing Test comes from the sense of competition it sparks amongst programmers and engineers which has motivated the new initiative for a multi-task competition. The one of the tasks is based on Winograd Schemas. This requires participants to grasp the meaning of sentences that are easy for humans to understand through their knowledge of the world. One simple example is: "The trophy would not fit in the brown suitcase because it was too big. What was too big?" Another suggestion is for the program to answer questions about a TV program: No existing program — not Watson, not Goostman, not Siri — can currently come close to doing what any bright, real teenager can do: watch an episode of "The Simpsons," and tell us when to laugh. Another is called the "Ikea" challenge and asks for robots to co-operate with humans to build flat-pack furniture. This involves interpreting written instructions, choosing the right piece, and holding it in just the right position for a human teammate. This at least is a useful skill that might encourage us to welcome machines into our homes.
Alan Turing did not test robots. (Score:1)
The Turing test was a CONCEPT, not an actual test.
Re: (Score:2)
Small change... (Score:2)
"The trophy would not fit in the brown suitcase because it was too big. What was too big?"
If you change this to "The trophy would not fit in the brown suitcase snugly because it was too big" I wouldn't be able to answer it, either.
Re: (Score:2)
Why would we need to replace the Turing test, if it's perfectly possible to ask these types of questions as part of the Turing test.
Re: (Score:2)
The most obvious replacement for the Turing test is the politician test. How 'smart' does a computer need to be to pretend to be smart when speaking on behalf of others. So receiving 'er' contributor input and producing a speech that sounds really good based upon those 'er' contributions, which doesn't mean much and surreptitiously hides the real intent of the 'er' contributors in the speech behind sounds good messages. Sort of a Bush vs Obama measurement scale, both doing much the same thing but one sound
Why a human in the IKEA challenge? (Score:2)
I like the idea of the IKEA challenge but why include a human? I would think having a robot
open a box, pull out the instructions, and assemble the piece of furniture would be huge.
Having a person involved just muddles the issue. You obviously might have to start with
simple furniture but this seems like a worthwhile challenge as assembling furniture at
times can even stump humans.
Re: (Score:2)
If you can afford a robot that could do this, your furniture is not going to be cheap junk that is assembled by the customer.
Maybe right now, but once the problem is solved, making it cheaper would probably be fairly
straightforward. The point isn't for it to be economical but that it's a task that is fairly simple
for a human but impossible for a computer. Of course, we have a long way to go as
demonstrated by this video sped up 50 times: https://www.youtube.com/watch?... [youtube.com]
Re: (Score:2)
To be fair the test has to be something possible for a human, also
Re: (Score:2)
Frankly, the "IKEA test" identifies crea
Re: (Score:3)
The extra human is to see whether the computer will end up in a emotional argument with the human about the best way to interpret the instructions.
Re: (Score:2)
Re: (Score:2)
Calculon is an actor, not a businessrobot.
It's cheaper to include extra screws than to pay customer service to deal with the the complaints of missing parts, or to cover the extra cost of a more thorough inspection process. The easiest way to tell if a unit was packed properly is to weigh it precisely, and only ship the units that weigh the correct amount. A missing screw may fall within the error margin of the scale, so by throwing in an extra screw or three, the risk of actually being short a part is grea
Re: (Score:2)
Presumably to immunize the experiment against shortcomings in the mechanics.
Just a waste of time (Score:1)
When it's funny? (Score:2)
Really -- someone suggests a computer program could identify when to laugh at a sitcom? When humans are likely to disagree rather strongly about which parts are the funniest? Heck, even Mycroft's first jokes were on the weak side of humor. It took a lot of coaching from the humans to get (his) jokes classy.
Re: (Score:2)
It would be a trivial program to listen for the laugh track.
Re:When it's funny? (Score:5, Funny)
The most common underlying basis of humor is subverted expectations. We expect people to behave according to the norms of society, we expect people to act to the best of their intelligence, we expect misfortune to be avoided, and we expect that words will be used according to their common meanings.
Subvert any of those expectations, and you have various kinds of humor. How funny a particular joke is perceived to be is related to how strongly the viewer is attached to their expectations. Since a computer is only an expert in the things they've been explicitly exposed to, it's very difficult to subvert their expectations. Watson would be familiar with all of the meanings of each word in a script, for example, so it would have a difficult time identifying the usual meaning that a human would expect from a situation, and would therefore likely fail to notice that when a different meaning was used, it was an attempt at humor.
As another example, consider a military comedy, like Good Morning, Vietnam. Much of the humor is derived from Robin Williams' fast-paced ad-lib radio show contrasting the rigid military structure, and the inversion where a superior at the radio station is practically inferior in every way. A computer, properly educated in the norms of military behavior, might recognize that the characters' behaviors are contrary to expectations, but then to really understand the jokes, the computer must also have an encyclopedic knowledge of pop culture from the period to understand why Williams' antics were more than just absurd drivel.
Finally, a computer must also understand that humor is also based largely on the history of humor. Age-old jokes can become funny again simply because they aren't funny in their original context any more, so their use in a new context is a subverted expectation in itself. Common joke patterns have also become fixed in human culture, such that merely following a pattern (like the Russian Reversal) is a joke in itself.
Algorithms simply haven't combined all of the relevant factors yet to recognize humor. Here on Slashdot, for instance, a computer would need to recognize the intellectual context, the pacing of a comment, the pattern of speech, and even how much class a commenter maintains, in order to realize when someone is trying to be funny.
Poop.
Just what we need! (Score:1)
An AI to add a laugh track to the Simpsons so you'll know when there has been a joke.
Why not Turing 2.0? (Score:2)
It seems like the startup investors would get sucked in then. Way more cool to be 2.0 than 1.0.
Give it a pile of junk (Score:3)
And tell it to make something useful.
Virtual junk is okay and any virtual tools can be used.
Of course there's a movement to replace it. (Score:3)
And of course there should be. But that doesn't diminish the importance of the Turing test.
The Turing test has two huge and closely related advantages (1) it is conceptually simple and (2) it takes no philosophical position on the fundamental nature of "intelligence". That such huge advantages necessarily entails certain disadvantages should come as no surprise to anyone.
The Turing test treats intelligence as a black box, but once you've contrived to pass the test the next logical step is to open up that black box and ask whether it's reasonable to consider what's inside "intelligent" or a tricky gimmick. That's a messy question, and that's *why* something like the Turing test is valuable. It is what social scientists call an "operational definition"; it allows us to explore an idea we haven't quite nailed down yet, which is a reasonable first step toward creating a useful *conceptual* definition. Science builds theories inductively from observations, after all.
If the Turing test were a suitable *conceptual* definition of intelligence than an intelligent agent would never fail it, but we know that can and does happen. We have to assume as well that people can be fooled by something nobody would really call "intelligence". Stage magicians do this all the time by manipulating audience expectations and assumptions.
Think of the Turing test as a screening test. Science is full of such procedures -- simple, cheap tests that aren't definitive but allow us to discard events that are probably not worth putting a lot of effort into.
Re: (Score:2)
the next logical step is to open up that black box and ask whether it's reasonable to consider what's inside "intelligent" or a tricky gimmick
When a computer truly passes the Turing test, the internal mechanism will be too complex for our judgement.
Re: (Score:2)
That's certainly a very interesting conjecture, but it's a little broad for my taste. I should think it more likely that we'll gain some insight, but that these insights will create new questions we can't answer right away.
Re: (Score:2)
I agree it will be interesting to look. At least we'll have better access to the internal information than with our human brains. But if the computer has intelligent behavior in every reasonable sense of the word, I don't think you'll be in a position to judge the internals as a "tricky gimmick" any more than you could call a human brain a tricky gimmick.
Another angle (Score:2)
What has become of those compression tests? Wasn't the answer to AI not (at least partially) found in the ability to compress?
I've got one (Score:2)
A test of intelligence should be dealing with unforeseen input. The problem with chatbots is that they are just giving pre-planned responses. How about trying to land a rocket on the moon while being bombarded with spurious input [wikipedia.org] from a radar device that was accidentally left on? Given the computers in use by NASA in 1969 that's pretty intelligent behavior.
Another would be landing a rocket on a small floating platform. We'll see how that plays out tonight.
Re: (Score:2)
Another would be landing a rocket on a small floating platform. We'll see how that plays out tonight.
That would be a very impressive display of controls algorithms, but not AI. You can build something doing essentially the same tasks with LEGO Mindstorms - taking sensor input, and using that to control physical motion. The only difference is that SpaceX has far more and far better sensors, has some very complicated and impressive intermediate math, and sends the output signals to rocket engines rather than electric motors.
That's all we need (Score:2)
That's all we need. Computers with a sense of humour:
"Oops! I deleted all your files!"
"Just kidding. I moved them to your Documents folder. :P"
Propaganda (Score:1)
The Turing Test has been abused, bypassed, and cheated to the point that almost no one knows what the actual Turing Test is. At this point, a new test needs to be created, a test that is difficult to cheat without making it obvious that it's not the real test. This could be "The Real Turing Test administered by [reputable group]".
Or we could make a new test, with incredibly explicit criteria that no one can nerf with a straight face and a different name. But from the sounds of it, it would be an easier test
How about replacing it with the ORIGINAL Test (Score:3)
The original Turing Test, as published in "Computing Machinery and Intelligence" as "Imitation Game" was not about whether a machine could successfully pretend to be a human.
He proposed a test, where a computer and men both pretended to be women, and the test would be passed if the computer would be more successful in lying about being a woman than the men were.
http://en.wikipedia.org/wiki/T... [wikipedia.org]
Re: (Score:2)
Turing was actually somewhat ambiguous. The first, and only detailed, formulation of the Imitation Game had an interrogator, a man, and a woman. After that, he switched to a person and computer. The question asked in the first game was intended to distinguish between a man and a woman (asking about hair style). The questions listed later aren't gender-related.
Re: (Score:1)
In my opinion the "what questions are asked" by the interrogator is only a small part of the test setup. I think the main point is "what is the question that is asked of the interrogator"
In that area the question "do you thing your opponent is a computer or a human" is influenced hugely by the interrogators knowledge and perception of what a computer should be able to do and what it should not be able to do. So asking the interrogator "find out if your opponent is a man or woman" might be a good way to have
Voight-Kampff ... (Score:2)
"Describe in single words, only the good things that come into your mind. About your mother."
Deep Mind's IQ Test Works (Score:2)
A rigorous definition of general intelligence now exists and has been applied by the Deep Mind folks. See this video lecture by Deep Mind's Shane Legg at Singularity Summit 2010 on a new metric for measuring machine intelligence [youtube.com].
If you want something more accessible to the general public, The Hutter Prize for Lossless Compression of Human Knowledge [hutter1.net] has the same theoretic basis as the test used by Deep Mind and has the virtue that it uses a natural language criterion, in the form of a Wikipedia snapshot. I
What is the purpose of the test? (Score:1)
Check your assumptions at the door (Score:2)
Or at least a real person.
Better go with answer number two. Doh!
Re:TL;DR People doesn't understand the Turing test (Score:5, Insightful)
TFA repeats a common misconception about the Turing Test. It is not a test of whether an AI can fool an average person, but whether it can fool an expert. ELIZA would never fool an AI expert, because that expert would be well aware than even a simple algorithm can be quite good at generating vacuous chit-chat. The pronoun disambiguation is a good test, because AI does that poorly, and humans do it well. But that is not a replacement for the Turing Test, that IS the Turing Test. Using humor is a good way to distinguish AI from humans. As anyone who has learned a foreign language, or raised children, knows, "getting jokes" is one of the last skills mastered. Humor often requires not only knowledge about the physical world, but deep understanding of cultural nuances. But I am not sure how useful that is, since no current AI would come close to passing it, and understanding jokes is probably not the most economically useful target for current AI research.
AI. It's like 3D TV. We don't have it. (Score:2)
There are no "current AIs", as that would require them to be intelligent, which in turn requires them to be conscious. And they aren't. Not even close.
If we really want to water the term "intelligence" down so that is applies to any clever algorithm, then perhaps we should spend a few minutes contemplating what it is we plan to call an actual intelligence, once we get that far.
"AI" has reasonable meaning in the context
Re: (Score:2)
Intelligence is not a black/white issue. It's more like a continuum of different levels.
Re: (Score:2)
Nonetheless, we know of no case where it has arisen or otherwise exists outside of a supporting consciousness. If you can point to an intelligence, it is a given, at present, that you are also pointing to a consciousness.
The fact that something can exist in various amounts or configurations does not in any way rule out the idea of its complete lack or presence. There are many things that can be accurately described just
Re: (Score:2)
An essential characteristic of intelligence is that we don't know how it works. Thus a computer program can never be intelligent, and the essential definitions of intelligence are pushed back over an horizon every time a test is beaten. Thus TFA.
Re: (Score:2)
We don't know how it works TODAY. That may change.
Intelligence: How does it work? (Score:2)
Actually, I think we do. [fyngyrz.com] We at least have an actual model, free of woo-woo, for which no counter evidence has been brought forth as yet.
Even the low level stuff seems to finally be yielding some clarity. [numenta.com]
Re: (Score:2)
You're assuming that we know how a sufficiently complicated program works, but it's easy to write a program and be unable to understand how it works (making it do something useful is much, much harder). I believe we can't understand intelligence, and therefore that an intelligent program would be far too complex to be understood by a human.
The definitions of intelligence are pushed back as we find ways to do more and more things without recognizable intelligence. It was once thought that chess would be
Re: (Score:2)
There are no "current AIs", as that would require them to be intelligent, which in turn requires them to be conscious.
Intelligence does not require "consciousness", any more than it requires a soul. Consciousness is a nebulous concept that has not even been defined in any objective falsifiable way. Intelligence is the ability to formulate an effective solution to novel problems. That is something that can tested and quantified.
Most people agree that "consciousness" is a property of internal state. Intelligence is a property of behavior. If something behaves intelligently, then it is intelligent. The internal mechanis
Re: (Score:1)
Most people agree that "consciousness" is a property of internal state.
I don't agree. Consciousness is just as much a property of behavior. Or do would you claim that you have no idea whether your friends and family are conscious ?
Re: (Score:2)
Or do would you claim that you have no idea whether your friends and family are conscious ?
Yes, I would claim that. I assume that they possess consciousness and self-awareness, because they are human, and I am human. But I have no objective, falsifiable test to distinguish self-aware objects from objects programmed to behave as if they were self-aware. Nor do I believe that such a test can exist, anymore than you can test if people have soul.
Here is the definition from Wikepedia: Consciousness is the quality or state of awareness, or, of being aware of an external object or something within o
Re: (Score:2)
Or do would you claim that you have no idea whether your friends and family are conscious ?
Yes, I would claim that. I assume that they possess consciousness and self-awareness, because they are human, and I am human. But I have no objective, falsifiable test to distinguish self-aware objects from objects programmed to behave as if they were self-aware.
But you are not saying that you have "no idea" whether other people have consciousness. Realistically, you know that they do, and a rock doesn't. You are just saying you can't prove it absolutely.
But this is true of almost anything, including whether the external world even exists or if you're the now legendary Brain in a Jar.
Nor do I believe that such a test can exist, anymore than you can test if people have soul.
No, there is ample evidence of consciousness, but none that souls exist.
Re: (Score:2)
Consciousness is by definition not provable by just about anything.
Once automatons mimic you, how would you show consciousness over their patterns?
Re: (Score:2)
Consciousness is by definition not provable by just about anything.
That doesn't mean it doesn't exist.
And, in fact, I know it exists, because I am conscious (even if the rest of the universe is not).
Believers in the possibility of artificial intelligence like to ignore the problem of consciousness, dismissing it as something unscientific or not provable. This then lets them claim that something like a chessc omputer is intelligent.
Qualities like consciousness, loving someone, appreciating the beauty of a piece of music or acting selflessly may be impossible to fully
Re: (Score:2)
Are you sure you have consciousness? Could it be an illusion? I believe at least one philosopher has argued that consciousness is an illusion (the name Dennet floats to mind).
Re: (Score:2)
The point is knowing that your consciousness exists doesn't help you know that other's consciousness exists. I know my consciousness exists, but how do you know my consciousness exists?
Re: (Score:2)
Intelligence does not require "consciousness", any more than it requires a soul. Consciousness is a nebulous concept that has not even been defined in any objective falsifiable way. Intelligence is the ability to formulate an effective solution to novel problems. That is something that can tested and quantifie
Ah, so evolution is intelligent then? Because that's exactly what evolution does.
Intelligence is a property of behavior.
No, it is associated with a property of behavior. Something can behave intelligently without being intelligent. For example, insects often form extremely complex and intricate structures (in some cases optimally designed structures). They're decidedly not intelligent, however, as their behavior is strictly governed by instinct. Extremely complex instincts that appear at first glance to be intelligence, but only instinct. The s
Re: (Score:2)
Intelligence is the ability to formulate an effective solution to novel problems.
Ah, so evolution is intelligent then? Because that's exactly what evolution does.
No. Evolution solves problems that occur repeatedly, over many generations. Evolved systems often fail spectacularly when confronted with novel situations.
insects often form extremely complex and intricate structures ... They're decidedly not intelligent, however, as their behavior is strictly governed by instinct
... and the reason it is considered "instinct" rather than "intelligence" is because the behavior does not adapt to novel situation.
Re: (Score:2)
To the extent that this is true (it isn't always), it is also true of solutions created by human means and intent -- products of our intelligence.
You're really not making a good case at all.
Re: (Score:2)
To the extent that this is true (it isn't always), it is also true of solutions created by human means and intent -- products of our intelligence.
Then that is not very intelligent. Intelligence is not something that is binary. It is a spectrum. Most people would agree that humans are more intelligent than monkeys, monkeys are smarter than dogs, and dogs are smarter than chickens. As you move up the scale, they will have more robust solutions to a wider range of problems. Build a linear fence between a chicken and its food, and the chicken will run back and forth. Do the same for a dog, and the dog will figure out that it can go around the fence
Re: (Score:2)
Wait, what? Failing when confronted with a novel situation isn't very intelligent? Your experience must have been quite different from mine. My life is a story of one failure after another, with successes being the exception rather than the rule. Not so much because I am not intelligent, but because I am. It's true enough that in most cases, my successes outweigh my failures, primarily because I continued to put effort into them, but they certainly aren't as numerous.
Re: (Score:2)
Please correct me if I am wrong, but I am under the impression that what you are describing here is something that evidences order, not intelligence. At best, these examples exhibit something that can be described from the outside as ordered behavior: a crystal grows according to inherent rules, but it is not doing any self-guiding.
Evidence of intelligence tends to come in the form behaviors more sophisticated, and at the same time, often quite a
Re: (Score:2)
Well, let me put it to you this way: So far, the only examples of intelligence we have spring from conscious entities. If intelligence can be achieved without consciousness, that would be absolutely fascinating. But seeing as there isn't even a hint of such a thing anywhere, it is just an insupportable idea at the moment.
"soul" is just code-speak for superstition. Doesn't even belong in this conversation. Consciousness is demon
Re: (Score:2)
Consciousness is demonstrable, even just within one's own head.
Consciousness is demonstrable only within one's own head.
Re: (Score:2)
So, no one is conscious but you?
Re: (Score:2)
Re: (Score:2)
Ah. but I can. And I have. [fyngyrz.com]
Re: (Score:2)
So, no one is conscious but you?
If something cannot be demonstrated, that does not necessarily mean it does not exist. There is no objective falsifiable test for "consciousness" or "self-awareness" other than "cognito ergo sum", and that is only valid within one's own mind.
"Intelligence" and "consciousness" are two different things. Intelligence is easy to demonstrate: solve a general selection of problems. "Consciousness" cannot be demonstrated, and few people even agree on a definition, and some philosophers question when it, like "f
Re: (Score:2)
Hold it right there. We're talking about consciousness. It can certainly be demonstrated. In fact, it's pretty obvious to a moderately interested, well educated observer except within the context of the very smallest nervous systems that support it. We can even switch it on and off with anesthetics in an otherwise healthy animal, and again, it is stunningly obvious when we do so when the subject is a human.
What you are describing when you assert a failure to demonstrate c
Re: (Score:2)
Also, somewhat of a separate issue, but interesting... what cogito ergo sum illuminates is not so much that one is conscious, but that one exists at all. Insofar as it relates to consciousness, it is a direct expression of exactly what I stated earlier, with no known exceptions thus far: intelligence is an emergent facet of consciousness. We know of exactly zero cases where the former exists without the latter, and again so far, we have no reason to assume it can.
Re: (Score:2)
"Intelligence" and "consciousness" are two different things. Intelligence is easy to demonstrate: solve a general selection of problems.
But that argument leads to the conclusion that "artificial intelligence" is demonstrated by the computer in my car which knows to turn my windscreen wipers on when it's raining.
Re: (Score:2)
Consciousness is demonstrable, even just within one's own head.
Consciousness is demonstrable only within one's own head.
Strictly speaking, you can't prove to me that anything has an objective existence outside of my own head, but that doesn't really get anyone very far.
Re: (Score:2)
Religion is demonstrable only within one's own head. Lots of people have personal religious experiences, and therefore adhere to some religion or another. It isn't nearly as widespread as conscious experiences are reported to be, so there's lots of people who never have had religious experiences and never will. Consciousness is demonstrable in the exact same way that religion is demonstrable to a large number of people.
Re: (Score:2)
If we really want to water the term "intelligence" down so that is applies to any clever algorithm, then perhaps we should spend a few minutes contemplating what it is we plan to call an actual intelligence, once we get that far.
"Master", I imagine.
Re: (Score:2)
understanding jokes is probably not the most economically useful target for current AI research.
A joke detector? That's funny. I mean a sarcasm detector? That's real useful.
Re: (Score:2)
A long time ago I used to work in the field of AI (Expert systems and Neural nets). IT frustrates and pisses me off no end how frequently press and even a lot of IT people fail to understand what is essentially a straight forward test and then complain about it being inadequate for modern computing. No computer has passed it, not even close. The test REQUIRES, a human and a computer, The test REQUIRES the expert to be aware that one is human and the other a computer. The test REQUIRES that they then get to
Re: (Score:2)
I quite liked how they handled it in recent film "The Machine" ( http://www.imdb.com/title/tt23... [imdb.com] ). Questions like "Which smells better, a hospital corridor or a donkeys ass?" and "Mary saw a puppy in the window and she wanted it. What did Mary want?"
Re: (Score:2)
"Mary saw a puppy in the window and she wanted it. What did Mary want?"
An ambiguous subject in a phrase is a classic problem in AI, however natural language algorithms (such as the one found in Watson) have been able to resolve ambiguous statements like your "Mary" example and the trophy/bag example in TFS, for over a decade now. The trick to resolving such ambiguities is the same one used by humans; context, probability, and lots of prior examples.
Re: (Score:1)
Bullshit. http://www.artificial-intelligence.com/comic/7
People's vanity about human exceptionalism has them move the goal-posts as required to preserve their sense of identity at the top of the food chain.
AI is already able to beat people at most isolated tasks. With ROS uniting the disparate fractured efforts under a single framework, the inefficiency of researchers working on problems in isolation from each other has been solved. There's a standard now, and a couple versions of jQuery from now: your perso
I'll Raise You an Expert... (Score:2)
"The WOPR spends all of its time thinking about [Turing Tests]. 24 hours a day, 365 days a year, it plays an endless series of [Turing test 'games'], using all available information on the state of [human sentience]. It has already proved the existence of [machine intelligence] as a game, time and time again. It estimates human and machine responses to our test responses to their responses, and so on. Estimates probabilities, tallies the score, and it looks for ways to ---"
"The point is, key decisions o
Re:TL;DR People doesn't understand the Turing test (Score:5, Insightful)
The pronoun disambiguation is a good test, because AI does that poorly, and humans do it well. But that is not a replacement for the Turing Test, that IS the Turing Test.
Indeed. Here's an excerpt from Turing's original paper [loebner.net] that described the "imitation game," replying to a possible objection that his test would not be able to be used to gauge true understanding as a human might:
Probably [the objector to the test] would be quite willing to accept the imitation game as a test. The game (with the player B omitted) is frequently used in practice under the name of viva voce to discover whether some one really understands something or has "learnt it parrot fashion." Let us listen in to a part of such a viva voce:
Interrogator: In the first line of your sonnet which reads "Shall I compare thee to a summer's day," would not "a spring day" do as well or better?
Witness: It wouldn't scan.
Interrogator: How about "a winter's day," That would scan all right.
Witness: Yes, but nobody wants to be compared to a winter's day.
Interrogator: Would you say Mr. Pickwick reminded you of Christmas?
Witness: In a way.
Interrogator: Yet Christmas is a winter's day, and I do not think Mr. Pickwick would mind the comparison.
Witness: I don't think you're serious. By a winter's day one means a typical winter's day, rather than a special one like Christmas.
And so on, What would Professor Jefferson say if the sonnet-writing machine was able to answer like this in the viva voce? I do not know whether he would regard the machine as "merely artificially signalling" these answers, but if the answers were as satisfactory and sustained as in the above passage I do not think he would describe it as "an easy contrivance."
THAT is the sort of standard of AI that Turing was envisioning could be passed in his "test." It isn't a computer pretending to be a non-responsive teenager with an attitude problem who doesn't really speak the same language as the interrogator (as some chatbots might claim).
It's an idea of AI as something that could debate word replacement in a Shakespearean sonnet, would understand and be able to process poetic scansion, understand the subtle word meanings and connotations in language, and be able to synthesize these various things together while applying such concepts to evaluations of classic literary references.
Turing's test then assumes an AI competent enough to have a flawless conversation on the level of a bright university student or even a colleague of Turing's. Now, granted, we might find the literature quiz a little unnecessary, but in a more general sense this example gets at the idea of probing the AI's understanding of concepts, connecting disparate uses of things together (like a literary character to an abstract concept to a matter of style or poetic form), and in general a fluent and adaptive recognition of linguistic meaning.
I think we would all agree that the various chatbots that have claimed in recent years to have "passed the Turing test" are NOWHERE near this level.
This is the kind of standard Turing himself explicitly mentioned in his original article on the test. And frankly, if I encountered an AI that could have a conversation this fluid and wide-ranging (even if not on literature specifically) in flawless English, I'd be happy to declare it "intelligent." But we don't have anything close to that -- and pretending the "Turing test" is obsolete and needs to be more strict is misunderstanding the ridiculously high expectations Turing himself set out many decades ago.
Re: (Score:1)
To easily detect most AI, tell it this:
Re: (Score:2)
The Turing Test was set up as a three-entity interaction: one questioner, one human, and one AI. The questioner is supposed to converse with both the human and the AI (presumably by typing and reading messages), and decide which of the others is the human and which the AI. There was no mention of expertise in any field, and it would be hard for Turing to put that in since there were no AI experts in Turing's day.
Two of the questions could be put into the Turing test easily: the pronoun assignment one
Re: (Score:2)
It is not a test of whether an AI can fool an average person, but whether it can fool an expert.
You are not allowed to redefine the test just because it makes you more comfortable to do so. The original paper simply said "A man, a woman, and an interrogator". It did not qualify that interrogator as an expert, but simply the one who poses the questions (thus, an interrogator)
Well, please re-read the original paper [loebner.net].
You are correct that the original test did not specify an AI expert as interrogator. On the other hand, read the types of dialogue Turing offers as examples. It's very clear that he is imagining "interrogators" (note that word -- it implies someone with a strong drive to ask probing questions) who are not only quite intelligent but also keep asking very probing questions designed to test the intellect of the person/thing on the other side.
The standard is clearly
Re: (Score:2)
The difference is that the "swiftboaters" were lying and ended up getting sued.
Re: (Score:1)
Thats fair. However, the article is fair in its opinion too.
What was good in 1950 may not be so relevant anymore.
The base of the test is probably fine. But an updated one for things we want an AI to do today is a good idea.
Much like the ACID tests for browsers. They help set the bar for what we want out of our computers.
Right now most 'AI' is brute force depth searching with some statistical weighting. Is that AI?
Re:TL;DR People doesn't understand the Turing test (Score:5, Insightful)
Investigators should be trying to find which one is human, not simply chatting with the computer. Too often people are simply connected to a chatbot and not told that it might be a computer until after the fact, no foil is involved, etc. The test is also often declared to be passed if even a single investigator fails.
Re: (Score:2)
You don't need two terminals. All you need is the human interrogator, and have him/her talk to either a human or a computer. I agree that they must be aware of the challenge, and also have the ability to ask some decent questions.
Re: (Score:2)
Re: (Score:2)
I don't see that as a problem with the test itself.
I see that as various individuals trying to cheat in order to claim that they have achieved something they have not.
Suppose someone claimed to have beaten the world record for the 200 meter dash. But could only do it with a 190 meter headstart.
Okay, no headstart but I get to use a motorcycle.
Okay, okay, no headstart and no motorcycle but I will be using "meters" that are 10cm long.
No one woul
Re: (Score:2)
Re: (Score:2)
The Turing Test is a thought experiment. It's just saying "if you can talk to this, and can't tell if it's a person or a computer, then it doesn't matter: it's intellegent." It's not a method for a scientific, practical process. It's just something to think about when considering what might constitute intelligence.
Re: (Score:2)
The Turing Test is a thought experiment. It's just saying "if you can talk to this, and can't tell if it's a person or a computer, then it doesn't matter: it's intellegent." It's not a method for a scientific, practical process.
If that's true, then why did Turing claim in his original paper that by the year 2000, computers would be able to fool humans and "pass the test" 30% of the time? Why state such a specific prediction for a test that was not intended to be practical and only a "thought experiment"?
It's just something to think about when considering what might constitute intelligence.
Why can't it be both? In Turing's time (and still today) there were (and are) people who think real strong human-like AI is impossible. In order to evaluate "intelligence," though, we need a standard test that we could agree
Re: (Score:2)