Author |
Topic: A Turing Test (Read 2634 times) |
|
rozencrantz
Forum Full Member
Arimaa player #4007
Gender: 
Posts: 17
|
 |
A Turing Test
« on: Sep 16th, 2010, 1:35pm » |
 Quote  Modify
|
I've read a lot about chess and Go players getting a read of the personality of their opponent, Yasunari Kawabata in particular talked a lot about how one's character is revealed in playing Go. I've even read some comments [citation needed] describing playing Chess at high levels as a rather intimate act.
So: is the art of Arimaa developed enough for that kind of thing? Do bots play noticeably differently from equally strong (or weaker) humans? Is the skill gap at the extreme end of things the only difference between human and bot play? (I doubt it)
And ultimately, could you play two unmarked opponents of similar skill and discern the human from the bot? Is there a way of trying this, with at least some degree of blinding? Has this been tried?
|
|
 IP Logged |
|
|
|
clojure
Forum Guru
Arimaa player #5004
Gender:
Posts: 207
|
|
Re: A Turing Test
« Reply #1 on: Sep 16th, 2010, 1:49pm » |
Quote Modify
|
This test has fatal flaw as a Turing test where the human tries to convince the watcher of him being human. So easy way is to start playing not for the win but encoding messages as moves.
But if both players were not informed they were being inspected, it's different matter. But to my knowledge current bots don't tend to for example learn between games. So if many a game is allowed, the one who is getting better at beating the other one, is human.
Also as a beginner player, I'd like to think that top human players easily distinguish sensible strategic movements from tactical, and the play yells of the overall understanding what to do, which bots struggle with. On the other hand, if player makes clearly a bad tactical move that does not make any advantages in long term, it's probably that the player is human if both are rated high.
These are the first things that popped up to my mind... Overall I agree that, for example in Go, player's characteristic might show up strongly but I suspect that best players can imitate a bit worse players quite well.
edit: I have no idea whether this kind of experiment has been tried.
|
| « Last Edit: Sep 16th, 2010, 1:51pm by clojure » |
IP Logged |
|
|
|
Sconibulus
Forum Guru
Arimaa player #4633
Gender:
Posts: 116
|
|
Re: A Turing Test
« Reply #2 on: Sep 16th, 2010, 1:57pm » |
Quote Modify
|
You can tell the difference between humans and bots when playing them, usually fairly easily. There are only two entities I recall playing that felt like there could have been any question. Fritzlein, because he crushed me so thoroughly and controllingly that he could as easily have been a superior bot as a superior human, and bot_Marwin, who feels fairly humanesque somehow. I think it's because of the way he decides moves, I believe I was told after my games that he chooses weighted-random from among those moves he decides are best.
|
|
IP Logged |
|
|
|
Nombril
Forum Guru
Arimaa player #4509
Gender:
Posts: 292
|
|
Re: A Turing Test
« Reply #3 on: Sep 16th, 2010, 3:53pm » |
Quote Modify
|
Interesting question. I sometimes wonder if I approach playing a bot and a human differently, I think this must be related to me having an expectation that a bot and a human will play differently.
I've noticed many bots (even good ones) will sometimes move a piece 1 step and then back to where it started (as 2 of 4 steps) when there isn't anything else it can find to do. If the bots aren't programmed to "disguise" themselves, this could be a big give away. So I think at this point it would be relatively easy to tell the difference, especially if the sample size were large enough.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: A Turing Test
« Reply #4 on: Sep 16th, 2010, 10:50pm » |
Quote Modify
|
Recall that in the Deep Blue vs. Kasparov match, the Turing Test was passed by Deep Blue. The evidence is that Kasparov accused IBM of cheating, saying that a computer could not have played such moves without human assistance. That incident makes me suspect that it is only easy to tell a human opponent from a computer opponent until such time as the opponent plays better than you.
|
|
IP Logged |
|
|
|
clojure
Forum Guru
Arimaa player #5004
Gender:
Posts: 207
|
|
Re: A Turing Test
« Reply #5 on: Sep 17th, 2010, 8:01am » |
Quote Modify
|
Fritzlein, I'm a bit confused why you use technical term so freely. In Turing test, the one who tries to distinguish the human from computer, is interacting with them, which is a crucial component.
Here's the original paper:
http://mind.oxfordjournals.org/content/LIX/236/433
But nevertheless the question rozencrantz is asking is interesting. Even if bots would outbeat humans, could they imitate human characteristics in an Arimaa game so that an observer cannot tell which one is a bot.
|
| « Last Edit: Sep 17th, 2010, 8:26am by clojure » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: A Turing Test
« Reply #6 on: Sep 17th, 2010, 11:38am » |
Quote Modify
|
on Sep 17th, 2010, 8:01am, clojure wrote:| Fritzlein, I'm a bit confused why you use technical term so freely. |
|
What I meant to say is that one can think of limiting the interaction to a chess game, rather than to a conversation, to get the Turing Test For Chess, and that Deep Blue passed the Turing Test For Chess. Is your objection to my post that I forgot to add the words "For Chess"? If so, then 99% of the disagreement between us was due to my being inattentive. On the other hand, if your objection was the people play games to play games and aren't really focusing on determining the identity of their opponent, then we have a more substantive difference in perspective.
I appreciate your point about the possibility of encoding conversation in chess moves, but I think that is not in the spirit of rosencrantz's original post, nor what I am most curious about. The more immediate question for me is not whether a chess game could be used as a tool in some rarefied way to distinguish a human from a computer, but rather whether chess when played for the sake of chess (e.g. trying to win every game) allows one to incidentally detect whether one's opponent is human or AI.
You refer to the Turing Test as technical term, but it is hardly as precisely defined as technical terms such as "renal cell carcinoma". For example, you might say that as long as there exists some pattern of questioning, however unrelated to normal human conversation, that allows an expert human interrogator to distinguish man from machine, the Turing Test has not been passed. I would tend to set a much lower bar, namely that if I treat my conversational opposite normally, as I would treat a human, and fail to detect that I am getting non-human responses, then the Turing Test is passed in my book. I mostly want to know whether whether conversation undertaken for the sake of conversation allows one to incidentally detect whether one's conversational partner is human or AI.
The "naturalness" of the conversation in question seems to matter somehow. For example, if I were in a real conversation, and someone asked me some bizarre question out of the blue for the purpose of determining whether or not I was human, a very human and natural response from me would be, "I don't have time or energy to convince you of my humanity. Goodbye." In order to set up the Turing Test as he described it, though, we have to make this thoroughly human response against the rules!
The standard that Turing proposed is more demanding than mine, in that the objective of the questioner is not to have a normal conversation, but explicitly to make a determination, in a highly artificial situation, whether or not the conversation partner is human. I have a hunch, though, that there are grey areas that would disturb even Turing, were he alive to see the details of the contest played out. If computers come close to succeeding, there will probably be a time period in which more and more specialized expertise in necessary to make a correct determination, and ordinary people with ordinary questions are less and less capable of doing it. Turing himself might consider machines intelligent even if one person somewhere still has the technique to discriminate but nobody else does.
|
| « Last Edit: Sep 17th, 2010, 12:41pm by Fritzlein » |
IP Logged |
|
|
|
Hippo
Forum Guru
Arimaa player #4450
Gender:
Posts: 883
|
|
Re: A Turing Test
« Reply #7 on: Sep 17th, 2010, 12:31pm » |
Quote Modify
|
on Sep 16th, 2010, 1:57pm, Sconibulus wrote:| You can tell the difference between humans and bots when playing them, usually fairly easily. There are only two entities I recall playing that felt like there could have been any question. Fritzlein, because he crushed me so thoroughly and controllingly that he could as easily have been a superior bot as a superior human, and bot_Marwin, who feels fairly humanesque somehow. I think it's because of the way he decides moves, I believe I was told after my games that he chooses weighted-random from among those moves he decides are best. |
|
What is the main difference in marwins play to other bots is ... marwin does not like passivity. It always choses a plan what to do ... usually plans longing several turns. These plans are not necessary optimal, but it forces his opponent to play reactively.
Against other bots its sufficient to gain small advantage and slowly improve the position. As they don't attack you could play according to strategical plan and improve your position till they lost.
On the contrary Marwin interrupts such a slow decay by counterstrikes really often. You could gain something from the counterstrikes as they are not safe, but if you miss some of them, you could lose a
lot.
This is why I have big problems with MarwinBlitz. ... Pondering is much more complicated in reactive game than in proactive.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: A Turing Test
« Reply #8 on: Sep 17th, 2010, 12:48pm » |
Quote Modify
|
on Sep 16th, 2010, 1:35pm, rozencrantz wrote:| I've even read some comments [citation needed] describing playing Chess at high levels as a rather intimate act. |
|
I recall from way back in the early days of the ICC, Garry Kasparov once logged on anonymously for some practice before a match, and was surprised to meet very stiff (albeit very human) resistance from one player. If I remember correctly, he was able to deduce precisely that his opponent was Peter Svidler. Sounds intimate to me...
|
|
IP Logged |
|
|
|
clojure
Forum Guru
Arimaa player #5004
Gender:
Posts: 207
|
|
Re: A Turing Test
« Reply #9 on: Sep 17th, 2010, 1:14pm » |
Quote Modify
|
Well, my objection was that chess is only a subset of the possibilities what human judge can use as a tool to identify which one is computer. I think the spirit of Turing test is that human should be able to use every way possible but physical detection. So the tests beauty is that it doesn't predefine how judge can work out the problem. If he had done that, he would have made the test less appealing as a universal test. Consider when the computer passes Turing test with predefined rules, which even don't probably capture the essence of human intelligence: people would object that humans were handicapped and keep continuing to believe that humans have intelligent in different class. Well, maybe they woudn't. But I would.
As I said, the question of limited Turing test with participants that are not even aware of being tested, is fascinating, but let's not call it Turing test. Even if I misused the word technical, it is still quite well established, and the true spirit is different from this even though they have the same kind of objective: to see whether we are similar to computers in current age.
So let's call it rather rozencrantz test. (Maybe they will make Watchmen 2 with Rorschach replaced with a robot).
|
| « Last Edit: Sep 17th, 2010, 1:15pm by clojure » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: A Turing Test
« Reply #10 on: Sep 17th, 2010, 1:31pm » |
Quote Modify
|
on Sep 17th, 2010, 1:14pm, clojure wrote:| As I said, the question of limited Turing test with participants that are not even aware of being tested, is fascinating, but let's not call it Turing test. Even if I misused the word technical, it is still quite well established, and the true spirit is different from this even though they have the same kind of objective: to see whether we are similar to computers in current age. |
|
Hmmm, I'm not sure how different the "true spirit" of two tests can be if they have the same objective and similar means. We're both talking about whether a computer can convincingly simulate a human through the medium of conversation. Were the Kasparov vs. Deep Blue match and the Kramnik vs. Deep Fritz match in a different spirit because Kramnik had many advantages (commodity hardware, unchanging opponent, pre-match opportunity to study opponent, in-match view of opening book, etc.) that Kasparov didn't have? In my mind they were both man vs. machine contests with the bar for computers raised or lowered my the match conditions. Similarly I think of what I propose as very much in the true spirit of the Turing Test but with the bar lowered for the computer by the conversation conditions. I'm a bit suspicious that humans who want to avoid calling computers intelligent will instinctively insist on the highest possible bar, regardless of whether it is in line with a common-sense understanding of conversationally simulating a human.
But maybe I am mistaken to view things so loosely that one can refer to a Turing Test rather than the Turing Test. Perhaps I converse with too many philosophers and not enough computer scientists. 
As for restricting the medium of interaction to a game rather than conversation, I agree with you that it shouldn't be called the Turing Test. That was my mistake in my first post in this thread. I should use some term like chess-turing-test to distinguish it from the (real) conversation-turing-test.
|
| « Last Edit: Sep 17th, 2010, 1:39pm by Fritzlein » |
IP Logged |
|
|
|
clojure
Forum Guru
Arimaa player #5004
Gender:
Posts: 207
|
|
Re: A Turing Test
« Reply #11 on: Sep 17th, 2010, 1:37pm » |
Quote Modify
|
I forgot to say this: if we were to limit it to particular game, for instance, it would make some serious problems.
First off, how do we know that the game is good enough a test. Clearly 3x3 tic-tac-toe would fail as one. So we would spend lots of time arguing whether that particular game captures intelligence... So we had to just talk about similar style in particular game...
Also how would you reliably setup this experiment? How is the bot selected, and how is the player selected that doesn't know he is participating in this contest?
|
| « Last Edit: Sep 17th, 2010, 1:39pm by clojure » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: A Turing Test
« Reply #12 on: Sep 17th, 2010, 1:45pm » |
Quote Modify
|
on Sep 17th, 2010, 1:37pm, clojure wrote:| First off, how do we know that the game is good enough a test. Clearly 3x3 tic-tac-toe would fail as one. So we would spend lots of time arguing whether that particular game captures intelligence... |
|
The argument about whether any particular test captures intelligence is unavoidable. The are people who argue that even if a machine could pass the Turing Test as you define it, that machine would not be intelligent. I engaged in an e-mail debate with such a person just last month; he insisted that human intelligence encompasses more than the ability to converse.
In my previous posts I was arguing about what I think deserves the label of "Turing Test", not what I think deserves the label of "intelligence". My personal opinion is that computers are already intelligent, albeit not yet as intelligent as humans. The Turing Test is only one factor in my judgement of how intelligent machines have become.
The tic-tac-toe-turing-test is obviously one that computers could pass, but it doesn't play a very big role in my understanding of machine intelligence.
|
| « Last Edit: Sep 17th, 2010, 1:48pm by Fritzlein » |
IP Logged |
|
|
|
clojure
Forum Guru
Arimaa player #5004
Gender:
Posts: 207
|
|
Re: A Turing Test
« Reply #13 on: Sep 17th, 2010, 1:55pm » |
Quote Modify
|
So let's focus on Turing test as in picking up human traits and not intelligence.
I think the spirit differentiates for example in how well one can change your behavior depending on the earlier conversations / games. With arbitrary conversation, you can quickly see whether the other one is following your thought. With game, it's much harder to see whether the participant is reacting similarly or is having true understanding what you mean.
Also, I'm still wondering how you can make fair setup of the test. If bots know they are participating, it's a failed test. How do you know the bot isn't been done humanish behaviour particularly in mind?
|
|
IP Logged |
|
|
|
clojure
Forum Guru
Arimaa player #5004
Gender:
Posts: 207
|
|
Re: A Turing Test
« Reply #14 on: Sep 17th, 2010, 1:58pm » |
Quote Modify
|
I kinda agree that "that human intelligence encompasses more than the ability to converse."
For example, some autistic people have incredible talent with music but when they try to describe it, they cannot; even so that their brain image shows how his brain disagrees what he is saying.
I saw on youtube where guy could repeat in piano the notes what the whole small orchestra was playing. It was something like 15-20 notes at one point of time interval.
If people like that participate in improvising beautiful music in real time, I think it's clear indication of intelligence.
Well it is conversation in generalized sense but...
Also some parts of human brains activate only when in visual contact with other person.
|
| « Last Edit: Sep 17th, 2010, 2:05pm by clojure » |
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print