Author |
Topic: How far away from "perfect play"? (Read 2811 times) |
|
Ail
Forum Guru
Rabbits can't push Rabbits!
Gender: 
Posts: 52
|
 |
How far away from "perfect play"?
« on: Feb 22nd, 2014, 8:13pm » |
 Quote  Modify
|
First of all: This thread is not about analysing a specifig game, it's about a thing that could be analysed about this game and other games as well.
One might ask: How could anything be compared to "perfect play", if we do not have any idea what perfect play looks like?
I came up with an idea how to determine how far away we are from perfect play without having to know that:
Of course it's no real science but I think it could be a nice way to estimate it.
We take a game of the best 2 players that took place between them and someone else and have them go through the game of the other and count how many times they would have played the same move.
If they agree about a move, we could then say that a move where both would have played the same was roughly "perfect".
All other moves could be seen as "not perfect".
At the end there should be a percentage of "perfect moves vs. total moves".
The higher the value, the closer we can assume our top players being to perfection.
In chess the "top players" (as in different engines) agree about a lot of moves. (in a small test only including 10 moves two different top-engines agreed 70% of the times)
I'd expect a significantly lower value for Armiaa here.
What do you think about this method? May it be worth being applied?
|
|
 IP Logged |
|
|
|
odin73
Forum Guru
Gender:
Posts: 65
|
|
Re: How far away from "perfect play"?
« Reply #1 on: Feb 23rd, 2014, 10:09am » |
Quote Modify
|
Indeed, probably almost everybody here would like to find the perfect game. But how serious can this be for Arimaa? Even finding a "perfect move" may cause some unsolvable problems.
First it depends on the strategy. Is it aggressive or a bit more sound and careful. So far, there is no consensus about the optimal playing style.
Even a human crowd may be wrong. Take this game: http://arimaa.com/arimaa/gameroom/opengamewin.cgi?client=1&gameid=14 5628&role=v&side=w
There is no proof that the crowed played an optimum (even when it won the game).
On the other hand Iīm quite sure that the latest bots find very different moves for that game as well as top players would. But this is no proof that the game was far from perfect.
In the case of chess the horizon for silicon intelligence is much wider than for Arimaa. It isnīt yet possible to search such deeply in the position and probably itīll be never possible. The playing versa a bot like "walking in the minefield" as stated by Kasparow may be still far away (even when itīs already true in some endgame positions).
So, finally, itīs probably hard job to find even a single perfect move. 
|
|
IP Logged |
|
|
|
Ail
Forum Guru
Rabbits can't push Rabbits!
Gender:
Posts: 52
|
|
Re: How far away from "perfect play"?
« Reply #2 on: Feb 23rd, 2014, 12:42pm » |
Quote Modify
|
I actually meant something completely different.
I wanted to know how much of the potential to become better still resides in this game.
As in: If we can't agree about what moves are the best, then there must be massive potential to improve.
For chess we think that perfect play would lead to a draw. This seems to be true when looking at top-engine-games nowadays. They draw very, very often. In Arimaa there are no draws so it cannot be taken to conclude the improving-potential.
|
|
IP Logged |
|
|
|
lightvector
Forum Guru
Arimaa player #2543
Gender:
Posts: 197
|
|
Re: How far away from "perfect play"?
« Reply #3 on: Feb 23rd, 2014, 7:48pm » |
Quote Modify
|
Random fact: With some simple machine learning, it's possible to exactly predict about 14% of the moves in games by players above 2200 over the last few years. I have some code that achieves this, it was the result of the first half of my undergraduate thesis research.
Most of the value of that number, however, has less to do with the strength of the moves by the predictor and more to do with the frequency with which forced or nearly forced moves occur in Arimaa. For example, during a trade, if you're the second player to capture, there's often nothing better to do than execute the capture, particularly if failing to capture would immediately allow the opponent to save his piece and win something for nothing rather than trading it.
|
|
IP Logged |
|
|
|
jdb
Forum Guru
Arimaa player #214
Gender:
Posts: 682
|
|
Re: How far away from "perfect play"?
« Reply #4 on: Feb 24th, 2014, 8:20am » |
Quote Modify
|
I could set up clueless to use the tablebases and the start method from the endless endgame event. We could start with a (difficult) position that is a win for the player, and see if they can play perfectly enough to win it.
If there is any interest, I can set this up.
|
| « Last Edit: Feb 24th, 2014, 8:21am by jdb » |
IP Logged |
|
|
|
chessandgo
Forum Guru
Arimaa player #1889
Gender:
Posts: 1244
|
|
Re: How far away from "perfect play"?
« Reply #5 on: Feb 24th, 2014, 4:31pm » |
Quote Modify
|
Interesting.
Jdb, I'm sure browni, clyring and some other EEE regulars would love to do that. I would certainly have a go at it given the chance.
|
|
IP Logged |
|
|
|
browni3141
Forum Guru
Arimaa player #7014
Gender:
Posts: 385
|
|
Re: How far away from "perfect play"?
« Reply #6 on: Feb 24th, 2014, 5:00pm » |
Quote Modify
|
I would try what jdb proposed. It sounds fun.
Also, my estimate for the percentage of realistic positions where chessandgo and I would play the same move at a slow time control is maybe 20%-30%, but this is basically a wild guess.
I chose chessandgo because I think he might be the player with which I agree most often.
|
|
IP Logged |
|
|
|
jdb
Forum Guru
Arimaa player #214
Gender:
Posts: 682
|
|
Re: How far away from "perfect play"?
« Reply #7 on: Feb 24th, 2014, 6:02pm » |
Quote Modify
|
I'll do my best to set it up this week.
There is a problem, I no longer have the DCR_dcr tablebase. I copied it to an external hard drive. For some reason the file got cut off at 4GB. Most annoying. There is a slight chance I can find another copy lying around, but not likely.
Best I can do is set up some 5 man positions.
|
|
IP Logged |
|
|
|
Ail
Forum Guru
Rabbits can't push Rabbits!
Gender:
Posts: 52
|
|
Re: How far away from "perfect play"?
« Reply #8 on: Feb 25th, 2014, 5:48am » |
Quote Modify
|
@browni:
That is kind of the statement I was interested in.
According to my theory a 20-30% professionals-agreeing rate means, that we are still quite far away from knowing what perfect play even could look like.
In my book this is a great thing as it shows that there is so much more potential of improvement in this amazing game!
|
|
IP Logged |
|
|
|
Kushiel
Forum Full Member
Arimaa player #9913
Gender:
Posts: 16
|
|
Re: How far away from "perfect play"?
« Reply #9 on: Sep 15th, 2014, 3:41pm » |
Quote Modify
|
Counting "how many times they would have played the same move" is probably not a good method for determining if a play was perfect.
If you ask different experts which move they'd make for a given board position, then compare answers, they might say different things even for completely solved games.
Take Tic-Tac-Toe for example. Ask 3 different experts what they'd do on the first move, and you could very well get 3 different answers, since all of the starting positions lead to a draw with perfect play. Ignoring symmetries, only 3 starting moves exist: edge, corner, and center. All are draws.
Asking if experts agree that a move was in the set of "perfect moves" is also not theoretically sound, but may offer practically better results for Arimaa.
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print