Arimaa Forum - Print Page


    
      
        Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Game Analysis >> How far away from "perfect play"?
        
(Message started by: Ail on Feb 22^nd, 2014, 8:13pm)

Title: How far away from "perfect play"?
Post by Ail on Feb 22^nd, 2014, 8:13pm

First of all: This thread is not about analysing a specifig game, it's about a thing that could be analysed about this game and other games as well.

One might ask: How could anything be compared to "perfect play", if we do not have any idea what perfect play looks like?

I came up with an idea how to determine how far away we are from perfect play without having to know that:

Of course it's no real science but I think it could be a nice way to estimate it.

We take a game of the best 2 players that took place between them and someone else and have them go through the game of the other and count how many times they would have played the same move.

If they agree about a move, we could then say that a move where both would have played the same was roughly "perfect".

All other moves could be seen as "not perfect".

At the end there should be a percentage of "perfect moves vs. total moves".

The higher the value, the closer we can assume our top players being to perfection.

In chess the "top players" (as in different engines) agree about a lot of moves. (in a small test only including 10 moves two different top-engines agreed 70% of the times)

I'd expect a significantly lower value for Armiaa here.

What do you think about this method? May it be worth being applied?

Title: Re: How far away from "perfect play"?
Post by odin73 on Feb 23^rd, 2014, 10:09am

Indeed, probably almost everybody here would like to find the perfect game. But how serious can this be for Arimaa? Even finding a "perfect move" may cause some unsolvable problems.

First it depends on the strategy. Is it aggressive or a bit more sound and careful. So far, there is no consensus about the optimal playing style.

Even a human crowd may be wrong. Take this game: http://arimaa.com/arimaa/gameroom/opengamewin.cgi?client=1&gameid=145628&role=v&side=w
There is no proof that the crowed played an optimum (even when it won the game).

On the other hand I´m quite sure that the latest bots find very different moves for that game as well as top players would. But this is no proof that the game was far from perfect.

In the case of chess the horizon for silicon intelligence is much wider than for Arimaa. It isn´t yet possible to search such deeply in the position and probably it´ll be never possible. The playing versa a bot like "walking in the minefield" as stated by Kasparow may be still far away (even when it´s already true in some endgame positions).

So, finally, it´s probably hard job to find even a single perfect move. ;)

Title: Re: How far away from "perfect play"?
Post by Ail on Feb 23^rd, 2014, 12:42pm

I actually meant something completely different.

I wanted to know how much of the potential to become better still resides in this game.

As in: If we can't agree about what moves are the best, then there must be massive potential to improve.

For chess we think that perfect play would lead to a draw. This seems to be true when looking at top-engine-games nowadays. They draw very, very often. In Arimaa there are no draws so it cannot be taken to conclude the improving-potential.

Title: Re: How far away from "perfect play"?
Post by lightvector on Feb 23^rd, 2014, 7:48pm

Random fact: With some simple machine learning, it's possible to exactly predict about 14% of the moves in games by players above 2200 over the last few years. I have some code that achieves this, it was the result of the first half of my undergraduate thesis research.

Most of the value of that number, however, has less to do with the strength of the moves by the predictor and more to do with the frequency with which forced or nearly forced moves occur in Arimaa. For example, during a trade, if you're the second player to capture, there's often nothing better to do than execute the capture, particularly if failing to capture would immediately allow the opponent to save his piece and win something for nothing rather than trading it.

Title: Re: How far away from "perfect play"?
Post by jdb on Feb 24^th, 2014, 8:20am

I could set up clueless to use the tablebases and the start method from the endless endgame event. We could start with a (difficult) position that is a win for the player, and see if they can play perfectly enough to win it.

If there is any interest, I can set this up.

Title: Re: How far away from "perfect play"?
Post by chessandgo on Feb 24^th, 2014, 4:31pm

Interesting.

Jdb, I'm sure browni, clyring and some other EEE regulars would love to do that. I would certainly have a go at it given the chance.

Title: Re: How far away from "perfect play"?
Post by browni3141 on Feb 24^th, 2014, 5:00pm

I would try what jdb proposed. It sounds fun.

Also, my estimate for the percentage of realistic positions where chessandgo and I would play the same move at a slow time control is maybe 20%-30%, but this is basically a wild guess.
I chose chessandgo because I think he might be the player with which I agree most often.

Title: Re: How far away from "perfect play"?
Post by jdb on Feb 24^th, 2014, 6:02pm

I'll do my best to set it up this week.

There is a problem, I no longer have the DCR_dcr tablebase. I copied it to an external hard drive. For some reason the file got cut off at 4GB. Most annoying. There is a slight chance I can find another copy lying around, but not likely.

Best I can do is set up some 5 man positions.

Title: Re: How far away from "perfect play"?
Post by Ail on Feb 25^th, 2014, 5:48am

@browni:
That is kind of the statement I was interested in.
According to my theory a 20-30% professionals-agreeing rate means, that we are still quite far away from knowing what perfect play even could look like.

In my book this is a great thing as it shows that there is so much more potential of improvement in this amazing game!

Title: Re: How far away from "perfect play"?
Post by Kushiel on Sep 15^th, 2014, 3:41pm

Counting "how many times they would have played the same move" is probably not a good method for determining if a play was perfect.

If you ask different experts which move they'd make for a given board position, then compare answers, they might say different things even for completely solved games.

Take Tic-Tac-Toe for example. Ask 3 different experts what they'd do on the first move, and you could very well get 3 different answers, since all of the starting positions lead to a draw with perfect play. Ignoring symmetries, only 3 starting moves exist: edge, corner, and center. All are draws.

Asking if experts agree that a move was in the set of "perfect moves" is also not theoretically sound, but may offer practically better results for Arimaa.