Arimaa Forum - Print Page


    
      
        Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Bot Development >> Old Bot Newly Tuned
        
(Message started by: Kushiel on Oct 11^th, 2014, 9:21am)

Title: Old Bot Newly Tuned
Post by Kushiel on Oct 11^th, 2014, 9:21am

As you may know, I'm using bot_akimot as the basis for my bot (which has yet to be named).

An interesting thing about bot_akimot is that the numbers it uses in it's evaluation function and UCT step generation were picked by hand. The original author didn't have time to optimize them before his thesis was published.
The good news is, all these values are stored off in a config file, so changing them around doesn't require recompiling the program, just supplying a different config file.

I'm currently using an evolutionary algorithm to optimize the config file for the original bot_akimot. My original motivation was because later I'd like to measure my improvement over bot_akimot. I don't think it would be fair to say I improved over bot_akimot if I made an engine which was nearly the same, but picked better numbers off the top of my head as parameters. I'd much prefer to see a tuned version of my modified engine play against a tuned version of his engine.

I read in the author's paper that "[bot_akimot] was able to achieve rating around 1600 with peak in history at 1733"
I'm now curious what rating it would achieve sans any other modifications, with a tuned config file, especially since it seems to be the only monte carlo based bot (I'm a huge fan of monte carlo) and as such it represents the peak of what monte carlo has achieved in arimaa.

What has been your experience with tuning? Do most bots use hand tuned parameters, or evolutionary algorithmically tuned parameters, or something else? I know I'm comparing apples to oranges because there are no parameters which guide pseudorandom playouts in alpha-beta. However I'm still curious, gut reaction, do you think this bot would play much better with a better-tuned config file?

Title: Re: Old Bot Newly Tuned
Post by Fritzlein on Oct 12^th, 2014, 11:14pm

What is the source of feedback to your evolutionary algorithm? If I remember correctly, bot_haizhi used self-play with wins and losses propagating back to the evaluation function parameters. It didn't work out well.

It is somehow "less evolutionary" to have the feedback come from expert human games in the database than from self-play, because for the former the immediate goal is to mimic strong humans rather than to surpass them, and the data set can't be expanded at will. Nevertheless, I believe bot_sharp's move ordering and late move reductions at the root come from a function trained on expert games.

I get the sense that hand-picked parameters still rule, and that self-play is used almost exclusively to test specific human-determined changes. I would love to hear otherwise from any developers who are using self-play for automated evolution.

Title: Re: Old Bot Newly Tuned
Post by Migi on Oct 13^th, 2014, 4:59am

I believe that if you implement it right, an evolutionary algorithm will certainly be able to improve the performance of bot_akimot significantly. However, keep in mind that an evolutionary algorithm itself has parameters, such as the population size, the amount of mutation, etc. How much the bot will improve (or deteriorate) will depend a lot on how you choose these parameters, and also on how room for improvement there is in the first place. I don't know how much effort tomik put into tweaking bot_akimot.

I know that in chess, the Stockfish engine used a basic self-play method (https://chessprogramming.wikispaces.com/Stockfish%27s+Tuning+Method) to tune its evaluation function, and was able to gain 40-70 ELO points. An evolutionary algorithm is basically a more advanced version of that method, so it should be able to get even better results.

It's hard to give you a number on how much elo bot_akimot could gain by tuning its config file, but I will say that if you manage to win the challenge only by tweaking the values in the config file, I would be very, very surprised. ;)

Title: Re: Old Bot Newly Tuned
Post by Swynndla on Oct 13^th, 2014, 5:14pm

I've played around with evolutionary algorithms, but not with arimaa bots ... but I also would be interested to hear how you get on. Evolutionary algorithms are slow to converge, but the advantages are, EA's are fairly easy to understand and program, and they are so general that it doesn't matter how complex the domain is (eg non-linear etc) it'll still work (if there's any gains to be made), and it'll converge towards optima even if you throw *lots* of variables at it.