Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Site Discussion >> Bot resistance statistics
        
(Message started by: aaaa on Mar 27th, 2010, 5:30pm)
Title: Bot resistance statistics
Post by aaaa on Mar 27th, 2010, 5:30pm
Partially in order to help with the determination of which bots to put on the ladder, I calculated how many games it typically took a human to "legitimately" beat a bot hosted by the server. Games counted for this purpose had to be rated and end in goal, immobilization, elimination, human time-out or human resignation. For each such bot below, the second column shows a calculated measure of central tendency (close to the median) of how many losses it inflicted on a player before being beaten by him or her. The third column shows the size of the set, i.e. how many humans have managed to beat the bot.
bot_Bomb2005P22.99346153
bot_Bomb2005Blitz2.99048105
bot_Bomb2005Fast2.88596114
bot_GnoBot2009Fast24
bot_Bomb2005P11.93763465
bot_GnoBot2009Blitz1.54
bot_Bomb2005Lightning1.2142914
bot_Clueless2006P11.11168197
bot_GnoBot2005Blitz1.10526114
bot_Clueless2005Fast1.0886179
bot_GnoBot2005Fast1.07333150
bot_Clueless2007Fast1.0602483
bot_OpFor2008Blitz1.0560
bot_Aamira2006Blitz1.04698149
bot_Clueless2005P11.04583240
bot_Clueless2007Blitz1.0151566
bot_Zombie2008Fast0.98969197
bot_Zombie2008Blitz0.98571470
bot_Clueless2005Blitz0.96969766
bot_GnoBot2005P20.962547267
bot_OpFor2008Fast0.95774671
bot_Clueless2005P20.928125
bot_Arimaazilla0.893466704
bot_Clueless2007P20.88297994
bot_Aamira2006Fast0.863388183
bot_Loc2007Blitz0.857143112
bot_Clueless2007P10.835366164
bot_Aamira2006P20.831382427
bot_Clueless2006P20.829787141
bot_Arimaazon0.582
bot_Clueless2005CC0.540
bot_Clueless2009Blitz0.512
bot_OpFor2009Fast0.56
bot_OpFor2008P10.488323471
bot_Loc2007Fast0.478571140
bot_GnoBot2006Blitz0.47222236
bot_Clueless2006Fast0.46511686
bot_Loc2007P20.463768207
bot_Clueless2009Fast0.46153813
bot_Loc2006P20.460674178
bot_Sharp2008P20.457627295
bot_Bomb2005CC0.45238142
bot_GnoBot2006P20.44578383
bot_Loc2006P10.435337549
bot_Loc2007P10.433697641
bot_Loc2005Fast0.4329997
bot_Bomb2004CC0.42857114
bot_GnoBot2006Fast0.42222245
bot_GnoBot2006CC0.40909122
bot_OpFor2009Blitz0.45
bot_Loc2005Blitz0.39393966
bot_Loc2006Blitz0.38095263
bot_Clueless2008Fast0.3758
bot_GnoBot2005P10.3746011252
bot_OpFor2008P20.364238151
bot_Loc2005P10.358429611
bot_Loc2005P20.358079229
bot_Bomb2005P30.3333333
bot_Loc2006Fast0.30769291
bot_Clueless2009P10.2857147
bot_OpFor2008CC0.2857147
bot_ShallowBlue0.2765541544
bot_Arimaalon0.273764789
bot_Sharp2008P10.272311874
bot_ArimaaScoreP20.2689411993
bot_Aamira2006P10.2540071622
bot_Aamira2006CC0.2544
bot_Clueless2009P20.2512
bot_GnoBot2006P10.246201329
bot_Loc2005CC0.21568651
bot_GnoBot2005CC0.245
bot_ArimaaScoreP10.1766013154
bot_Clueless2008P10.17647117
bot_Clueless2006Blitz0.1666676
bot_Clueless2006CC0.13953543
bot_Loc2006CC0.12548
bot_GnoBot2004CC0.062516
bot_ArimaaScoreP305
bot_Clueless2007CC02
bot_Clueless2008Blitz04
bot_Clueless2008CC02
bot_Clueless2008P204
bot_Clueless2009CC07
bot_Loc2008CC03
bot_Occam2004CC07
bot_OpFor2009CC01
bot_Rat2009Blitz01
bot_Rat2009CC01
bot_Rat2009P101
bot_Rat2009P201
bot_Sharp2008CC05
bot_Zombie2008CC01
Title: Re: Bot resistance statistics
Post by FireBorn on Mar 28th, 2010, 1:21am
wow, really? I'm at the top of the ladder and I'm playing bots like 6-7 times before winning :(
Title: Re: Bot resistance statistics
Post by amalgam on Mar 28th, 2010, 2:31am
Don't feel too bad, FireBorn, it took me 7 tries to beat Bomb2005Fast, and a ridiculous 24 times to beat Bomb2005Blitz!  And the one game I won, I really shouldn't have.

And hopefully, Heyckie doesn't mind me mentioning it here, but he has even more losses against Bomb2005Fast than I do against Bomb2005Blitz.

But anyway, according to this table, how many games should it take the average person to complete the bot ladder?
Title: Re: Bot resistance statistics
Post by 99of9 on Apr 15th, 2010, 4:48pm
Interesting metric, but what's the scoring function when compiling a ladder?  Do you want all the bots to be high on this list?  Or widely distributed?

My gut instinct is that we want a wide distribution of playing styles (and levels).
Title: Re: Bot resistance statistics
Post by Hirocon on Apr 17th, 2010, 3:43pm
I finally beat the bot ladder today.  It took me 6 games to beat bot_Bomb2005Blitz and 17 games to beat bot_Bomb2005Fast.

Almost all of my losses against these bots were resignations, as I have a bad habit of resigning after making blunders or getting behind material.  I really need to break this habit.  In my winning game against bot_Bomb2005Fast, I lost two pieces because of a mis-click, and in my winning game against bot_Bomb2005Blitz, I was way behind materially when I won.  That's why you don't resign!
Title: Re: Bot resistance statistics
Post by omar on Apr 19th, 2010, 9:42am

on 04/17/10 at 15:43:37, Hirocon wrote:
I finally beat the bot ladder today.  It took me 6 games to beat bot_Bomb2005Blitz and 17 games to beat bot_Bomb2005Fast.

Almost all of my losses against these bots were resignations, as I have a bad habit of resigning after making blunders or getting behind material.  I really need to break this habit.  In my winning game against bot_Bomb2005Fast, I lost two pieces because of a mis-click, and in my winning game against bot_Bomb2005Blitz, I was way behind materially when I won.  That's why you don't resign!


Way to go. I just checked out your blitz game. That was quite wild :-)
Title: Re: Bot resistance statistics
Post by omar on Apr 19th, 2010, 9:49am

Quote:
bot_Bomb2005Blitz      2.99048      105


I would have expected this to be much higher. Who are the players that beat this bot only after losing two or less games? They deserve some special recognition :-)
Title: Re: Bot resistance statistics
Post by ocmiente on Apr 19th, 2010, 1:29pm

Quote:
Who are the players that beat this bot only after losing two or less games? They deserve some special recognition


Not sure about that... but I've only played it one time and won  :) http://arimaa.com/arimaa/games/jsShowGame.cgi?gid=128627&s=b  Can't say that I'm proud of it though...

That bot has a weaknes for bait and tackle.  http://arimaa.com/arimaa/mwiki/index.php/Bot_Slaying#Bait_and_Tackle


Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB © 2000-2003. All Rights Reserved.