Arimaa Forum - Print Page


    
      
        Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Bot Development >> Handicap Aware Bots (Bot Bashers for Bots)
        
(Message started by: Kushiel on Sep 15^th, 2014, 2:07pm)

Title: Handicap Aware Bots (Bot Bashers for Bots)
Post by Kushiel on Sep 15^th, 2014, 2:07pm

I believe that playing handicap games would be a good way to measure improvement in bots. I got the idea from the Bot Bashers Challenges, where players would try to beat bots while under a self-imposed handicap (http://arimaa.com/arimaa/mwiki/index.php/Bot_Bashers).

All of the handicaps listed on that page are interesting challenges: Material Handicaps, 3-step Handicap, Immobilization Only Victory, Goal Only Victory, No Capture Immobilization Victory, and any combinations thereof. Winning as fast as possible isn't actually a handicap, but winning in a certain number of moves is.

I'm interested in a Single Handicap Tournament with the following format: Each handicapped bot plays each non-handicapped bot twice, once on each color. Two winners are then crowned: The handicapped bot who did the best against the non-handicapped field, and the non-handicapped bot who did the best against the handicapped field.

This could be generalized to a Double Handicap Tournament in which 2 different handicaps are selected: Handicap A and Handicap B. Each competitor selects a handicap to play under. Each Handicap A bot plays against every Handicap B bot twice, once on each color. The Handicap A bot who performed the best against the Handicap B field would be crowned, as would the Handicap B bot which performed the best against the Handicap A Field.
The Single Handicap Tournament would then be a special case in which one of the Handicaps was "No Handicap."

Is there any interest in such a tournament, or feedback for why it would or wouldn't be a worthwhile endeavor to set it up?

I feel it could be a learning/proving grounds for new developers to try to tackle intelligent opponents with restrictions rather than climbing across stupid and only-slightly-less-stupid bots on their way to becoming actual competitors.

It could also be a proving grounds for highly skilled bots to show just how superior they are to the general population. Especially since current human world champions are able to beat bot WCs with a horse handicap, I feel like any bot that becomes similarly skilled to human world champions should be able to perform similar feats against the current bot population.

Title: Re: Handicap Aware Bots (Bot Bashers for Bots)
Post by leon_messerschmidt on Sep 16^th, 2014, 5:57am

Interesting idea. Although I think the real value may be limited to bots at the lower or upper extremes of the ladder.

For my own bot that falls somewhere in the middle, there are plenty of bots that I can easily beat and there are plenty of bots that beat me every time.

Title: Re: Handicap Aware Bots (Bot Bashers for Bots)
Post by Kushiel on Sep 16^th, 2014, 7:41am

For the bots that beat you every time, do you ever wonder how close you are to being even with them? Is it 1 horse, 1 dog, 1 cat or even 1 rabbit?

I think a 3-step tournament would mostly be a chance for very low bots to have a chance against a very high bot (as you stated). I mentioned that first because my bot is just getting started, but a 1 rabbit tournament would be appropriate for more moderate bots to attempt to pick off a slightly higher bot.

Also, the higher bots might actually be able to maintain a perfect record in a 1 rabbit tourney, so there's something to strive for there (as opposed to the 3-step tourney where the Handicapped Champ will likely win <20% of games).

Title: Re: Handicap Aware Bots (Bot Bashers for Bots)
Post by aaaa on Sep 16^th, 2014, 9:57am

It's been said that the most natural handicap is a time handicap. Unfortunately, this is not supported by the server, but it could be forced on the side of the client.

Title: Re: Handicap Aware Bots (Bot Bashers for Bots)
Post by Kushiel on Sep 16^th, 2014, 5:48pm

I'd like to distinguish between "soft" and "hard" handicaps (If anyone knows of better existing terms let me know).

Hard handicaps change the game tree itself (either through a different starting node or by eliminating some branches) and thus change the optimal sequence of moves through a game.

Soft handicaps (including time constraints) make finding the optimal move(s) more difficult, but do not change which moves are optimal.

I'd agree that soft handicaps are more natural, but I'd still prefer to play in hard handicap games.

I associate being able to win hard handicap games with being able to convert games which are theoretical wins into practical wins (or converting a theoretical loss into a practical win). This association is natural from my definition. You can reach theoretical win positions in a natural game from a starting even position, and being able to convert them is a valuable ability. You may also end up in a theoretical loss, and finding a way to overcome it is also valuable.

I'm not sure what to associate soft handicaps with, other than being able to beat a weaker opponent. You won't ever reach a situation equivalent to a soft handicap through natural play (your opponent won't get weaker over the course of the game unless time really does become a factor, which I understand is rare in arimaa).

Title: Re: Handicap Aware Bots (Bot Bashers for Bots)
Post by Fritzlein on Sep 16^th, 2014, 9:03pm

A time handicap is perfectly natural, equivalent to playing against someone who is simply better (or worse) at the game than you are. Just like a material handicap, the intent is usually for a time handicap to be equal and opposite to the skill difference, but a time handicap encourages play closer to what would occur between people of equal skill in a non-handicap game.

In a game with a large material handicap, a comeback is impossible without serious mistakes by the opponent, so one plays in a way that relies on blunders, or even attempts to induce them. Human material handicap wins over bots typically rely on effectively exploiting known weaknesses of the bots.

Conversely, if I am given a material handicap, I can use strategies that aren't normally available. For example, if my opponent has no camel, I can attack with a horse on each wing from the get-go. Normally I would have to fear the opposing camel taking one of my horses hostage, but I don't mind the opposing elephant taking a horse hostage, since that leaves my camel as the strongest free piece. (I guess the same would be true if my opponent blundered a camel in an even game, but frankly, I lose interest in a game if my opponent blunders a camel in the opening. Being able to convert the win is a valuable skill, but it is a skill that I already have almost all of from being able to win an even game; it's not really worthy of special study.)

In a time handicap game, in contrast, one can win by playing good moves and hoping the opponent plays moves that are slightly less good. This is true whether giving or receiving the handicap. To me that feels much more like what I do in every normal game of Arimaa. I therefore like all the time-handicap bots (a.k.a. P1 and P2) in the ladder.

Title: Re: Handicap Aware Bots (Bot Bashers for Bots)
Post by Kushiel on Sep 18^th, 2014, 7:57am

I find it's interesting that we agree on the characteristic differences between time and material handicaps, but it leads you to prefer time handicaps while it leads me to be more interested in material handicaps.

Since it's a matter of opinion which kind of handicap you'd like to play (or have your bot play), I think a simple poll is more appropriate than debate at this point.

If you'd be interested in playing in a Handicap Tournament, which handicaps would you be most interested in? Please answer specifically what handicaps you'd like e.g., "3 min/move vs. 1 min/move" or "two rabbit handicap."

If there's enough interest, I'd like to try extending the Arimaa Engine Interface to allow engines to agree upon handicaps, or at the least pass chats back and forth through the controller (with handicap negotiations then being performed within the chat protocol). The Arimaa Game Notation already includes chats, so it could be conveniently attached at the beginning of every game exactly what handicaps it was being played under.