Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Events >> 2008 Computer Champioship
        
(Message started by: omar on Mar 2nd, 2008, 1:09am)
Title: 2008 Computer Champioship
Post by omar on Mar 2nd, 2008, 1:09am
I think I may have to delay the start of the Computer Championship a bit. When I was checking to see if the bots were ready I found that bot_Zombie had not been uploaded at all.  Evan had indicated he wanted to enter bot_Zombie and had paid the registration fee. I'm afraid that maybe he did not get my email with the account info to upload his bot. I've been trying to contact him, but from the lastest posting in  his blog I think he is off sking this weekend. Karl happened to have a copy of Zombie and we are going to try and upload that.

I don't mind delaying the start of the games a bit. I would much rather have all the bots that entered be able to play.
Title: Re: 2008 Computer Champioship
Post by Janzert on Mar 2nd, 2008, 6:23am
Thanks for the note Omar. I hope you're able to make contact.

Janzert
Title: Re: 2008 Computer Champioship
Post by IdahoEv on Mar 2nd, 2008, 11:33am
Omar contacted me and I uploaded Zombie right away.   I haven't touched the code in a long time so I hope I have it configured right, but it seems to be playing okay.

I was simply overloaded with travel (I'm posting this from a remote cabin in the mountains of Idaho) and it slipped my mind.   But everything is up now, so game on!
Title: Re: 2008 Computer Champioship
Post by omar on Mar 2nd, 2008, 6:26pm
Thanks Evan, I checked out Zombie and it seems to be running fine.

I should have been able to start the bot tournament today, but there are a couple of other things which are causing delays. Both of which are my fault.

First, I needed to give the names of the 3 players that will be defending the Arimaa challenge to the WCC TD before the start of the tournament. I was a little late in contacting players to see if they would be interested. At present I have two confirmations and waiting to hear back from the third player.

Second I forgot to order the second server until yesterday. The hosting company has not set it up yet and it will take me about a day after it is setup to get it configured and ready for the tournament.

I apologize to the bot developers and spectators for the delay. However, I think I should still be able to get the tournament finished by the March 15th date as planned.
Title: Re: 2008 Computer Champioship
Post by arimaa_master on Mar 3rd, 2008, 5:28am

on 03/02/08 at 18:26:31, omar wrote:
Thanks Evan, I checked out Zombie and it seems to be running fine.

I should have been able to start the bot tournament today, but there are a couple of other things which are causing delays. Both of which are my fault.

First, I needed to give the names of the 3 players that will be defending the Arimaa challenge to the WCC TD before the start of the tournament. I was a little late in contacting players to see if they would be interested. At present I have two confirmations and waiting to hear back from the third player.

Second I forgot to order the second server until yesterday. The hosting company has not set it up yet and it will take me about a day after it is setup to get it configured and ready for the tournament.

I apologize to the bot developers and spectators for the delay. However, I think I should still be able to get the tournament finished by the March 15th date as planned.


As for the spectators side: donīt worry about it too much. We should be happy that you found the time to let this tournament happen :).
Title: Re: 2008 Computer Champioship
Post by omar on Mar 5th, 2008, 12:53am
I am happy to announce that the second server was setup on Wednesday and I was able to get it configured and ready by the end of the day. Thus the games can begin tomorrow.

I found that David Fotland had not upload his bot (bot_bomb). He said he was really busy at work and did not get to work on it as much as he wanted to. I substituted Bomb2005CC in its place. Since David registered properly and is the developer of Bomb2005CC he will still receive any prizes earned by this bot.

Looking forward to an exciting tournament. Good luck to all the developers.
Title: Re: 2008 Computer Champioship
Post by RonWeasley on Mar 5th, 2008, 7:39am
I think this is the first year Bomb is not the top seed.  Challenge defenders beware.
Title: Re: 2008 Computer Champioship
Post by RonWeasley on Mar 5th, 2008, 3:27pm
Sorry about the jinx, Janzert.  They never work when I want them to.

But as soon as I mention seeding, the top seed OpFor gets upset by the bottom seed sharp.  Congratulations to bot_sharp!
Title: Re: 2008 Computer Champioship
Post by Fritzlein on Mar 5th, 2008, 4:45pm
Round 2 pairing appears to have the colors wrong.  OpFor vs. Zombie should be Zombie vs. OpFor, since Opfor was Gold in the first round while Zombie was silver in the first round.

I recall from last year that there was a color-assignment bug in the pairing program.  This has apparently not been resolved, which requires hand-verification of colors each round.
Title: Re: 2008 Computer Champioship
Post by lightvector on Mar 5th, 2008, 4:47pm
Hooray, my bot isn't completely useless! ;D

Anyways, I have to laugh at all the games played so far. All three games look absolutely alien compared to the human WC games. Particularly the way all the bots are launching into undefended capturing races against each other right at the beginning.

Do humans avoid these lines because the outcomes can be so widely variable? Or is it just more profitable usually to defend and try to take hostages and frames? It would be great if stronger players could comment on how unsound/sound these lines really are.

Anyways, it's really quite amusing to see the differences.
Title: Re: 2008 Computer Champioship
Post by omar on Mar 5th, 2008, 7:06pm

on 03/05/08 at 16:45:33, Fritzlein wrote:
Round 2 pairing appears to have the colors wrong.  OpFor vs. Zombie should be Zombie vs. OpFor, since Opfor was Gold in the first round while Zombie was silver in the first round.

I recall from last year that there was a color-assignment bug in the pairing program.  This has apparently not been resolved, which requires hand-verification of colors each round.


Thanks for noticing that. I've manually changed the color assignment for this game.

Although we had a couple of technical problems today the first round of the computer tournament is completed. Hopefully we won't have any more issues.
Title: Re: 2008 Computer Champioship
Post by RonWeasley on Mar 5th, 2008, 8:38pm
lightvector, muggles do occasionally play a capture race game.

If memory serves, I think jdb used to do this, with success, and then shoot a rabbit through the hole.  I think such a game takes away an advantage a strong player may have since a player becomes strong by learning how to coordinate many pieces to steadily increase a sustainable advantage through multiple threats at different areas of the board.  The capture race removes pieces and makes the goal race more chaotic, so the strong player takes a risk by playing that kind of game.

I would avoid a capture race in a fast game against a good bot because of its accurate goal calculations.  And NEVER try it against a tactical wizard like chessandgo.
Title: Re: 2008 Computer Champioship
Post by Fritzlein on Mar 5th, 2008, 10:19pm

on 03/05/08 at 16:47:34, lightvector wrote:
Hooray, my bot isn't completely useless! ;D

Congratulations!


Quote:
...the way all the bots are launching into undefended capturing races against each other right at the beginning.

Do humans avoid these lines because the outcomes can be so widely variable? Or is it just more profitable usually to defend and try to take hostages and frames?

I think the key factor in the rarity of races among strong players is that it takes two to tango.  There is no race unless both players want to race.  Defense is not always stronger than offense, but it is usually a viable option for either player.  Whenever both players consent to a goal race, or even a capture race, the results for the loser are catastrophic, which means that the loser would not have even entered the race if he had had good judgment.  If both of the players have good judgment, then you won't see either of them throw away the game by entering a losing race.

In the endgame there are more situations in which defense can become a losing proposition.  You might not be able to stop all of the opposing threats, and therefore be compelled to generate threats of your own.  That might be a time when someone with good judgment would enter an unfavorable race, because the race at least offers more hope than a doomed defense.

In the opening, however, there is a sort of equilibrium between offense and defense.  The power of your attack depends on how many pieces you advance and how recklessly.  If you are bound and determined to be the attacker rather than the defender, you can raise the stakes to the point that the other person has to break off what they were doing to defend, or else lose the race.  But in order to make your attack the more powerful one, you have to stick your neck out with a camel advance or something else punishable by proper defense.

One could view the opening as an auction for attacking rights: whoever bids higher gets to be the attacker while the other player defends.  If the attacker bid too high then the defender has the advantage, whereas if the defender gave up his attack at too low a bid, the attacker has the advantage and gets something for nothing.  If the bid was just right, then neither the attacker nor the defender has the advantage, and the players have naturally gravitated to the equilibrium point between attack and defense.

Of course, the tactically superior player can win with an unsound cheeseball attack, even if the weaker player correctly chose to defend.  By the same token the tactically superior player can punish even a sound attack with excellent defense.  This frustrates beginners (e.g. "I lose when I attack with EH and lose when I defend against EH") and this obfuscates the true equilibrium point, but it sure keeps Arimaa interesting and fun!  :)
Title: Re: 2008 Computer Champioship
Post by IdahoEv on Mar 6th, 2008, 1:37am
it's also worth noting that if you roughly model bots as material eval applied to a depth search (which is not far from correct), they have no fear or avoidance of a capture race whatsoever.   A bot "feels" exactly the same way about a camel-for-camel trade as it does about an untouched board.  

The point at which the trading of pieces deviates in favor of one side or the other is way outside the search horizon at the time the race is initiated.   By then, both bots are committed to making the best of the situation as dictated by the board, and constructing a proper defense is out of the question because each side has one decimated trap.
Title: Re: 2008 Computer Champioship
Post by arimaa_master on Mar 7th, 2008, 3:48am

on 03/06/08 at 01:37:54, IdahoEv wrote:
it's also worth noting that if you roughly model bots as material eval applied to a depth search (which is not far from correct), they have no fear or avoidance of a capture race whatsoever.   A bot "feels" exactly the same way about a camel-for-camel trade as it does about an untouched board.  

The point at which the trading of pieces deviates in favor of one side or the other is way outside the search horizon at the time the race is initiated.   By then, both bots are committed to making the best of the situation as dictated by the board, and constructing a proper defense is out of the question because each side has one decimated trap.


I have two notes for two guys :)

1) Omar: It would be nice to add "Event coverage" link to Computer Championship (like is already in World Championship)

2) Adanac (or Fritzlein): here http://katieandkarl.pbwiki.com/2008+Arimaa+Computer+Championship
is comment "Loc vs. Clueless" the same one like in "Bomb vs. Zombie"
Title: Re: 2008 Computer Champioship
Post by Adanac on Mar 7th, 2008, 6:15am

on 03/07/08 at 03:48:57, arimaa_master wrote:
I have two notes for two guys :)

1) Omar: It would be nice to add "Event coverage" link to Computer Championship (like is already in World Championship)

2) Adanac (or Fritzlein): here http://katieandkarl.pbwiki.com/2008+Arimaa+Computer+Championship
is comment "Loc vs. Clueless" the same one like in "Bomb vs. Zombie"


Thanks for noticing that.  I'll have to re-write the Loc-Clueless game since I already saved over the proper Word document summary  :-[
Title: Re: 2008 Computer Champioship
Post by omar on Mar 9th, 2008, 8:07am
I was looking at the logs of the sharp vs Zombie game and it seems that there may have been a technical problem (not a fault of Zombie) which caused Zombie to timeout. I have canceled the bot games for today, until I investigate this some more and track down what happened.
Title: Re: 2008 Computer Champioship
Post by RonWeasley on Mar 9th, 2008, 4:20pm
I spoke with Omar today.  We decided to continue the bot_sharp v bot_Zombie game from the position where the arimaa site interrupted Zombie's move.  The reserve times may be restored a bit generously since the exact reserve times are unavailable.

We considered restarting from the beginning but decided the original moves under tournament conditions should count.

TD
Title: Re: 2008 Computer Champioship
Post by omar on Mar 9th, 2008, 10:49pm
Thanks for posting this Ned. I was able to restore the game and continue it from the same position. bot_Zombie made the same move for 20b that it had tried to send when the game timed out. After this game finishes I will continue the tournament tomorrow with round 5. Will try to setup the games a bit later so that the developers and fans will have a chance to see when the bots play and come to watch the games.
Title: Re: 2008 Computer Champioship
Post by The_Jeh on Mar 10th, 2008, 12:01am
In this game between Zombie and Sharp, Sharp spotted a forced goal in four on move 41w (And it is possible it spotted the forced goal on 40w, because it moved in 33s, although we don't have confirmation for that.) I've added the position as a puzzle. Here's the link:

http://arimaa.com/arimaa/puzzles/show.cgi?p=p37

Perhaps this position is a bit simple, but it's still impressive.

Would someone add it to the wiki for me? Thanks.
Title: Re: 2008 Computer Champioship
Post by lightvector on Mar 10th, 2008, 10:28am
Very interesting.

I have run my sharp on that position on my laptop, and I am quite confident that it does not actually see a forced goal yet. At that point, what it sees is a horse capture and a strong goal threat. It does see the goal on the move after that though.

I looked at my code and discovered that I had implemented a relatively low cap on the number of ply in the search: 16. I'm not quite sure why I set it that low, since it would help a bit to set it higher for positions like this. I'm going to change this in the next version. Anyways, because practically every move other than the moves actually made allows immediate 4-step goal by one side or the other, the goal tree can cut off a HUGE amount of the search tree. So sharp is actually hitting the 16-ply cap at full width in mere seconds (and consequently, seeing up to 20-ply goals using the goal tree). I suppose it also doesn't hurt that the first move is a horse capture, so it is searching that move first.

Just to see how far sharp could search without a depth cap, I ran it again, and apparently it would get to about 19 or 20 ply at full-width under the tournament time controls. I have to say that this position is quite amazing, since I have never seen my bot get that deep before (without being cut off at, say, 12 ply by seeing a goal).

Anyways, you probably want to do some more rigorous analysis to make sure that the position really is goal in 4 moves. I wonder what Bomb has to say. =)
Title: Re: 2008 Computer Champioship
Post by Janzert on Mar 10th, 2008, 11:03am
Ahh, I had thought the quick move in 14 seconds indicated that it saw the goal. Your explanation actually make more sense, even if it is a bit more mundane. ;)

Janzert
Title: Re: 2008 Computer Champioship
Post by The_Jeh on Mar 10th, 2008, 12:06pm
Upon review, I don't think it is goal in four, but I do think it is goal in five. I'll have to make some changes.

[edit] or maybe six? At this point, it is not a puzzle, but just an endgame.
Title: Re: 2008 Computer Champioship
Post by Janzert on Mar 10th, 2008, 7:47pm
Omar, if possible and not a problem for someone else could the round 6 bomb vs opfor game be moved to after 3:30pm EDT?

I won't be able able to spectate till then. If it can't be changed though, not a problem.

Thanks,
Janzert
Title: Re: 2008 Computer Champioship
Post by omar on Mar 10th, 2008, 10:39pm
No problem. Changed time to 4 pm.
Title: Re: 2008 Computer Champioship
Post by Janzert on Mar 10th, 2008, 11:03pm
Thanks, Omar.

Janzert
Title: Re: 2008 Computer Champioship
Post by IdahoEv on Mar 11th, 2008, 3:06am
This is a really interesting time for the  Arimaa AI competition!   All of the top four bots in the CC beat at least one of the other four and lost to at least one ... no bot is the clear leader against all others any longer!

Bomb would still seem to be the leader, except that it lost to OpFor, which in turn lost to Zombie.   Perhaps OpFor has an unusual strategy that exploits a weakness in Bomb, but which is weaker against Zombie and Sharp?  

I suspect some developers will be putting in a few hours this year...  I know I will.
Title: Re: 2008 Computer Champioship
Post by Adanac on Mar 11th, 2008, 7:21am
I'll try to add summaries for the round 5 Computer Championship tonight at katieandkarl.pbwiki.com.  My time will be slightly limited, and game # 72026 had an incredibly difficult endgame.  If anyone has some time today for an analysis:

1.  Did Zombie have a forced win after its 33rd move?  If not, how incredible that those 2 seventh-rank rabbits couldn't be forced north!

2.  Rather than wasting its final step(s) on move 35, did Zombie have anything stronger?

3.  Was it a blunder for sharp to advance a rabbit to h6 on move 39 (Would any human advance a rabbit south in that situation!?).  Or was it a forced win for sharp after 39 moves?

If anyone has time to take a look, that's great!  If not, I'll take a closer look later.  To make my life easier, I'll probably start with a summary of Opfor-Bomb tonight and do Zombie-sharp second  :D
Title: Re: 2008 Computer Champioship
Post by Adanac on Mar 11th, 2008, 10:04pm
After a closer analysis tonight, the Zombie - sharp endgame was definitely a forced win for Zombie.  However, it's not the quick forced goal it appears to be after a quick glance at move 33.  I haven't had time for a complete analysis but I haven't yet found a way for silver to refute 34w Ef4e Eg4n cg6w Eg5n.  Also, the move Zombie played also seems to force a win after a long sequence of moves.

Also, if Zombie had simply captured the cat on move 37, it looks like it would have been an easy victory - sharp can bring its elephant to d6 and camel to g4 but Zombie can capture two silver rabbits on the east side and begin a very powerful gold rabbit advance to h5.  Even though sharp can stop the immediate goal threats, it gives Zombie time to completely dominate the board and force multiple other threats.

I'll write up a summary for this game tomorrow.  I won't go into too much detail but I'll mention that Zombie had a powerful advantage at move 33, but lost it through an inaccurate sequence of moves.
Title: Re: 2008 Computer Champioship
Post by Janzert on Mar 11th, 2008, 10:16pm
Adanac, did you see Fritzlein's comment attached to the game (http://www.arimaa.com/arimaa/gameroom/comments.cgi?gid=72026)? He did some analyses with Bomb looking for a forced goal the other night and couldn't find one.

Janzert
Title: Re: 2008 Computer Champioship
Post by aaaa on Mar 13th, 2008, 11:19am
Congratulations to David Fotland for winning the Arimaa Computer World Championship for the fifth time in a row!
Title: Re: 2008 Computer Champioship
Post by RonWeasley on Mar 13th, 2008, 1:02pm
I'm joining aaaa and others congratulating David Fotland and bot_bomb2005CC for winning the Arimaa Computer World Championship.  We had some new bots this year that specifically targeted bomb as their competition.  Still, bomb continues to be the standard by which we measure all bots.  This bot has great staying power.  Once again, all the other developers are thinking about next year.

Thanx also to the other competitors.  We had some of the most exciting games ever.  We're all looking forward to improved versions in the coming months and a challenging championship next year.

TD
Title: Re: 2008 Computer Champioship
Post by Janzert on Mar 13th, 2008, 3:09pm
I'll add my congratulations to David Fotland as well. Bomb certainly still deserves the title.

I hope we can provide you with some more competition by next year. :)

Janzert
Title: Re: 2008 Computer Champioship
Post by Fritzlein on Mar 13th, 2008, 8:33pm
Every year that Bomb wins again without further improvement makes the initial achievement all the more amazing.  I was impressed with Bomb's prowess already in 2005, but I had no idea what a technical feat it was for a bot to be that good.  When I first started playing in mid-2004, I took Bomb as evidence that Arimaa probably wasn't terribly computer-resistant after all, and Omar's claims were overblown.

As a group we know much more about Arimaa strategy than we did three years ago.  In hindsight it seems clear that Arimaa really is hard for software to tackle, and human approaches to the game have non-trivial strength.  But it is interesting how that additional strategic information hasn't been enough to help other developers unseat the old-fashioned style of Bomb.  Apparently additional human-type knowledge of how to play is less important than engineering excellence encoding the fundamental strategies.

That said, it is impressive that two newcomers finished second and third ahead of last year's second-place finisher.  Kudos to sharp and Opfor as well.  The non-Bomb state of the art does seem to be improving, and we can count on the bar being raised before too long.
Title: Re: 2008 Computer Champioship
Post by IdahoEv on Mar 14th, 2008, 2:23am
My congratulations to the entire field, but particularly to the newcomers Sharp and OpFor!


Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB Đ 2000-2003. All Rights Reserved.