Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Events >> 2011 Computer Championship
        
(Message started by: omar on Jan 3rd, 2011, 7:33pm)
Title: 2011 Computer Championship
Post by omar on Jan 3rd, 2011, 7:33pm
The qualifying period for bots is underway during the month of January. Bot developers should have their bots play against the benchmark bots.

Please keep in mind that when event games are in progress the bots on the server will be turned off. Start early so that you have enough time to complete all the games.

Also please try not to start more than one CC or Fast bot so that the server is not overloaded. If someone already has a CC or Fast bot running, don't start another CC or Fast bot.

Since there are 8 bots entered this year all bots will be in the WCC tournament and the qualifying period will determine the seeding for the bots. Best of luck.
Title: Re: 2011 Computer Championship
Post by rbarreira on Jan 3rd, 2011, 8:09pm

on 01/03/11 at 19:33:58, omar wrote:
Please keep in mind that when event games are in progress the bots on the server will be turned off. Start early so that you have enough time to complete all the games.

Also please try not to start more than one CC or Fast bot so that the server is not overloaded. If someone already has a CC or Fast bot running, don't start another CC or Fast bot.


I will try to follow this requirement, but it may be hard to get many games done if only one CC/Fast bot can be running. Very often there are people playing these bots, especially now that some are trying to beat the advanced bot ladder.
Title: Re: 2011 Computer Championship
Post by omar on Jan 5th, 2011, 12:29am

on 01/03/11 at 20:09:10, rbarreira wrote:
I will try to follow this requirement, but it may be hard to get many games done if only one CC/Fast bot can be running. Very often there are people playing these bots, especially now that some are trying to beat the advanced bot ladder.


Yes, this is going to be a bit problematic. I also have to run the qualifying games for the bots which have not changed since last year.
Title: Re: 2011 Computer Championship
Post by rbarreira on Jan 5th, 2011, 6:03am
http://arimaa.com/arimaa/wcc/2011/qual.htm

This is a nice link to see the qualification status. It seems to be missing a game between bot_marwin and bot_Sharp2010Blitz though (or maybe it's not updated very often?).
Title: Re: 2011 Computer Championship
Post by Fritzlein on Jan 5th, 2011, 8:39am
It's very considerate behavior to get in your qualifying games early.  Last year we found that there wasn't enough incentive to start early, compared to the hope that some last-minute improvements would strengthen the bot and increase its score for starting later.  Hopefully, even though the incentives seem tilted towards a later start, enough developers will be good citizens and start qualifying early to avoid a last-minute rush and the consequent server overload.
Title: Re: 2011 Computer Championship
Post by omar on Jan 5th, 2011, 11:35am

on 01/05/11 at 06:03:59, rbarreira wrote:
http://arimaa.com/arimaa/wcc/2011/qual.htm

This is a nice link to see the qualification status. It seems to be missing a game between bot_marwin and bot_Sharp2010Blitz though (or maybe it's not updated very often?).


It is updated once an hour.
Title: Re: 2011 Computer Championship
Post by rbarreira on Jan 22nd, 2011, 9:22am
I was wondering if the Computer Championship games will have the two-move delay as the human event games have. It would be nice to not have it if possible.

I don't think anyone will implement a speech recognizing/natural language interpreter in their bot in order to cheat, at least not anytime soon ;)
Title: Re: 2011 Computer Championship
Post by omar on Jan 27th, 2011, 1:32pm
Good suggestion. I've changed it to only use delays if one of the players is human.
Title: Re: 2011 Computer Championship
Post by Janzert on Feb 8th, 2011, 8:24pm
Omar,

I see in the gameroom that you sent out the login information emails, but I don't seem to have received one yet.

Janzert
Title: Re: 2011 Computer Championship
Post by omar on Feb 9th, 2011, 8:07am

on 02/08/11 at 20:24:46, Janzert wrote:
Omar,

I see in the gameroom that you sent out the login information emails, but I don't seem to have received one yet.

Janzert


Yes, I had sent a message to all the bot developers about a week ago.

I've sent the message again through the forum private message system.

If anyone else did not get it, please let me know.
Title: Re: 2011 Computer Championship
Post by foggy on Feb 11th, 2011, 3:54am
It seems that developers allready have granted access to some machine to setup the bots. But there is no information on Arimaa page or forum about hardware that will be used for championship. If this information is already known, it would be interesting to see it. If hardware is not known yet, bot setup on other machine probably is redundant.
Title: Re: 2011 Computer Championship
Post by rbarreira on Feb 11th, 2011, 4:23am

on 02/11/11 at 03:54:28, foggy wrote:
It seems that developers allready have granted access to some machine to setup the bots. But there is no information on Arimaa page or forum about hardware that will be used for championship. If this information is already known, it would be interesting to see it. If hardware is not known yet, bot setup on other machine probably is redundant.


Since I have a login at the server, I checked the hardware: X3360 CPU @ 2.83 GHz, and 4 GB of RAM. This means higher RAM and an equal CPU compared to last year (the X3360 is a server version of the Q9550), so it should be pretty much the same to setup as last year.
Title: Re: 2011 Computer Championship
Post by omar on Feb 12th, 2011, 7:51pm
Thanks for the reminder. I've updated the hardware info.

http://arimaa.com/arimaa/challenge/2011/hardware.html
Title: Re: 2011 Computer Championship
Post by rbarreira on Feb 18th, 2011, 3:34am

on 02/12/11 at 19:51:43, omar wrote:
Thanks for the reminder. I've updated the hardware info.

http://arimaa.com/arimaa/challenge/2010/hardware.html


I believe you meant:

http://arimaa.com/arimaa/challenge/2011/hardware.html

I saw jdb and Janzert in the chatroom confused on whether the server had 2 or 4 GB of RAM, I'm guessing they didn't notice the link was for last year's hardware.
Title: Re: 2011 Computer Championship
Post by Janzert on Feb 18th, 2011, 8:35am
ahh, yes I was indeed looking at the hardware for last year. Although I managed to get there, not by following Omar's link above, but by using the first "hardware" link in this year's tournament rules page. Obviously without reading either the sentence it was a part of or the page title once loaded.

Janzert
Title: Re: 2011 Computer Championship
Post by omar on Feb 18th, 2011, 12:30pm
Oops, copied the wrong link. Fixed it now. Also fixed on the WCC rules page. Thanks.
Title: Re: 2011 Computer Championship
Post by Fritzlein on Feb 28th, 2011, 11:33pm

on 01/27/11 at 13:32:07, omar wrote:
Good suggestion. I've changed it to only use delays if one of the players is human.

What about the disabling of over-1500-rated server bots during event games?  Will that be in force for the WCC, or only in event games when one of the players is human?  My memory of what a hassle it was to have server problems during one past WCC leads me to suggest not taking any chances with overload, even if it means server bots being disabled for half of the day all week long.  I will see if I make bot_ArimaanatorFast available during some of that time.
Title: Re: 2011 Computer Championship
Post by foggy on Mar 1st, 2011, 12:59pm
Do we have some place to see the CC2011 results and schedule? I've heard about the result (marwin-clueless) in chatroom, but search for results/games was not successful.
Title: Re: 2011 Computer Championship
Post by rbarreira on Mar 1st, 2011, 1:21pm

on 03/01/11 at 12:59:30, foggy wrote:
Do we have some place to see the CC2011 results and schedule? I've heard about the result (marwin-clueless) in chatroom, but search for results/games was not successful.


http://arimaa.com/arimaa/wcc/2011/showGames.cgi
Title: Re: 2011 Computer Championship
Post by omar on Mar 1st, 2011, 7:54pm
I contacted the TD today regarding a timeout we had in the bot_clueless vs bot_marwin game.


Quote:
Hi Ned,

The bot_clueless vs bot_marwin game ended with clueless losing on time. It was
also losing on the board so it did not have an adverse effect. An investigation
of the problem found nothing wrong with network. However the gold.arimaa.com
server seemed to be running very slow. An investigation of that found that
clueless itself had hogged too much memory and operating system did not have
enough memory for swapping and began using the disk which usually causes it to
run much slower. So the fault was of clueless and not the server. To prevent
this from happening in future games, Jeff Bacher would like to change the
setting for clueless to use a smaller hash table. He has asked me if I can make
the change. The rules state:

Once the programs have been submitted for the tournament, they may not be
modified manually, but can modify themselves if it can be done as an automated
process.

So I would like to get your decision on this matter.

Omar



Quote:
Omar,

I'm going to let jdb make this change.

This was a really hard decision.  I want to have the intended version of Clueless compete, but the letter of the rule does not allow it.  So I compared this situation to others, in my imagination, where a control parameter, like maxSearchDepth, was set wrong at submission.  I would not let a programmer change the move selection algorithm, but where does the algorithm end and adaptation to the run-time environment begin?  Well, I decided there is a boundary.  I'm having trouble quantifying it, but I'm going to use intuition and decide that hash table size is adaptation to run-time environment, which I'm going to allow for everybody on a case-by-case basis.

I realize that one can make the case that managing run-time environment resources is part of AI.  That's not the part of AI I'm interested in.  I'm also worried about somebody intentionally planning to adapt a parameter in their bot based on it's play in early rounds.  I would not like to allow that.  The difference is it being intentional.

TD


I edited the clueless.cfg file to set:
number_of_hash_table_entries = 10000000
Previously it was set to 30000000.
Title: Re: 2011 Computer Championship
Post by Fritzlein on Mar 1st, 2011, 9:31pm

Quote:
I'm going to use intuition and decide that hash table size is adaptation to run-time environment, which I'm going to allow for everybody on a case-by-case basis.

That's how I would feel about it.  There is a meaningful distinction between the engine per se and its manifestation on a particular piece of hardware.
Title: Re: 2011 Computer Championship
Post by Janzert on Mar 1st, 2011, 10:47pm
So just as a note of curiosity there are actually two equally valid pairings for round two. The pairing actual chosen by the current pairing program is:
Marwin - Briareus
Sharp - Bomb
Opfor - Clueless

There is also an alternate pairing that carries the same weight:
Sharp - Marwin
Opfor - Briareus
Clueless - Bomb

Janzert
Title: Re: 2011 Computer Championship
Post by chessandgo on Mar 10th, 2011, 3:08am
If I understand correctly, the final CC game will take place today at 5pm CET, so I'm available to comment, and Karl as well apparently.

I think it would be great if people from the bot community could jump in the studio whenever they want during the commentary, to tell us about their own bot, how aggressively they prune / how far they search / what tricks they use / etc, about sharp and marwin, what kind of positions are better suited to one or the other, about how to interpret things like bots playing fast / slowly on particular moves, how their own bot or hope copies evaluate the position / the move(s) they suggest, and so on. So guys, please bring your mics along! :)
Title: Re: 2011 Computer Championship
Post by Fritzlein on May 7th, 2011, 6:00pm
Congrats, lightvector, on the $119 first place prize, plus the $92 game winner prize... and the $4,000 bonus prize!

http://www.thecrimson.com/article/2011/5/7/house-hoopes-prize-award/
Title: Re: 2011 Computer Championship
Post by JimmSlimm on May 7th, 2011, 8:13pm
wow congratz lightvector :)
Title: Re: 2011 Computer Championship
Post by Janzert on May 7th, 2011, 9:58pm
Very nice, congratulations.

Janzert
Title: Re: 2011 Computer Championship
Post by 99of9 on May 7th, 2011, 11:01pm
Congratulations!  Is the thesis available somewhere?  I can't wait to read it.
Title: Re: 2011 Computer Championship
Post by lightvector on May 8th, 2011, 1:03am
Thanks!

I've *just* finished exams, yay. That concludes the single busiest semester in my undergrad education (and the only semester in which I've had to pull multiple all-nighters).

I don't know if it will get posted online by Harvard somewhere, but if not, then at some point in the next week or so, I'll fix a few typos and some errors that I noticed and post it up myself.
Title: Re: 2011 Computer Championship
Post by rabbits on May 8th, 2011, 7:53pm
Congrats Lightvector!  That's awesome!
Title: Re: 2011 Computer Championship
Post by UruramTururam on May 9th, 2011, 3:38am
Wow, that's an achievement.

I'd also like to read your thesis once it's available.
Title: Re: 2011 Computer Championship
Post by omar on May 14th, 2011, 7:00pm
That's totally awesome David, congrats. Looking forward to seeing your thesis.


Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB © 2000-2003. All Rights Reserved.