Author |
Topic: Will the 2010 Computer Championship be open? (Read 13770 times) |
|
Fritzlein
Forum Guru
Arimaa player #706
Gender: 
Posts: 5928
|
 |
Re: Will the 2010 Computer Championship be open?Or
« Reply #105 on: Jan 8th, 2010, 6:10pm » |
 Quote  Modify
|
on Jan 8th, 2010, 5:44pm, BlackKnight wrote:| ...but I will introduce Arimaa to two of my classes as we go along. |
|
Awesome!
|
|
 IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Will the 2010 Computer Championship be open?
« Reply #106 on: Jan 9th, 2010, 11:37pm » |
Quote Modify
|
on Jan 7th, 2010, 9:01am, Fritzlein wrote:
In some sense bomb is getting an automatic entry, because Omar is doing work on Fotland's behalf that other developers don't have to do. I don't like the possiblity that there is an actively-developed bot getting squeezed out. I would prefer the eighth spot in the tournament to go to Rat or akimot because their respective developers have put in time in the past year, even if bomb is stronger than both Rat and akimot. I truly hope that neither developer was deterred from entering by the thought that the field was too tough. I hope that neither was going to take up the eighth space in bomb's absence.
|
|
It was only recently that Fotland mentioned to me that he didn't get a chance to work on Bomb. If there had been more than 8 bots I would not have run the qualifying games for Bomb and it would get most likely get excluded. Initially I was not planning to run the qualifying games for Bomb and let it get seeded in the last position. But then I thought it's not that much work for me to run those games and it would not mess up the seeding.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Will the 2010 Computer Championship be open?Or
« Reply #107 on: Jan 9th, 2010, 11:48pm » |
Quote Modify
|
on Jan 8th, 2010, 5:44pm, BlackKnight wrote:
I have an even much busier schedule this term, but I will introduce Arimaa to two of my classes as we go along. |
|
I've been contemplating sending out a flyer about Arimaa and the Arimaa challenge to all the professors who teach an Intro to AI class. It would be a lot of work though trying to compile such a list.
|
| « Last Edit: Jan 10th, 2010, 12:40pm by omar » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Will the 2010 Computer Championship be open?
« Reply #108 on: Jan 10th, 2010, 8:53am » |
Quote Modify
|
on Jan 9th, 2010, 11:37pm, omar wrote:| It was only recently that Fotland mentioned to me that he didn't get a chance to work on Bomb. If there had been more than 8 bots I would not have run the qualifying games for Bomb and it would get most likely get excluded. Initially I was not planning to run the qualifying games for Bomb and let it get seeded in the last position. But then I thought it's not that much work for me to run those games and it would not mess up the seeding. |
|
Given that there were only only seven other participants, it is good to have even an unaltered Bomb taking part. Given that Bomb is taking part, it will be good to have it seeded correctly.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Will the 2010 Computer Championship be open?
« Reply #109 on: Jan 10th, 2010, 8:58am » |
Quote Modify
|
Now that marwin has reached the maximum score of 32, I see more clearly that the second tiebreaker of completion time is actually important. Marwin has only two losses, so marwin will have the top seed unless some other bot gets a four-game winning streak against all the benchmark bots with only one loss total. By going first, tize has secured a large advantage, because anyone coming after needs to score half his losses. Maybe the qualifying procedures this year aren't going to work out so badly after all.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Will the 2010 Computer Championship be open?
« Reply #110 on: Jan 13th, 2010, 9:55am » |
Quote Modify
|
The initial spate of qualifying games has tapered off, so that now I think we are actually behind schedule, i.e. 40% of the month is gone but less than 40% of the qualifying games have been played. In terms of server load we are even further behind because the fast and blitz bots create more load per game than the P1 and P2 bots. I hope that the last-minute rush of CC qualifying games doesn't bog down the server.
|
|
IP Logged |
|
|
|
tize
Forum Guru
Arimaa player #3121
Gender:
Posts: 118
|
|
Re: Will the 2010 Computer Championship be open?
« Reply #111 on: Jan 15th, 2010, 2:23am » |
Quote Modify
|
Quote:
bot_pragmatictheory
...
bot_Bomb2005Fast
* s L 1263437897 131721
* g L 1263441124 131722
streak is 0, losses is 2, total score is 16, total losses is 4, finished is 1263007331
bot_OpFor2009Fast
* s L 1263515921 131816
* g L 1263518395 131821
streak is 0, losses is 2, total score is 16, total losses is 6, finished is 1263007331
|
|
Quote:
For bots which tie on qualifying score, the first tie-breaker is zero minus the losses prior to longest winning streak with each bot. Losses after the longest winning streak are ignored.
|
|
These four losses shouldn't count in the tie-breaker score as they are all played after the winning streak of 0.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Will the 2010 Computer Championship be open?
« Reply #112 on: Jan 15th, 2010, 12:19pm » |
Quote Modify
|
Or it could be considered loses prior to the winning streak of 0.
Unfortunately we didn't discuss this detail prior to the start of the qualifying.
Might as well discuss it now and see if it is critical enough to warrant changing it.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Will the 2010 Computer Championship be open?
« Reply #113 on: Jan 15th, 2010, 1:29pm » |
Quote Modify
|
on Oct 28th, 2009, 5:04pm, aaaa wrote:| Wouldn't it be better if the first tiebreaker only applied to all the qualifying games up to the last one that contributed to the primary score? Otherwise, there could be an incentive for one to stop early. |
|
on Jan 15th, 2010, 12:19pm, omar wrote:Or it could be considered loses prior to the winning streak of 0.
Unfortunately we didn't discuss this detail prior to the start of the qualifying. |
|
The motivation behind the current score/tiebreak method that aaaa proposed was explicitly to make it risk-free to play more games. In that sense we did discuss it. The decision was implicitly that your tiebreak never gets worse unless your score has gotten better, because if you can lower your tiebreak without raising your score, you have an incentive to stop playing qualifying games.
I disagreed with the idea of making it risk-free to play more games, but I lost the argument, so it's OK. Now that we have made it risk-free to keep trying in all other cases, it just seems wrong for the first games against a bot to be risky. It makes no sense if a record of
WLLLLLLWLLLLLLLLLLLL
against a benchmark gets no tiebreak penalty, while a record of
LLLLLLLL
is minus eight in the tiebreaker. Maybe in a logical-coding-procedure sense it could be consistent, but in terms of rewarding/punishing developer behavior it would be wildly inconsistent. Also it would not be true to the literal wording of aaaa's proposal that was supposedly accepted, because it would be counting games in the tiebreak after the last game that contributed to the bot's primary score.
|
| « Last Edit: Jan 15th, 2010, 1:32pm by Fritzlein » |
IP Logged |
|
|
|
jdb
Forum Guru
Arimaa player #214
Gender:
Posts: 682
|
|
Re: Will the 2010 Computer Championship be open?
« Reply #114 on: Jan 15th, 2010, 2:18pm » |
Quote Modify
|
on Jan 15th, 2010, 2:23am, tize wrote:
These four losses shouldn't count in the tie-breaker score as they are all played after the winning streak of 0. |
|
I agree.
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: Will the 2010 Computer Championship be open?
« Reply #115 on: Jan 15th, 2010, 9:38pm » |
Quote Modify
|
on Jan 15th, 2010, 2:23am, tize wrote:| These four losses shouldn't count in the tie-breaker score as they are all played after the winning streak of 0. |
|
+1
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender:
Posts: 1413
|
|
Re: Will the 2010 Computer Championship be open?
« Reply #116 on: Jan 19th, 2010, 6:06pm » |
Quote Modify
|
on Jan 13th, 2010, 9:55am, Fritzlein wrote:| The initial spate of qualifying games has tapered off, so that now I think we are actually behind schedule, i.e. 40% of the month is gone but less than 40% of the qualifying games have been played. In terms of server load we are even further behind because the fast and blitz bots create more load per game than the P1 and P2 bots. I hope that the last-minute rush of CC qualifying games doesn't bog down the server. |
|
Sorry, I've contributed a big fraction to this by not being organized. I've finally updated gnobot's opening book, so am ready to start my charge for the year! (Sorry, no major improvements this year.)
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Will the 2010 Computer Championship be open?
« Reply #117 on: Jan 19th, 2010, 7:42pm » |
Quote Modify
|
Hey, I'm glad that you are at least going to try to get GnoBot seeded properly rather than accepting the #8 seed. It's better to have the last-minute rush of games now instead of... well, instead of at the last minute. 
I wonder how much the opening book will raise GnoBot's seed. If the book is too effective, it may be an argument for next year's qualifying bots to be blitz bots only, with no P1 or P2 bots, so that opening books will at least have to cope with minor clock fluctuations from the benchmark bots.
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: Will the 2010 Computer Championship be open?
« Reply #118 on: Jan 20th, 2010, 1:43am » |
Quote Modify
|
on Jan 19th, 2010, 7:42pm, Fritzlein wrote:| I wonder how much the opening book will raise GnoBot's seed. If the book is too effective, it may be an argument for next year's qualifying bots to be blitz bots only, with no P1 or P2 bots, so that opening books will at least have to cope with minor clock fluctuations from the benchmark bots. |
|
Interestingly both ArimaaScoreP1 and Aamira2006P2 went out of book very early because of some algorithmic randomness. I checked and it was definitely them that deviated.
In the end these bots don't affect anything much because they are all a bit too weak. So I'd argue for cutting them for that reason alone. I'm not sure tailoring the rules or selection bots to suit or not suit participant styles is a good idea. Are book wins less legitimate? They force other bots in the tournament to be somewhat adaptive/randomized, which in itself removes another system humans could use to maintain their dominance.
I think the qualifying method will spread our bots well. Gnobot is likely to clock up quite a lot of losses if it ever gets to 32 streaks. Marwin's result is amazing. I think everyone is sitting at 16 in trepidation because it seems so impossible!
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Will the 2010 Computer Championship be open?
« Reply #119 on: Jan 20th, 2010, 8:25am » |
Quote Modify
|
on Jan 20th, 2010, 1:43am, 99of9 wrote:| In the end these bots don't affect anything much because they are all a bit too weak. So I'd argue for cutting them for that reason alone. |
|
Agreed. And luckily we will have some new bots to replace them with, e.g. bot_Marwin2010Blitz.
Quote:| I'm not sure tailoring the rules or selection bots to suit or not suit participant styles is a good idea. Are book wins less legitimate? |
|
Yes, I think book wins are less legitimate for this purpose because they are a less robust indicator of tournament performance. If someone comes up with a way to ace the qualifying, we want that to also be a way to ace the tournament.
Quote:| They force other bots in the tournament to be somewhat adaptive/randomized, which in itself removes another system humans could use to maintain their dominance. |
|
I think book wins within the computer tournament are entirely legitimate, and I'm very happy for GnoBot's book win over Bomb in 2009 for exactly the reasons you mention. Because of GnoBot, every developer should implement some kind of defense against losing the same way twice. But whatever defense is implemented by developers in 2010 can't be backported to old benchmark bots.
Furthermore, if everyone is like JDB in letting parallelism be the main line of defense, but Omar turns off parallel search for game room bots (as I think he must to conserve server resources), then opening books could be extremely effective against benchmark bots while being wholly ineffective in the tournament itself. That kind of disconnect would be a flaw in the qualifying procedure.
Quote:| I think the qualifying method will spread our bots well. Gnobot is likely to clock up quite a lot of losses if it ever gets to 32 streaks. Marwin's result is amazing. I think everyone is sitting at 16 in trepidation because it seems so impossible! |
|
I'm pleased the system is working so far, but it hasn't really been stressed yet since many bots are sitting at 16, except marwin, which didn't have many losses. I'm imagining many bots coming up against a benchmark they can't beat consistently, and playing all 20 games to try to stretch a one-game winning streak into a two-game winning streak. And we have only eleven days left for that to happen. Yikes! This could still be a train wreck on the last few days of the month.
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print