Author |
Topic: 2009 Arimaa Challenge (Read 7812 times) |
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender: 
Posts: 1413
|
 |
Re: 2009 Arimaa Challenge
« Reply #90 on: Mar 25th, 2009, 8:10pm » |
 Quote  Modify
|
on Mar 25th, 2009, 4:43pm, omar wrote:Ron, from your ruling I am assuming that game 100123 (bot_clueless vs ChrisB) should also be continued. Please confirm.
|
|
No, that had to be scrapped and restarted because both bots were playing on the same server. Ron's phrase "bot play was not tainted before the timeout" is the key to deciding which should be continued and which should be scrapped. The games that were already scrapped including this one were either due to bots running on the same server, or under the wrong time control.
The game ChrisB needs to continue is 100645.
|
| « Last Edit: Mar 25th, 2009, 8:13pm by 99of9 » |
 IP Logged |
|
|
|
ChrisB
Forum Guru
Arimaa player #2339
Gender:
Posts: 147
|
|
Re: 2009 Arimaa Challenge
« Reply #91 on: Mar 25th, 2009, 9:42pm » |
Quote Modify
|
on Mar 25th, 2009, 8:10pm, 99of9 wrote:
The game ChrisB needs to continue is 100645. |
|
Yes, Omar and I already corresponded about continuing 100645, and the likely time for continuing that game will be this Friday at 0400 GMT (11 pm Thursday for me). I also mentioned to Omar that I'm not sure whether the March 25 TD decision brings game 100123 back into play. I see arguments both for and against continuing that game, so whatever Ned decides will be fine with me.
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: 2009 Arimaa Challenge
« Reply #92 on: Mar 26th, 2009, 12:53am » |
Quote Modify
|
on Mar 25th, 2009, 9:42pm, ChrisB wrote:| I also mentioned to Omar that I'm not sure whether the March 25 TD decision brings game 100123 back into play. |
|
I very much doubt that that is what Ned meant, but am happy to wait for him to clarify. Of course if that game came back into play, we might have to resurrect a whole bunch, some which have already been replayed!
|
|
IP Logged |
|
|
|
RonWeasley
Forum Guru
Harry's friend (Arimaa player #441)
Gender:
Posts: 882
|
|
Re: 2009 Arimaa Challenge
« Reply #93 on: Mar 26th, 2009, 3:43am » |
Quote Modify
|
on Mar 25th, 2009, 4:43pm, omar wrote:
Ron, from your ruling I am assuming that game 100123 (bot_clueless vs ChrisB) should also be continued. Please confirm.
|
|
No, not this one where multiple bots were running on the same server. This game is nullified.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2009 Arimaa Challenge
« Reply #94 on: Mar 26th, 2009, 1:49pm » |
Quote Modify
|
Thanks for the clarification.
|
|
IP Logged |
|
|
|
RonWeasley
Forum Guru
Harry's friend (Arimaa player #441)
Gender:
Posts: 882
|
|
Re: 2009 Arimaa Challenge
« Reply #95 on: Mar 27th, 2009, 5:01am » |
Quote Modify
|
on Mar 25th, 2009, 4:43pm, omar wrote:
The silver server is scheduled to be terminated at the end of this month. So after that time there will only be one server to play on. Also the official challenge match games are scheduled to begin on April 5th, so I will have to stop the screening on April 4th.
|
|
Noted. I approve the end of screening to be April 4.
|
|
IP Logged |
|
|
|
woh
Forum Guru
Arimaa player #2128
Gender:
Posts: 254
|
|
Re: 2009 Arimaa Challenge
« Reply #96 on: Mar 28th, 2009, 2:12am » |
Quote Modify
|
Omar, the link in the gameroom to the bot scores page isn't working. I think you need to leave out the 's' at the end.
|
|
IP Logged |
|
|
|
Arimabuff
Forum Guru
Arimaa player #2764
Gender:
Posts: 589
|
|
Re: 2009 Arimaa Challenge
« Reply #97 on: Mar 28th, 2009, 9:13am » |
Quote Modify
|
on Mar 28th, 2009, 2:12am, woh wrote:| Omar, the link in the gameroom to the bot scores page isn't working. I think you need to leave out the 's' at the end. |
|
True, when I remove the "s" by hand the link works.
|
|
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: 2009 Arimaa Challenge
« Reply #98 on: Mar 28th, 2009, 8:16pm » |
Quote Modify
|
It's amazing to note that despite having made no less than 6(!) attempts to play a preliminary game, it now looks that I will have made no official contribution whatsoever! Still, considering the purposes of this screening, I think, even leaving it like that (and I'm not changing my mind here), I've done good if I do say so myself; with respect to giving the defenders games to study, six games, even including incomplete ones, should be gnarly, no? Especially since, as you'll see below, they unfolded quite diversely. Also, given the whole mess that has plagued my games, it's probably for the best that I won't be responsible for any differentiation of the two bots. Chalking that first infamous game against Clueless up to me getting back into Arimaa after a short hiatus, it's hard to identify the stronger (anti-human) bot of the two. If, however, GnoBot manages to qualify with exactly one point difference, it will still leave a bad taste in my mouth.
Since the multitude of my games seems to have caused a bit of a confusion here, here's a summary of them:
Game 99880: After inflicting massive (what I would call fatal) material damage, Clueless times out, at first seemingly because it was being too cavalier managing its time; the game is later voided due to a wrong time control system being in use. I came close to resigning that game and the advantages I enjoyed in the other timed-out games come nowhere close to Clueless's in this one. So I reject the criticism that I'm being inconsistent; it's a fact of life that a later resumption of a game will advantage the human player over the computer, so only playing out a game that's clearly won by the latter should not be dubious.
Game 99969: In a much more promising position for me, Clueless times out again. At first judged as a legitimate outcome, some time afterwards more timeouts point to an interfering cron job by the server as the culprit. Now ruled to be resumed.
Game 100048: I win against GnoBot in an entertaining and (more importantly) legitimate game.
Game 100135: I finally get a game against Clueless to reach its natural ending (with me successfully converting a horse frame), turns out the bot was sharing computer resources for some time. Game gets discounted (and unrated).
Game 100273: This time, I'm the one having my horse framed, in a game against GnoBot. I still manage to win eventually. Counts.
Game 100358: Clueless times out yet again, this time early on. It is ruled to be resumed due to a mysterious network lag. At the time, I enjoyed a reasonable advantage, which Bomb2005P2, I believe, evaluated as 2.97, no doubt due to the horse I'm framing. Aided by a substantial amount of (unfair) pondering, I'm somewhat more troubled by the position.
Tentatively, I could hypothesize that GnoBot is relatively strong in the opening (not in a small part due to its book) but has trouble following through on any advantage accrued in that stage, whereas Clueless seem to be more of an all-rounder.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2009 Arimaa Challenge
« Reply #99 on: Mar 29th, 2009, 2:59pm » |
Quote Modify
|
Aaaa, I'm sorry to hear that you are still declining to finish your clueless timeouts as requested by the TD, but thanks anyway for your six attempts at participation. Although you haven't contributed to the score of the bots or helped choose between them, you have indeed shed some light on their strengths and weaknesses, which is one of the explicit purposes of the screening period.
Will you also decline to participate in any future screening period, because your decision to withdraw is not based on what is expedient at present, but rather based on the principle that the rules are inherently unfair?
|
| « Last Edit: Mar 30th, 2009, 6:32pm by Fritzlein » |
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2009 Arimaa Challenge
« Reply #100 on: Apr 3rd, 2009, 9:39am » |
Quote Modify
|
Game 101769 between sforry and bot_clueless timed out today.
I think this game needs to be continued since a network problem caused the timeout.
http://gold.arimaa.com/~clueless/logs/3426.netLog
|
| « Last Edit: Apr 3rd, 2009, 9:41am by omar » |
IP Logged |
|
|
|
RonWeasley
Forum Guru
Harry's friend (Arimaa player #441)
Gender:
Posts: 882
|
|
Re: 2009 Arimaa Challenge
« Reply #101 on: Apr 3rd, 2009, 9:50am » |
Quote Modify
|
on Apr 3rd, 2009, 9:39am, omar wrote:
Yes. Please continue the game.
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: 2009 Arimaa Challenge
« Reply #102 on: Apr 4th, 2009, 4:45am » |
Quote Modify
|
Congratulations in advance to Clueless/Jeff for winning the screening. I'll try to give you a bigger challenge next year  . After all the technical problems, I'm glad the bots ended up separated by a decent margin. I'm also glad they both got a few wins in the end - the start of the screening was looking quite humiliating for both bots.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2009 Arimaa Challenge
« Reply #103 on: Apr 4th, 2009, 6:36am » |
Quote Modify
|
Sadly, the screening period closed on another invalid game. Woh's epic 144-move win over GnoBot, lasting six hours and thirty-nine minutes, can't count because woh was assigned the wrong color. By my count that makes nine invalidated games for the tournament: woh's wrong-color game, four timeouts that weren't completed, and four games that overlapped on the same server.
Fortunately we had such enthusiastic participation that many games were completed and the bots were clearly discriminated. We had fifty-one valid games, including two server timeouts that were restarted and played to completion, so we beat last year's participation in spite of all the issues.
2008 2009
---- ----
19 21 Participants
41 51 Valid games
16 21 Valid Pairs
7 7 Discriminating Pairs
Every participation statistic increased except for the number of discriminating pairs, i.e. the number of times a player beat one bot and lost to the other with the same color. It may be that when the two bots are closer in strength, as they were this year, it is less likely for any pair of games to provide discrimination. Last year bomb beat sharp 6-1 in the discriminating pairs; this year clueless beat GnoBot 5-2.
Clueless finished with a record of 15-10 in valid games, for a performance rating of 1941. Clueless' performance was very consistent. Other than splitting two games with woh (who is rated right near clueless' performance), Clueless defeated everyone rated 1910 or below while losing to everyone rated 2002 and above. This seems to peg clueless' true strength (at this time control and hardware) in a rather narrow range.
GnoBot finished with a record of 12-14 in valid games, for a performance rating of 1791. GnoBot's performance was much more erratic, with two wins against over-2000 players (The_Jeh and naveed) but also five losses to under-1900 players (Tuks, Simon x2, sforry, and joe). One must wonder whether some of the holes in GnoBot's strategy could be plugged without undermining its strengths.
For comparison, during the 2008 screening, bomb's performance rating was 1918, and sharp's was 1576. So this year the race was not only closer, the bar was higher.
All in all the screening period fully fulfilled its function. The strengths and weaknesses of both bots were probed, giving the Challenge defenders much useful information. The bot that would be easier for the defenders to defeat was eliminated. This is precisely what is supposed to happen during the screening.
|
| « Last Edit: Apr 4th, 2009, 6:39am by Fritzlein » |
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: 2009 Arimaa Challenge
« Reply #104 on: Apr 4th, 2009, 7:22am » |
Quote Modify
|
on Apr 4th, 2009, 6:36am, Fritzlein wrote:| By my count that makes nine invalidated games for the tournament: woh's wrong-color game, four timeouts that weren't completed, and four games that overlapped on the same server. |
|
You forgot about four games which didn't count because they used the wrong time control.
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print