Author |
Topic: Game archive irregularities (Read 2462 times) |
|
Janzert
Forum Guru
Arimaa player #247
Gender: 
Posts: 1016
|
 |
Game archive irregularities
« on: Jan 19th, 2006, 5:27pm » |
 Quote  Modify
|
While not really about bot development I figured this is more relevant here than the general forum. Although everyone probably reads both anyway. 
While parsing through the game archives recently I've run into a number of places where either the formatting or data is incorrect. I thought I would post here so it could either be corrected or at least give a heads up for anyone trying to do the same thing in the future. Some of the things I'm actually not sure what the correct change would be. Anyway here's the things I've found so far. Also I've only gone through games to the end of 2005.
The last 4 games of allgames2004.txt do not have both players present until 2005 (one has one of the players, a bot, seated in 2004 the rest don't have any events happen until 2005).
The last game in allgames2004.txt (game id 10807) is repeated as the first game in allgames2005.txt.
allgames2003.txt, line 22, game id 86, has an extra \n after the last move in the move list.
The following lines in allgames2005.txt do not have a \n after the last event, actually they all appear to only be partially written events.
Line Game id
1337 12142
1492 12297
1621 12428
1685 12492
1824 12631
6197 17005
6549 17357
The following games have some events missing the raw integer timestamp and the brackets around the long form timestamp.
File Line Game id
allgames2003.txt 3071 3137
allgames2003.txt 3072 3138
allgames2003.txt 3073 3139
allgames2003.txt 3074 3140
allgames2003.txt 3075 3141
allgames2003.txt 3451 3518
allgames2004.txt 3294 8082
Paul Lefert's rating is set to zero for one game ( 11128 ). The game before, it is 1703 and the game after, it is 1687.
Game 20310 has two moves for 11w recorded. The first one is actually 10b recorded as 11w.
10b ec5e ed5e Md4n rd8s\n
11w ec5e ed5e Md4n rd8s\n
11w Ee6e Ef6s cf7s cf6x Re3n\n
Some of the events might give an idea as to what happened.
1128731702 [Sat Oct 8 00:35:02 2005] move 10w received from w\n
1128731838 [Sat Oct 8 00:37:18 2005] move 10b received from b\n
1128731838 [Sat Oct 8 00:37:18 2005] move 10b received from b\n
1128731840 [Sat Oct 8 00:37:20 2005] move 11w received from b\n
1128731840 [Sat Oct 8 00:37:20 2005] gameserver rejected move; posted by the wrong side\n
1128731840 [Sat Oct 8 00:37:20 2005] move 11w received from b\n
1128731840 [Sat Oct 8 00:37:20 2005] gameserver rejected move; posted by the wrong side\n
1128731849 [Sat Oct 8 00:37:29 2005] move 11w received from w\n
1128731864 [Sat Oct 8 00:37:44 2005] move 11b received from b\n
1128731949 [Sat Oct 8 00:39:09 2005] b player has left\n
1128731959 [Sat Oct 8 00:39:19 2005] b player joining\n
1128731964 [Sat Oct 8 00:39:24 2005] b player present\n
1128731999 [Sat Oct 8 00:39:59 2005] move 11b received from b\n
Janzert
|
|
 IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender:
Posts: 1413
|
|
Re: Game archive irregularities
« Reply #1 on: Jan 19th, 2006, 5:35pm » |
Quote Modify
|
Wow, thanks for your detailed checking Janzert...
I'm surprised none of my database reading programs crashed on the formatting issues.
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: Game archive irregularities
« Reply #2 on: Jan 19th, 2006, 5:44pm » |
Quote Modify
|
Yours are probably more resilient than mine.
Actually as long as someone isn't parsing out individual moves and events they probably won't run into any problems.
Janzert
|
| « Last Edit: Jan 19th, 2006, 5:45pm by Janzert » |
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: Game archive irregularities
« Reply #3 on: Jan 19th, 2006, 5:58pm » |
Quote Modify
|
on Jan 19th, 2006, 5:44pm, Janzert wrote:| Actually as long as someone isn't parsing out individual moves and events they probably won't run into any problems. |
|
I do parse the moves (but not the events) ... to construct an opening book (and some other tables a while ago). I think I must've just been lucky on the resilience front - I certainly planned to only accept the correct format.
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: Game archive irregularities
« Reply #4 on: Jan 26th, 2006, 10:54pm » |
Quote Modify
|
Btw, found a couple more problems. Not really serious but just in case it helps someone else avoid problems by knowing they exist.
Game 46, 21b silver wins by goal but the game continues.
Then apparently some bugs in the immobilization detection code early on.
Some games were record as time (t) wins instead of immobilization (m).
m termination recorded as t, Game 253 (253), 24b
m termination recorded as t, Game 1333 (1330), 42b
m termination recorded as t, Game 1986 (1983), 19w
m termination recorded as t, Game 4892 (4888), 71b
m termination recorded as t, Game 5028 (5024), 62b
m termination recorded as t, Game 5162 (5158), 37b
m termination recorded as t, Game 5758 (5754), 44w
and several games were terminated when a side moved into immobilization before allowing the other side to make a final move.
Last move of immo game found with replies, Game 90 (90), 16b
Last move of immo game found with replies, Game 249 (249), 15b
Last move of immo game found with replies, Game 250 (250), 16b
Last move of immo game found with replies, Game 258 (258), 21w
Last move of immo game found with replies, Game 259 (259), 31b
Last move of immo game found with replies, Game 271 (271), 23b
Last move of immo game found with replies, Game 281 (281), 18w
Last move of immo game found with replies, Game 337 (337), 21b
Last move of immo game found with replies, Game 343 (343), 17b
Last move of immo game found with replies, Game 361 (361), 20w
Last move of immo game found with replies, Game 362 (362), 25b
Last move of immo game found with replies, Game 380 (380), 39b
Last move of immo game found with replies, Game 419 (419), 21w
Last move of immo game found with replies, Game 421 (421), 19b
Last move of immo game found with replies, Game 438 (438), 18b
Last move of immo game found with replies, Game 482 (482), 26b
Last move of immo game found with replies, Game 483 (483), 19b
Last move of immo game found with replies, Game 484 (484), 30b
Last move of immo game found with replies, Game 559 (557), 18b
Last move of immo game found with replies, Game 571 (569), 19w
Last move of immo game found with replies, Game 894 (891), 65b
Last move of immo game found with replies, Game 1153 (1150), 19w
Janzert
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: Game archive irregularities
« Reply #5 on: Mar 1st, 2006, 4:07pm » |
Quote Modify
|
Two more slight gotchas I ran into.
The plycount field in the database actually counts move number not ply. I'm not sure if it records the move number when the game ends or the maximum move number in the game (this could be different because of takebacks). Takebacks that occur in the middle of the game do not appear to effect it.
Game #3136 is unique in that it has no moves and also because it has a termination type of f.
Janzert
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: Game archive irregularities
« Reply #6 on: Mar 1st, 2006, 6:06pm » |
Quote Modify
|
Thanks for keeping up this record. I will certainly consult it next time I need to auto-process the database.
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: Game archive irregularities
« Reply #7 on: Mar 16th, 2006, 8:50pm » |
Quote Modify
|
The recent game (#25959) between bot_Clueless2006P1 and bot_Clueless2006Fast lasting 600 moves not only set the record for longest game, but also unfortunately added another instance of not all events being recorded.
The last whole event recorded is move 504w. The next event is cut off in the middle of the timestamp.
Janzert
|
| « Last Edit: Mar 16th, 2006, 8:50pm by Janzert » |
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: Game archive irregularities
« Reply #8 on: Mar 17th, 2006, 7:49am » |
Quote Modify
|
Game 25990 is another game with a double move recorded. As with #20310 the first move recorded is actually a repeat of the previous move.
30w Hh6n Hh7s rh8s\n
30b Hh6n Hh7s rh8s\n
30b df6s df5s eg3w rh3w\n
The events show two moves submited for 30w. Maybe a race condition in the side to play checking code?
1142014934 [Fri Mar 10 18:22:14 2006] move 30w received from w\n
1142014934 [Fri Mar 10 18:22:14 2006] move 30w received from w\n
1142014943 [Fri Mar 10 18:22:23 2006] move 30b received from b\n
1142015007 [Fri Mar 10 18:23:27 2006] move 31w received from w\n
Janzert
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: Game archive irregularities
« Reply #9 on: Mar 23rd, 2006, 1:26pm » |
Quote Modify
|
Games that have a termination type of 'a' used to have a result of 'u' (games 1990, 1991, 7955, etc.). Recently though they are given no result at all (games 23634, 24143).
Janzert
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Game archive irregularities
« Reply #10 on: Mar 25th, 2006, 11:11am » |
Quote Modify
|
on Mar 16th, 2006, 8:50pm, Janzert wrote:The recent game (#25959) between bot_Clueless2006P1 and bot_Clueless2006Fast lasting 600 moves not only set the record for longest game, but also unfortunately added another instance of not all events being recorded.
The last whole event recorded is move 504w. The next event is cut off in the middle of the timestamp.
Janzert |
|
The 'event' field could only store 64K which is why it got chopped off. I've changed it now to store up to 16M.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Game archive irregularities
« Reply #11 on: Mar 25th, 2006, 11:15am » |
Quote Modify
|
on Mar 17th, 2006, 7:49am, Janzert wrote:Game 25990 is another game with a double move recorded. As with #20310 the first move recorded is actually a repeat of the previous move.
30w Hh6n Hh7s rh8s\n
30b Hh6n Hh7s rh8s\n
30b df6s df5s eg3w rh3w\n
The events show two moves submited for 30w. Maybe a race condition in the side to play checking code?
1142014934 [Fri Mar 10 18:22:14 2006] move 30w received from w\n
1142014934 [Fri Mar 10 18:22:14 2006] move 30w received from w\n
1142014943 [Fri Mar 10 18:22:23 2006] move 30b received from b\n
1142015007 [Fri Mar 10 18:23:27 2006] move 31w received from w\n
Janzert |
|
This happened in my postal game against Derek (dtj) also. I was finally able to trace this problem and fix it. It was being caused by the send button being clicked twice and two processes getting started on the server to add the move.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Game archive irregularities
« Reply #12 on: Mar 25th, 2006, 11:40am » |
Quote Modify
|
on Mar 23rd, 2006, 1:26pm, Janzert wrote:Games that have a termination type of 'a' used to have a result of 'u' (games 1990, 1991, 7955, etc.). Recently though they are given no result at all (games 23634, 24143).
Janzert |
|
Changed these games to have a result='u'. Modified the code which was most likely causing this.
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: Game archive irregularities
« Reply #13 on: Mar 25th, 2006, 5:37pm » |
Quote Modify
|
Thanks Omar.
Janzert
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Game archive irregularities
« Reply #14 on: Mar 27th, 2006, 10:28am » |
Quote Modify
|
Thanks for reporting these, Brian.
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print