Arimaa Forum - Print Page


    
      
        Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Bot Development >> GnoBot submits illegal move?
        
(Message started by: robinz on Feb 6^th, 2011, 11:14am)

Title: GnoBot submits illegal move?
Post by robinz on Feb 6^th, 2011, 11:14am

Hi,

I'm not sure if this is the right place in the forum to post this - apologies if someone has to move it.

I just played a game against bot_GnoBot2005P2, but the game abruptly ended at the bot's move 13s, with the message "This bot lost due to submitting an illegal move [] [no steps taken". Not only is this a very unsatisfying way to "win" a game, but it also suggests something is wrong with the bot, and perhaps the game client too? After all, a bot surely shouldn't be trying to submit an illegal move. And, if it does, surely the client should just reject it, not adjudge the game a loss? (I have in the past tried to make a move which is forbidden due to repetition, but wasn't told that I had lost, just that I had to choose a different move.) Although, I guess, if this is a bug in GnoBot which forces it to submit an illegal move, it would end up losing on time in any case.

Apologies if this is a known issue - but if not, I thought Toby, at least, should know about it :)

Title: Re: GnoBot submits illegal move?
Post by 99of9 on Feb 6^th, 2011, 6:29pm

Hi robinz,

If it was related to move repetitions, it is possible that it was gnobot's fault. Sorry to cause your dissatisfaction. Unfortunately there would be no point rejecting the move and continuing the game, because gnobot would not do anything any differently. On the other hand, I've seen some previous non-repetition instances where I could not reproduce the problem, and had to conclude that the server had caused it (e.g. cpu or memory overload). But either way, it's been some time since Gnobot2005 was in development...

Thanks for the note.

Toby

Title: Re: GnoBot submits illegal move?
Post by Fritzlein on Feb 6^th, 2011, 7:39pm

Indeed, while 99of9 would want to fix any errors in GnoBot, Omar would not want to incorporate those fixes into GnoBot2005. The point of GnoBot2005 is to be an unchanging benchmark of how well GnoBot played during the 2005 Computer Championship. This issue actually became heated when GnoBot gained the ability to learn; Omar turned off the learning for the server version of GnoBot because he didn't want it to get better over time due to additional learning.

Still, there is a valid point about the letdown due to winning by illegal moves. There is a case to be made that the user experience would be better if we kicked out any old bot with bugs, and replaced it on the ladder with a new, non-buggy bot. There are two disadvantages to this course of action, though. First, it is nice to have fixed benchmarks that don't change over the years, so that we can calibrate our rating system and measure developer progress.

Second, the definition "bug" is fuzzy anyway. Submitting illegal moves is clearly a bug, but what if a bot has an elelphant-suicide bug that triggers in certain circumstances? It isn't very fun to win when a bot wigs out and kills its elephant, so should we kick such bots off the ladder? If so, what about an elephant-blockade bug in which a bot lets its elephant be easily blockaded because it incorrectly thinks its elephant is not blockaded? (OK, some people get off on beating the same bot over and over again with the same elephant blockade, so I shouldn't say that is no fun, even if it looks mind-numbingly boring to me. :))

We have removed some bots from the ladder that routinely crashed (such as Gnobot2006, if I remember correctly), but as long as Gnobot2005 crashes out of relatively few games, I support leaving it in the ladder. I like the history, the continuity, the feeling that new players are even today going through what I went through years ago.

In any case, thanks for reporting the issue, robinz. Please don't be shy about raising any issues that occur to you. Even if we differ about the best solution, I appreciate that you care enough to file a report.

Title: Re: GnoBot submits illegal move?
Post by robinz on Feb 7^th, 2011, 4:06am

Ah, yes, I see - I somehow hadn't twigged that this was a deliberately old version of the bot, although the name kind of makes it obvious. I fully understand your reasoning, in that case :)

Title: Re: GnoBot submits illegal move?
Post by rbarreira on Feb 7^th, 2011, 5:08am

The new bot interface (AEI) is a bit better in this regard, since it restarts the bot if it crashes. This means that for unlucky crashing bugs, the bot will get a second chance to submit a move.

For deterministic bugs such as ignoring the repetition rule, the bot doesn't get a second chance, but in that case there's nothing the interface can do, since there isn't (and shouldn't be, IMO) any mechanism for the interface to reject the bot's move and ask for a new one.

Title: Re: GnoBot submits illegal move?
Post by foggy on Feb 7^th, 2011, 2:45pm

It is not the only bot with such problem. A couple of weeks ago (Game 168651) bot_Loc2006Fast lasted just 5 moves. I just assumed this bot is old, any developement activities (like complying to rule changes) discontinued and it could be considered somewhat normal.