Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Bot Development >> Gnobot 2006 - Frozen in time
        
(Message started by: 99of9 on Jun 7th, 2006, 9:55pm)
Title: Gnobot 2006 - Frozen in time
Post by 99of9 on Jun 7th, 2006, 9:55pm
Hi all

I'm interested in your opinions on the following topic:

Gnobot_2006 has its automatic learning turned off, so that it plays exactly the same as it did during the final game of the CC.

I can see Omar's point in doing this, however:

* This artificially reduces gnobot_2006 to a deterministic bot (apart from load-effects).  During the CC, gnobot could play openings with some variability, because its internal state changed slightly between each game.  Other bots have variabilty hard coded in with a random number generator.  So because the random number seed is not frozen, they can continue to play differently each time.

* What do we want to freeze?  Arimaa-playing-ability, or Arimaa-playing-technology?  We could choose to freeze the latter, and allow our "technology" to make use of the fact that games continue to be played.  We already allow the time-limited versions of the bots to improve with cpu speed.  If we chose this option, history-learning bots would not suddenly learn things their technology did not allow for.  (Gnobot would not suddenly discover a deep understanding of goal threats.)  

Since the Arimaa Challenge is about game-AI, I vote for a technology-freeze rather than an ability-freeze.

What do you all think?
Title: Re: Gnobot 2006 - Frozen in time
Post by Fritzlein on Jun 7th, 2006, 11:08pm
I'd rather see Gnobot2006 learn.  Like you say, the technology is frozen in time, even if the opening book is not.  If the objective of having old versions of the bot available to play is to showcase the state of the art as it was at times past, then a frozen opening book doesn't do this as well as having learning continue.  In fact, a frozen opening book is showcasing a Gnobot dumber than the one that actually played.

But isn't there an issue of server resources?  What does it add to the server load for Gnobot to be constantly updating its book?  And does each version of Gnobot2006 keep its own book?  How much disk space are we talking about?  I also can see a case against giving Gnobot more server resources than other bots get.
Title: Re: Gnobot 2006 - Frozen in time
Post by 99of9 on Jun 8th, 2006, 12:15am

on 06/07/06 at 23:08:47, Fritzlein wrote:
But isn't there an issue of server resources?

I think Omar made this choice based on his freezing principles.


Quote:
What does it add to the server load for Gnobot to be constantly updating its book?

Hardly anything.  It takes a few seconds each time a game ends.


Quote:
And does each version of Gnobot2006 keep its own book?

No, all the books would be identical, so they can use the same directory.


Quote:
How much disk space are we talking about?  I also can see a case against giving Gnobot more server resources than other bots get.

This is more pertinent.  The book does take up quite a lot of space.  Gnobby already uses lots more disk space than other bots.  The space taken is proportional to the number of games played on the server, so I can imagine it doubling in the next couple of years.  But disk space is cheap nowadays...  Anyway, I don't think this is the reason that Omar made the choice.
Title: Re: Gnobot 2006 - Frozen in time
Post by PMertens on Jun 8th, 2006, 4:08pm
Please make it learn ...

Now imagining that in the future several bots might want to use different kinds of opening books ... hmmm, maybe eventually it could be considered to open up one openly available opening book API for all because otherwise it might be difficult to argue why not to allow it for all in different variations.

That said I am pretty certain the opening book logic you implented could be made a little more powerful ;-)

Gnobot is not learning opponent related moves, am I correct ? That is probably good, because it prevents that eventually older bots will simply be bashed on trampled paths ;-)
Title: Re: Gnobot 2006 - Frozen in time
Post by 99of9 on Jun 8th, 2006, 5:19pm

on 06/08/06 at 16:08:29, PMertens wrote:
That said I am pretty certain the opening book logic you implented could be made a little more powerful ;-)

I'm pretty certain it can too!


Quote:
Gnobot is not learning opponent related moves, am I correct ? That is probably good, because it prevents that eventually older bots will simply be bashed on trampled paths ;-)

If you mean "Does it play differently against different opponents?"  Then the answer is no.  So it will not bash all the others on trampled paths unless it finds a line which bashes them *all* (and works ok against humans).
Title: Re: Gnobot 2006 - Frozen in time
Post by unic on Jun 8th, 2006, 6:16pm
I agree that freezing book learning makes no sense at all.  Any reasonable player learns from losses - having a program (if it's been programmed to do so) avoid playing the same loss makes perfect sense.
Title: Re: Gnobot 2006 - Frozen in time
Post by PMertens on Jun 11th, 2006, 1:57pm
so far it is 8000 (rating points) vs 2000  ;)
Title: Re: Gnobot 2006 - Frozen in time
Post by omar on Jun 17th, 2006, 12:37pm
My intent is to freeze "ability" in order to preserve the bot as it was for historical purposes. GnoBot2006CC should mean the version as it was (ability wise) when submitted after the computer championship. If it continued to learn then it would continue to improve, but we would lose the version that we had. So if a developer wanted to answer the question "how would the bot that I am working on do against the version of GnoBot that played in the 2006 computer championship" they would need GnoBot2006CC to be frozen in ability. The version that continued to learn would not be the same as the version (ability wise) that played in the 2006 computer championship.

If GnoBot2006CC was not frozen in terms of learning then yes it would help determine how good this bot could get with the learning technology of that version. But the proper way to do that is to have an unfrozen version with a different name; maybe GnoBot2006CCL. Where the 'L' indicates learning is turned on.

So the real issue for me is: should I make the 'L' version available or should Toby. That is, should I run this bot on some hardware that I posses or should Toby run it from his hardware. On the surface it seems that it doesn't really matter, but it does. The difference Im concerned about is who will have possession of the data files that represent GnoBot's improved brain. If I run the 'L' version then I will have possession of the data files. Now consider this: would it be fair (to the other developers) if I give those data files to Toby at the end of the year just before the next computer championship. Rather then debate this I think it might be best if Toby runs the version with learning turned on.
Title: Re: Gnobot 2006 - Frozen in time
Post by PMertens on Jun 17th, 2006, 5:55pm
I would bet some money, that toby could easily reproduce the data file from the game history ;-)


Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB © 2000-2003. All Rights Reserved.