Arimaa Forum - Print Page


    
      
        Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Bot Development >> Arimaa Engine Interface (AEI)
        
(Message started by: Janzert on Oct 9^th, 2008, 7:55pm)

Title: Arimaa Engine Interface (AEI)
Post by Janzert on Oct 9^th, 2008, 7:55pm

Pretty much ever since I mentioned that OpFor uses a rewritten interface to the gameroom that allows it to run continuously during the game and ponder on the opponents move I have also been promising to release it. There have been been a few reasons I kept putting it off. The biggest reasons are, zero documentation and pretty ugly code. Unfortunately both of those are still completely true. I've decided though to get it out there so people that are interested can at least take a look at it.

So with the caveat that this is not something anyone is likely to currently find instantly or easily usable or even understandable the code can be viewed at http://bazaar.launchpad.net/~janzert/+junk/aei/files or if you have bazaar (http://bazaar-vcs.org/) installed you can get the branch with "bzr branch lp:~janzert/+junk/aei".

A couple things that will probably help anyone that does try and look at it now. AEI is based mostly on UCI (Universal Chess Interface (http://en.wikipedia.org/wiki/Universal_Chess_Interface) or specification download (http://download.shredderchess.com/div/uci.zip)). Also that gameroom.py started as and still pretty much is a straight transliteration of the 'bot' perl script, from the arimaa gameroom interface scripts, to python with AEI added in rather than calling a getmove engine.

Hopefully fairly soon I can get some documentation and maybe a sample engine written. In the meantime if anyone does look at it and has any question feel free to ask and I'll try and explain it; maybe it'll help me get real docs written.

[Edit: Ahh, forgot to mention short term plan is to get some documentation, medium term is add some sort of example engine and long term is for a GUI client that uses AEI for playing bots and also will connect to the gameroom]

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by Polyfractal on Oct 10^th, 2008, 12:45am

Thanks for this, it looks really useful. I may try to use it for my project. I've been digging through your code and a lot of it seems to make sense as far as layout (I think I have a general handle on how it operates). If I do use it I can help document as I figure things out. :)

My current system is fragile and cumbersome to use. My getMove program has to check and see if my main bot engine's thread is running, spawn if not, then communicate with the bot engine through a state file. The bot engine meanwhile must continually poll the state file for updates. If something goes wrong somewhere, both threads can be in trouble due to the fragile communication method.

This seems like a much more elegant solution. Will it be allowed for tournament play?

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Oct 10^th, 2008, 8:22am

Great, glad to hear it makes sense to someone besides myself. Anything you come up with I'm sure would help in writing the docs even if it's as simple as the times you had a hard time understanding what was going on.

Yes, Omar has allowed this to be used in the computer championship. It's actually the only interface OpFor has ever used. I had initially planned to write a layer similar to what it sounds like you are currently doing. But I really didn't like the idea of the extra layers and how kludgy the whole idea seemed. Happily Omar allowed the direct interface.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by aaaa on Oct 10^th, 2008, 9:37am

Another disadvantage with having a program instance for each move that I haven't seen mentioned, is that it makes it hard to get a profile from it on a whole-game basis.

on 10/10/08 at 00:45:30, Polyfractal wrote:

My getMove program has to check and see if my main bot engine's thread is running, spawn if not, then communicate with the bot engine through a state file.

I myself did some tentative experimentation with the intent of figuring out whether such an approach might work, but I ran into the problem that I couldn't get control to be returned to the bot script until not only the getMove program would finish, but also everything it has spawned in the mean time. I'm not exactly an expert on concurrency, so I might have missed something (possibly basic) here.

Title: Re: Arimaa Engine Interface (AEI)
Post by Polyfractal on Oct 10^th, 2008, 9:51am

Indeed. My bot scripts creates an additional "stats" log file that it dumps to, containing its own internal state and other statistics. It works, but it is irritating and another file to look after.

Regarding threading, there is a way to asynchronously spawn threads. This is a nonblocking call and returns control directly to the original program. I don't have my code handy at the moment but I can post the function later when I get home. Granted, this is for Windows, but I would assume there is a similar method for Linux.

The problem with this, however, is that the bot thread is entirely independent and not owned by any other process. If something goes wrong, it will continue to spin its wheels with no governance. My getMove program has to therefore keep an eye on the bot thread and terminate it if there is any errant behavior.

All in all, a kludgy system that makes me sad. :(

Janzert: I noticed there is a "match" script. Does that mean AEI can play local games against two different bots?

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Oct 10^th, 2008, 8:25pm

Yes, match.py will run a set of games between two engines. Also analyse.py, simply feeds a given position to an engine to think about. Both of these a very basic scripts that just barely get the job done (e.g. the command line to run an engine and any options to send to it must be set by editing the script itself). Hopefully they'll be expanded into easier to use and more flexible forms in the future.

Since I'm mentioning the purpose of those two I may as well list what each file is for:

aei.py: is unsurprisingly a library to run and control an AEI based engine
analyse.py: takes a given board position and gives it to an engine to think about
board.py: implements an Arimaa board representation*, step and move generator and some parsing and output. It actually represents most of the core of a bot, writing an example bot based off it should be fairly easy. Once I get around to doing it that is. ;)
gameroom.py (and gameroom_example.cfg): this is the interface to the gameroom
match.py: runs a basic tournament between to engines

Janzert

* as a little bit of trivia and history, this was my second implementation of an Arimaa move generator. My first was in python and used an array based board representation. This one is obviously again in python but uses a bitboard representation. 2 1/2 was a rewrite of this to make it work as rpython for pypy (http://codespeak.net/pypy/dist/pypy/doc/home.html) (an obscure python compiler and many other things but nothing you really need to know about probably). Finally, at least for now, my third is in the D programming language (http://www.digitalmars.com/d/) and is another bitboard based representation although quite a bit different from this one and with many ideas cribbed from Fotland's description of Bomb. So it would seem that I tend to write a new Arimaa move generator every 18-24 months. :)

Title: Re: Arimaa Engine Interface (AEI)
Post by omar on Oct 17^th, 2008, 4:33pm

Thanks for posting this Janzert. I have a feeling this code base will soon become the one that I recommend as the gameroom interface used by bots :-)

Title: Re: Arimaa Engine Interface (AEI)
Post by Oystein on Oct 26^th, 2008, 10:11am

Great work Janzert. I have successfully played against my engine using aei and your gameroom script. But clearly documentation would be very useful.

I have tried to make an overview of aei. What I have written is probably not easily understandable, but hopefully not totally nonsense. I have never used uci, and have never programmed in python so I assume I made some errors. Please notify me of any mistake.

Commands from controller:
* aei
- First command to establish the interface to use. Waits for 'id ...' and 'aeiok'
* isready
- Ping engine. Waits for 'readyok'
* newgame
- Starts a new game
* setposition <position>
- Set the position. Positions are given by the side to move (w/b) and a string
consisting of the piece letter or a space for each of the 64 squares.
* setoption name <id> [value <x>]
- Used for timecontrol informations. Probably also intended for settings like hash size.
* makemove <move>
- Make a move
* go
- Start search
* ponder
- Tells the engine to start pondering
* infinite
- Search until 'stop' command
* movetime <time>
- Search for given time
* depth <depth>
- Search <depth> steps deep
* stop
- Stop current search. Expect best move found so far from engine
* checkeval <"current" | position>
- Ask for evaluation of current or given position
* quit
- Quit

From engine:
* id - Identify the engine
* name <name>
* author <author>
* aeiok
- Conform to use aei
* readyok
- Answer to [isready]
* bestmove <move>
- Best move found in search
* info <string>
- Send information like best move so far, score, pv, depth searched.
<Must be specified better.>
* log <string>
- Any information the engine wants to log

Comments
w and b are used for the two sides rather than g and s. Maybe this should be changed.
uci state that the engine should ignore unknown commands. I dont like that solution. Perhaps it is better to use the log command to state the problem?

Some differences from uci:
- Time information is set by the setoption command for aei, and by go command for uci. I like the uci solution best.
- I dont understand exactly how pondering works in uci, but apparently the controller inform the engine on which move the engine "should" ponder on. I dont see the point in this.
- Lacking commands
* go searchmoves <move1> .... <movei>
I think this can be useful for analysis. But I think there should be possible to list partial moves. Like 3 steps and let the engine find the last step, rather than listing all possibilities. Also move sequenses could be useful for situations where we have found possible refutation to some moves.
* go nodes <x>
* go mate <x>
Also useful for analysis.

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Oct 26^th, 2008, 7:42pm

on 10/26/08 at 10:11:35, Oystein wrote:

Great work Janzert. I have successfully played against my engine using aei and your gameroom script. But clearly documentation would be very useful.

Wow, great to hear you are able to find it useable even without any documentation. I had my computer hard drive crash last week so have been putting my computer back together and haven't had a chance to work on this at all.

I should mention that I've also never implemented a UCI engine. My knowledge of it comes strictly from the protocol document and various email and forum messages I've read.

Quote:

I have tried to make an overview of aei. What I have written is probably not easily understandable, but hopefully not totally nonsense. I have never used uci, and have never programmed in python so I assume I made some errors. Please notify me of any mistake.

Looks like you have most of it. This would make a good base to build from for the protocol doc. Mind if I use it?

Quote:

Commands from controller:
* aei
- First command to establish the interface to use. Waits for 'id ...' and 'aeiok'

* setposition <position>
- Set the position. Positions are given by the side to move (w/b) and a string
consisting of the piece letter or a space for each of the 64 squares.

I had thought this board notation was defined somewhere but now that I'm looking for it I guess I just copied it from the perl move generator script omar provides.

Quote:

* setoption name <id> [value <x>]
- Used for timecontrol informations. Probably also intended for settings like hash size.

Yes, this is meant to set all configuration options for the engine. So while there should be a certain predefined set of options that are standard across all or most engines an engine is also free to define its own as well (e.g. besides timecontrols and hash size opfor allows the weighting given to different evaluation components to be changed).

Quote:

* go
- Start search
* ponder
- Tells the engine to start pondering
* infinite
- Search until 'stop' command
* movetime <time>
- Search for given time
* depth <depth>
- Search <depth> steps deep

The only subcommands actually meant to be used here are ponder and infinite. The movetime and depth are a relic of my initial copying from uci that I later changed my mind on, see below.

Quote:

* checkeval <"current" | position>
- Ask for evaluation of current or given position

This is something I added for debugging OpFor and I'm not sure whether it should stay in or not. It's meant to tell the engine to statically evaluate either the current or given board position and report back any debugging information and the score (e.g. OpFor also reports the score for each component of the eval). I suppose it would be useful for anything that uses a traditional evaluation function but not something like a monte-carlo searcher.

Quote:

From engine:
...
* info <string>
- Send information like best move so far, score, pv, depth searched.
<Must be specified better.>

This is meant to send information to the controller about the current search. Primarily things that would be displayed specially in a GUI separate from a general log.

Quote:

* log <string>
- Any information the engine wants to log

Log messages starting with "Warning:" will be handled specially by the gameroom interface. I should probably extend it and specify that "Warning:" and "Error:" and maybe "Debug:" be logged at the corresponding levels.

Quote:

Comments
w and b are used for the two sides rather than g and s. Maybe this should be changed.

Yes, now that Omar has stated his intention to shift to the usage of g and s I agree this should follow that.

Quote:

uci state that the engine should ignore unknown commands. I dont like that solution. Perhaps it is better to use the log command to state the problem?

I think it is appropriate for the engine to throw an error and exit on an unknown command, on the principle that it's usually better to try and catch an error as close to the point that it happens than figure out what went wrong later.

It is probably best to simply log a warning or error when receiving an unknown or incorrect options though. Since GUI clients may allow users to send any option they want.

Currently OpFor will throw an exception and exit on an unknown command, but will silently eat unknown options. I have had that latter case bite me once or twice though when I misspelled an option so logging a warning would have been a good idea.

Quote:

Some differences from uci:
- Time information is set by the setoption command for aei, and by go command for uci. I like the uci solution best.

First just so it's clear the uci "go movetime <x>" command is used to have the engine move after a fixed amount of time. In a regular timecontrol based game it sends the time information with wtime, btime, winc, binc and movestogo subcommands.

With arimaa the timecontrol system is a little more complex (and better developed in my opinion). Most of this information is set at the beginning of the game so was more natural to send as options. I also didn't like using the go command to tell the bot search to a fixed depth as that only really applies to traditional alpha-beta search. Another thing is I wanted to be able to update the time used for a move so far throughout the search in case of more accurate information coming in later because of network lag and such. So I ended up deciding to take all the timing and search depth settings off of the go command. I do admit that using setoption for things like the reserve time available and time used so far in a move does feel a little odd, but I think it's worth it since any separate command is going to basically just duplicate the functionality of setoption.

Quote:

- I dont understand exactly how pondering works in uci, but apparently the controller inform the engine on which move the engine "should" ponder on. I dont see the point in this.

Basically all modern chess engines ponder by assuming that the opponent is going to reply the way it expects and starts searching as if that move has already been made. If the opponent does make the expected move then it can reply very quickly since it has already thought about its own reply for the whole of the opponents move. But if the opponent makes an unexpected reply it throws away the whole ponder search and starts fresh with the actual move played.

UCI's ponder mechanism is completely geared around this method. When the engine sends the move to make on the board it also sends the expected reply. The controller then tells the engine to go ahead and ponder. When the opponent's reply comes in if it is the expected reply then the controller tells the engine that there was a ponder hit and the engine continues searching but switches from pondering to a normal search. Otherwise it stops the engine and gives the actual line of play (UCI also always sends the full move list instead of just the next move).

Quote:

- Lacking commands
* go searchmoves <move1> .... <movei>
I think this can be useful for analysis. But I think there should be possible to list partial moves. Like 3 steps and let the engine find the last step, rather than listing all possibilities. Also move sequenses could be useful for situations where we have found possible refutation to some moves.
* go nodes <x>
* go mate <x>
Also useful for analysis.

All of these would certainly be useful. A couple of things though.

I don't think nodes is something that should be a subcommand of go because it may not make sense for non-traditional searchers. I think it would make sense to make it a standard option like depth (for a fixed depth search) and hash (for the hash table size), etc..

Multiple arimaa moves in a single command as is used in searchmoves is somewhat problematic as it quickly makes for very long lines and it becomes difficult for a human looking at the protocol while debugging. Also you have to use some delimiter other than space since the moves will contain spaces and can be a variable number of steps. I think it would be better to implement this as a separate command that would send one move each time the command is given. Maybe something like "allowedmove <move>".

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by Oystein on Oct 27^th, 2008, 8:21am

on 10/26/08 at 19:42:29, Janzert wrote:

Looks like you have most of it. This would make a good base to build from for the protocol doc. Mind if I use it?

Of course not.

Quote:

It is currently unimplemented but eventually (i.e. once there is a gui controller) I want to add something similiar to uci where the engine can tell the controller what options are available.

Yes, this is meant to set all configuration options for the engine. So while there should be a certain predefined set of options that are standard across all or most engines an engine is also free to define its own as well (e.g. besides timecontrols and hash size opfor allows the weighting given to different evaluation components to be changed).

Sounds good.

Quote:

* checkeval <"current" | position>
- Ask for evaluation of current or given position

This is something I added for debugging OpFor and I'm not sure whether it should stay in or not. It's meant to tell the engine to statically evaluate either the current or given board position and report back any debugging information and the score (e.g. OpFor also reports the score for each component of the eval). I suppose it would be useful for anything that uses a traditional evaluation function but not something like a monte-carlo searcher.

I think this is to specific to be a standard command, but what about allowing generic commands like options are handled?
Engine sends: addcommand checkeval
GUI sends: usecommand checkeval
Just thinking loud.

Quote:

I agree.
But if the engine sends allowed option with the option command, then the gui can check the spelling.

Quote:

Yes.

Quote:

With arimaa the timecontrol system is a little more complex (and better developed in my opinion). Most of this information is set at the beginning of the game so was more natural to send as options. I also didn't like using the go command to tell the bot search to a fixed depth as that only really applies to traditional alpha-beta search. Another thing is I wanted to be able to update the time used for a move so far throughout the search in case of more accurate information coming in later because of network lag and such. So I ended up deciding to take all the timing and search depth settings off of the go command. I do admit that using setoption for things like the reserve time available and time used so far in a move does feel a little odd, but I think it's worth it since any separate command is going to basically just duplicate the functionality of setoption.

You have obviously thought more about this than me.

Quote:

Then I think it is clear that the UCI solution is not the best for AEI.

Quote:

I see your point. Since you included 'go depth', I think it was natural to add 'go node'. But that argument does not hold any more.

Quote:

Multiple arimaa moves in a single command as is used in searchmoves is somewhat problematic as it quickly makes for very long lines and it becomes difficult for a human looking at the protocol while debugging. Also you have to use some delimiter other than space since the moves will contain spaces and can be a variable number of steps. I think it would be better to implement this as a separate command that would send one move each time the command is given. Maybe something like "allowedmove <move>".

Good point.

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Oct 31^st, 2008, 2:47pm

I added protocol description and readme documents. The protocol document is still missing an example and probably needs other work but will hopefully ease understanding of the protocol.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Mar 22^nd, 2009, 3:18pm

Last night I added a fairly minimal implementation of an engine as an example. It can be found in simple_engine.py and will play moves by choosing random steps.

While writing this I noticed a few small things I plan to change in the protocol soon.

The setposition command specifies the side as w or b, this should change to allow either w,b or g,s with the intent to only use g,s at some point in the future.

The setoption specifies standard options of lastmoveused and tcmoveused. In the gameroom interface they are actually both currently prefixed with tc (tclastmoveused and tcmoveused). Since these are not actually part of the timecontrol but are rather informational I plan to remove the tc prefix from both.

The go command specifies an infinite option. I plan to remove this as it is equivalent to no time control information set and the depth left as unlimited.

I also am not quite satisfied with the info command right now as there is no way to group several information messages together into one snapshot. I may fix this by adding an option id or timestamp value directly after the info command and before the type field. I'll probably hold off on that until I get further along with a gui client and see if or how much of a problem it becomes.

If anyone else has run into things that are awkward or irritating let me know so I can see if there is a way to make it better. Any changes that should be made I would like to get in very soon before there is much relying on the current protocol.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by jdb on Mar 22^nd, 2009, 5:20pm

Quote:

The go command specifies an infinite option. I plan to remove this as it is equivalent to no time control information set and the depth left as unlimited.

I agree that "go infinite" is not needed.

However, I think the time control / depth info should be specified or have a defined default behaviour.

For example the command sequence,

setposition blah blah
go

What should the engine do?

Title: Re: Arimaa Engine Interface (AEI)
Post by jdb on Apr 21^st, 2009, 9:30am

Janzert, any idea what this is about?

Quote:

2009-04-21 12:49:30 INFO:gameroom:Received move 24w Ef4e Dg3w Df3w Rc1n
2009-04-21 12:49:30 INFO:gameroom.position:
24b
+-----------------+
8| r r d r r r |
7| r d r |
6| X c X c R |
5| H e |
4| R E h |
3| X R D X R |
2| R R R C |
1| R |
+-----------------+
a b c d e f g h

2009-04-21 12:49:44 INFO:gameroom:Sending move cg6n Rh6w Rg6w Rf6x cg7s
2009-04-21 12:49:45 INFO:gameroom.position:
25w
+-----------------+
8| r r d r r r |
7| r d r |
6| X c X c |
5| H e |
4| R E h |
3| X R D X R |
2| R R R C |
1| R |
+-----------------+
a b c d e f g h

2009-04-21 12:50:45 INFO:gameroom.net:empty response from server will try again, sleeping for 0 seconds
2009-04-21 12:51:05 INFO:gameroom.net:Received URLError, reason type <class 'socket.gaierror'>
2009-04-21 12:51:05 INFO:gameroom:Leaving game
2009-04-21 12:51:05 INFO:gameroom:Removed run file at ./running/125143b.bot
2009-04-21 12:51:27 ERROR:gameroom:Caught unkown exception #1, restarting.
Traceback (most recent call last):
File "./gameroom.py", line 803, in main
table.playgame(engine_ctl, bot_greeting, options['onemove'])
File "./gameroom.py", line 295, in playgame
state = self.updatestate(30)
File "./gameroom.py", line 195, in updatestate
gamestate = post(self.url, values, "Table.updatestate")
File "./gameroom.py", line 79, in post
if hasattr(err, "reason") and len(err.reason) > 0 and err.reason[0] == 10060:
TypeError: object of type 'gaierror' has no len()

2009-04-21 12:51:32 ERROR:gameroom:Could not find game against 125143 with side 'b'

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Apr 21^st, 2009, 10:06am

It looks like there was initially a network error using an exception that I failed to expect or handle correctly causing the gameroom interface itself to throw a TypeError exception. It then tried to restart but was unsuccessful in reconnecting to the game.

It looks like the initial error 'socket.gaierror' should only get thrown on server hostname lookup or address errors. Unfortunately though because of the second error the details of it were lost. I've reworked the error handling there slightly to hopefully better handle unexpected errors, so hopefully at least good information can get recorded if it happens again.

Janzert

p.s. I noticed in chat you mentioned about the current interface not reporting the various extra information about the game (opponent, rated, etc.); I'll try and get that added by the end of the week.

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Jul 6^th, 2009, 9:50pm

End of the week turned into 2 months later but finally I've added opponent and game rating status reporting to the gameroom interface.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Aug 4^th, 2009, 1:43pm

I have a draft for an updated protocol document that can be found here (http://arimaa.janzert.com/aei-protocol-draft.html). Most of the changes are simply clarifications from the previous version. The primary change, added after Omar's encouragement, is the addition of a message (protocol-version) from the engine to the controller telling the controller what protocol version the engine expects. This will allow future controllers to remain compatible with old engines.

There are two more additions I would eventually like to make. The first is a way for engines to tell controllers what options it supports (similiar to UCI "option" message). The second is a standard method to tell an engine to only search certain moves from a position. But both of those will probably wait till a later version.

If there is anything else that you see needing clarification or changing please speak up now, I want to make a real release soon so it will be easily available for developers working on bots for the 2010 CC.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by jdb on Sep 28^th, 2009, 5:41am

on 07/06/09 at 21:50:56, Janzert wrote:

End of the week turned into 2 months later but finally I've added opponent and game rating status reporting to the gameroom interface.

Janzert

I am working on adding this into clueless. Something is not working.

In the gameroom.py file, line 303 starts a block that sends the rating and opponent name info to the engine. There is a similar block starting on line 324. They are not the same. When I cut and pasted to make them the same, everything worked properly. Is this a bug, or am I doing something wrong?

Title: Re: Arimaa Engine Interface (AEI)
Post by jdb on Sep 28^th, 2009, 5:59am

The AEI sends a message to the engine telling if the game is rated or not. It sends the message right after the game has been opened in the gameroom and no one has joined the game yet. This works fine if the opponent is playing in rated mode. If the opponent is playing in unrated mode, the server forces the game to be unrated. This currently happens after the message has already been sent to the engine. Is it possible to have the AEI handle this situation?

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Sep 28^th, 2009, 10:29pm

Thanks for the bug report and fix, revision 35 should include the fix.

Regarding unrated mode; I had pretty much forgotten about that mode again (I think my mind instinctively blocks it out since I dislike it so much :/). I'll have to look at when the server unrates the game and move the report to after that or possibly report it again when it changes. Although I'm not sure there is much utility in the latter¹.

The rated status of the game is currently reported at line 829 copying/moving that line to the two sections you found where the ratings are reported would probably also give the correct status. But I don't have time to test it at the moment.

Janzert

¹ Except for allowing me to harbor fantasies of spitting out a message telling the player "This bot refuses to participate in unrated mode games." and leaving the game.

Title: Re: Arimaa Engine Interface (AEI)
Post by jdb on Sep 29^th, 2009, 1:39am

on 09/28/09 at 22:29:24, Janzert wrote:

I'll try it out.

Edit: Tried it, looks like its more complicated than I can figure out.

Title: Re: Arimaa Engine Interface (AEI)
Post by Fritzlein on Sep 29^th, 2009, 5:26am

on 09/28/09 at 22:29:24, Janzert wrote:

¹ Except for allowing me to harbor fantasies of spitting out a message telling the player "This bot refuses to participate in unrated mode games." and leaving the game.

Why not do that, or use the random-setup strategy? It seems to me developers should have the right to choose between rated and unrated games as much as their opponents.

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Sep 30^th, 2009, 7:06pm

First thing I should note is that I had forgotten to announce that I set up a real project for AEI on launchpad and the latest code can now be found at https://code.launchpad.net/~janzert/aei/trunk or if using bzr directly "bzr branch lp:aei" will get it.

The interface now will resend the games rated status after the opponent joins. This should be the status the game stays unless the player unrates it after the game. By keeping track if the status changes the engine can also tell if the game was started unrated or if unrated mode is being used.

on 09/29/09 at 05:26:30, Fritzlein wrote:

Why not do that, or use the random-setup strategy? It seems to me developers should have the right to choose between rated and unrated games as much as their opponents.

I had the impression that cutting off the game was considered very anti-social by the community.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Nov 2^nd, 2009, 8:40pm

AEI site now up at http://arimaa.janzert.com/aei/.

I also put up a 1.0beta release there. I don't expect any more changes in the protocol before the final release. Mostly I need to get a lot more documentation before the final 1.0 release. Hopefully I can also do some code clean up in the implementation.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by omar on Nov 5^th, 2009, 8:36am

Thanks for all the effort you've put into this project. I know having this will make thing much easier for bot developers. I have updated the pages which guide new bot developers to recommend using the AEI. I would strongly recommend that all bot developers start transitioning to use the AEI instead of the Bot Interface Kit.

Title: Re: Arimaa Engine Interface (AEI)
Post by Polyfractal on Dec 19^th, 2009, 11:39am

Just getting around to implementing this. Perhaps I'm just really dumb, but roundrobin.py does not seem to be running on my system. Any ideas?

This is the output I get:

Code:

C:\Zach\DistributedNeuron\aei-1.0beta1>roundrobin.py
Number of rounds: 1
At timecontrol 3s/30s/100/60s/10m
Giving these settings to all bots:
hash: 50
Traceback (most recent call last):
File "C:\Zach\DistributedNeuron\aei-1.0beta1\roundrobin.py", line 277, in <mod
ule>
sys.exit(main(sys.argv))
File "C:\Zach\DistributedNeuron\aei-1.0beta1\roundrobin.py", line 250, in main

gengine = run_bot(gbot)
File "C:\Zach\DistributedNeuron\aei-1.0beta1\roundrobin.py", line 243, in run_
bot
engine = EngineController(StdioEngine(cmdline))
File "C:\Zach\DistributedNeuron\aei-1.0beta1\pyrimaa\aei.py", line 64, in __in
it__
universal_newlines = True)
File "C:\Python26\lib\subprocess.py", line 621, in __init__
errread, errwrite)
File "C:\Python26\lib\subprocess.py", line 830, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Dec 19^th, 2009, 12:08pm

Unfortunately the error messages are usually pretty unfriendly tracebacks, sorry. :(

In this case it is unable to start a bot using the command line given in roundrobin.cfg. Specifically it looks like the first bot specified in the bots list for the tournament.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by Polyfractal on Dec 19^th, 2009, 12:21pm

Doh! Silly me, I was thinking the cmdLine option was what you wanted passed to the bot as arguments, not the actual command to start the bot.

Fixed and works great now. Thanks!

Title: Re: Arimaa Engine Interface (AEI)
Post by rbarreira on Mar 25^th, 2010, 6:19am

Hi,

Thanks a lot for AEI, I'm just now starting a bot using it!

How many of the bots are currently using it? Janzert I suppose your own bots are, so AEI has probably been tested quite a lot.

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Mar 25^th, 2010, 10:40am

Of the current active bots the only two I know for sure are using AEI are Clueless and OpFor. I have used AEI exclusively since I started developing OpFor. Jdb switched Clueless over from the getmove interface in the last year.

One thing I do want to add at some point is a method like UCI has for the engine to tell the controller what options are available. I've felt that this isn't very important or useful until a GUI controller is developed though.

Other than that the current protocol has met my needs, but I'd love to hear from others if there are changes or additions that would work better for them.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by 99of9 on Mar 25^th, 2010, 6:33pm

Hi Janzert,
If you have a set of instructions that says how to convert from the old bot script to the AEI, I would probably do it. I would prefer not to make any significant program changes at this point, so would not make use of any of your new functionality (e.g. pondering). Is there a way to just run a new executable each move?

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Mar 26^th, 2010, 7:30am

While I think it is pretty close to the same degree of difficulty to start a bot using either interface, it's probably harder to convert between them. Because the assumptions an engine will inevitably make are going to be quite different. But I believe Clueless is the only bot that has been converted between the two so I'm sure Jdb could comment on this better than I can. So unless you want to take advantage of something AEI makes easy (pondering, not restarting every move) I don't think it is worth the effort of converting.

Also I'm planning on writing an adapter that will run getmove based engines under an AEI controller. Although the primary use I see for this is to run getmove bots under an AEI gui.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by jdb on Mar 26^th, 2010, 12:23pm

Here is the code from clueless to interface to gtp.

The indentation messed up. I have no idea how to fix it.

Code:

package arimaa3;

import ai_util.*;

import java.util.*;
import java.io.*;
import cataract.GtpCommand;

public class ArimaaEngineInterface {
public ArimaaEngineInterface() {
}

private GameState initial_position = new GameState();
private ArrayList<String> move_list = new ArrayList<String>();
private EngineThread engine_thread = new EngineThread();
private ArimaaEngine engine = new ArimaaEngine("clueless.cfg");
private ArimaaServerInfo info = new ArimaaServerInfo(initial_position);

// All messages must by sent thru here so they get logged
private void send_message(String text) {
text += "\n";
LogFile.message("AEIs--> " + text);
System.out.print(text);
System.out.flush();
}

private void control() {

BufferedReader reader = new BufferedReader(new InputStreamReader(System.in));

while (true) {
try {
// Get the message
String message = reader.readLine();

// Write message to log file
LogFile.message("AEIr--> " + message);

// If connection broken, just shutdown
if (message == null) {
engine.shutdown_search();
return;
}

GtpCommand command = new GtpCommand(message);

// Figure out which command it is
if (command.command.equals("aei")) {
send_message("id name clueless");
send_message("id author Jeff Bacher");
send_message("id version 2009");
send_message("aeiok");
}

else if (command.command.equals("isready")) {
send_message("readyok");
}

else if (command.command.equals("quit")) {
engine.shutdown_search();
return;
}

else if (command.command.equals("stop")) {
engine.abort_search();
}

else if (command.command.equals("newgame")) {
engine.abort_search();
initial_position = new GameState();
move_list.clear();
}

else if (command.command.equals("makemove")) {
String move_text = command.getRestOfCommand();
move_list.add(move_text);
}

else if (command.command.equals("setposition")) {
String position_text = command.getRestOfCommand();
initial_position = new GameState(position_text);
move_list.clear();
}

else if (command.command.equals("setoption")) {
String name = command.arguments.get(1);
String value = command.arguments.get(3);
set_option(name,value);
}

else if (command.command.equals("go")) {
if ( command.getRestOfCommand().equals("ponder") ) {

}
else {
engine_thread.run();
}
}

// Try sending command on to the engine
else {
// Unknown command received, just log it and ignore the command
}

} // Loop forever

// This should never happen!
// We're sunk if we get here, just exit
catch (Exception e) {
StringWriter sw = new StringWriter();
e.printStackTrace(new PrintWriter(sw));
String stackTrace = sw.toString();
LogFile.message("Exception " + e);
LogFile.message(stackTrace);
}
}

}

private void set_option(String name, String value_text) {
name = name.toLowerCase();

if ( name.equals("rated") ) {
info.is_rated = value_text.equals("1") ? true : false;
}

if ( name.equals("opponent")) {
info.enemy_name = value_text;
}

if ( name.equals("tcmove") ) {
int value = Integer.parseInt(value_text);
info.tc_move = value;
}

if ( name.equals("tcpercent") ) {
int value = Integer.parseInt(value_text);
info.tc_percent = value;
}

if ( name.equals("tcmax") ) {
int value = Integer.parseInt(value_text);
if (value == 0 ) {
value = Integer.MAX_VALUE;
}
info.tc_max_reserve = value;
}

if ( name.equals("tcturntime") ) {
int value = Integer.parseInt(value_text);
if (value == 0 ) {
value = Integer.MAX_VALUE;
}
info.tc_max_turn_time = value;
}

if ( name.equals("wreserve")) {
int value = Integer.parseInt(value_text);
info.tc_white_reserve = value;
}

if ( name.equals("breserve")) {
int value = Integer.parseInt(value_text);
info.tc_black_reserve = value;
}

}

private void populate_arimaa_server_info() {

// Get the current gamestate
GameState current_position = new GameState();
for( String move_text : move_list ) {
ArimaaMove move = new ArimaaMove(move_text);
current_position.play(move,current_position);
}
info.position = current_position.toEPDString();
info.side_to_move = current_position.getSideToMove();

// Create the move list
String move_list_text = "";
int half_move_count = 0;
for( String move_text : move_list ) {
move_list_text += getMoveIdentifier(half_move_count);
move_list_text += " "+move_text;
move_list_text += "%13";
half_move_count++;
}

move_list_text += getMoveIdentifier(half_move_count);
info.move_list = move_list_text;

}

private String getMoveIdentifier(int half_move_count) {
String result = "";

int move_number = half_move_count/2 + 1;
int side_to_move = move_number & 0x01;

result += move_number;
result += (side_to_move == 0) ? "w" : "b";

return result;
}

// Separate thread to run the engine
private class EngineThread implements Runnable {
public void run() {
long start_time = System.currentTimeMillis();

populate_arimaa_server_info();

MoveInfo move = engine.genMove(info);
// remove any pass words, as arimaa-online doesn't want them
String final_move = move.move_text.replaceAll(" pass", "");
send_message("bestmove " + final_move);

long elapsed_time = System.currentTimeMillis() - start_time;
LogFile.message("Elapsed time: "+elapsed_time+" ms");
}
}

public static void main(String args[]) {
LogFile.setMessageDisplay(false);
LogFile.message( (new Date()).toString());

ArimaaEngineInterface main = new ArimaaEngineInterface();
main.control();
}

}

Title: Re: Arimaa Engine Interface (AEI)
Post by rabbits on Mar 27^th, 2010, 4:47pm

on 03/25/10 at 18:33:19, 99of9 wrote:

Is there a way to just run a new executable each move?

I just added a simple getmove adapter script to my GUI project. I have used it with bot_Score1 and bot_Fairy, so maybe it will work with bot_Gnobot.

Title: Re: Arimaa Engine Interface (AEI)
Post by Polyfractal on Jun 18^th, 2010, 10:48am

Quick question regarding AEI since I haven't worked with it in ages (haven't programmed in ages now that I think about it):

How does AEI handle illegal moves when playing locally bot-against-bot? Does it terminate the game or report an error and allow the bot engine to submit another attempt?

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Jun 21^st, 2010, 1:24pm

AEI the protocol doesn't specify what the controller should do when receiving an illegal move. This was explicit and I can give the reasoning for what I think should be done in a few specific cases, but I don't think this is what you're trying to find out.

Since you specify playing locally I assume you're wondering how the "roundrobin.py" script handles illegal moves. Currently it doesn't do any explicit checking, but it does track the current state of the board so it can tell when the game has ended. This should result in a large number of illegal moves being caught and cause an exception to be thrown. But there are also a number of cases that will just silently be accepted. This is certainly something that can and should be improved. Both to catch all illegal moves and to properly assign the result.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by FireBorn on Jun 21^st, 2010, 4:07pm

I am curious why there is no checking for illegal moves.

I looked into using AEI to evolve my bots, but if I did they would be getting away with all kinds of illegal moves!

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Jun 21^st, 2010, 9:49pm

on 06/21/10 at 16:07:26, FireBorn wrote:

I am curious why there is no checking for illegal moves.

Why there is no checking where? Specified in the protocol, implemented in the gameroom interface, implemented in roundrobin.py, or one of the other controller scripts included in the standard implementation?

The protocol doesn't specify checking for a couple of different reasons. Two big ones are: the appropriate response by a controller varies from one controller to another depending on how that controller is being used, and although the name may not imply it the AEI protocol is quite generic and at the vary least should handle many Arimaa variants without any change.

The main reason the gameroom interface does no checking is because one of the highest priorities for it is "no game should ever be lost by an engine because of the interface". Adding any sort of checking would always open the possibility of rejecting a move that the server would have accepted (maybe the server is actually playing a variant ;) ).

The only reason roundrobin.py doesn't check is simply that it's never been added. The underlying board representation already has methods to check a moves legality, the appropriate calls to actually check just need to be added to roundrobin.py. I've just never needed it although I'm sure I'll probably get to it. But I won't guarantee the legality check methods are fully correct. I've only used them in a very rudimentary GUI I started a while back. Rabbits has also sent some fixes for them though, so I imagine he must be using them as well. Although I don't know how extensively or for what purpose.

Quote:

I looked into using AEI to evolve my bots, but if I did they would be getting away with all kinds of illegal moves!

Certainly the AEI protocol would be perfectly suitable for controlling evolving bots. Also quite certainly none of the controllers included in the standard implementation are suitable for doing so, even if they did check for move legality.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by FireBorn on Jun 22^nd, 2010, 8:36am

Thanks, I get it now

on 06/21/10 at 21:49:48, Janzert wrote:

I ended up using Chimaera, which has a lot of illegal move checking, but even then I had to add some checking to ensure proper move notation.

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Jun 23^rd, 2010, 8:00pm

Great; so it sounds like what you needed was more of a general engine.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by rbarreira on Jun 28^th, 2010, 1:37am

I've noticed that my bot cannot play games that have already started (for example, if the bot crashes the gameroom controller attempts to restart it; this probably also happens if trying to move on a postal game, though I haven't tried).

I'm thinking that this is due to the wrong assumption that it will always receive a "go" command either asking for a gold or a silver setup. In fact this is how my bot today can tell whether it's supposed to play as gold or silver.

I suppose that to fix this, the only correct way to know whether we're playing Gold or Silver is to wait for the first "go" or "go ponder" command. Is there an easier way, especially something that tells us our side before moves start getting sent with the makemove command?

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Jun 28^th, 2010, 7:30am

The only way to determine what side the engine should think about is to check the current state of the game. Even after the first "go" command it shouldn't assume that it will always be asked to think about the same side after that or that the move it returns will be the actual move played.

And yes from the engine's perspective with the current interface joining an ongoing postal game will look the same as joining or rejoining an ongoing live game. After starting the engine and initializing it with the game settings the interface with "catch the engine up" by sending it a series of makemove commands for all the moves of the game already played.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by rbarreira on Jun 28^th, 2010, 7:35am

Thanks for the reply. Except for the input/output of Arimaa notation, my code is not based on gold/silver sides but instead on my/opponent sides, so I guess I'll have to work around this in some way.

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Jun 28^th, 2010, 10:55am

My thought on how to handle it in that case would be to reverse the board after every makemove command and keep a separate field to track what color's turn it actually is.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by doublep on Jul 4^th, 2010, 9:20am

I started working on AEI support in Badger. However, I can't get to start roundrobin script:

Code:

cmdline = /home/paul/badger/tools/gtp-aei.py --command '/home/paul/badger/badger -T all'

Causes the following failure in (my own) 'gtp-aei.py' script:

Code:

getopt.GetoptError: option -T not recognized

The intention is to pass -T as part of --command value, not as a separate argument. I guess the problem is that subprocess.Popen() in 'aei.py' is invoked without shell = True argument. Can we have that as an option? I'd hate to split configuration in a hundred different files and would prefer to pass most of it through command line.

Also, if 'roundrobin.py' fails for any reason, including Ctrl-C abortion, it should kill all spawned engine processes. I had about 8 'simple_engine.py' running and wondered wtf my system was so slow...

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Jul 11^th, 2010, 4:00am

on 07/04/10 at 09:20:49, doublep wrote:

I really thought I used to have shell = True but ran into some problems. But I can't find any evidence of this actually being the case, so I've gone ahead and switched to using the shell.

Quote:

Also, if 'roundrobin.py' fails for any reason, including Ctrl-C abortion, it should kill all spawned engine processes. I had about 8 'simple_engine.py' running and wondered wtf my system was so slow...

I agree this is certainly bad behaviour. I added some automatic cleanup in aei.py in case a caller fails to do it, this should take care of the problem once and for all.

Both of the above changes are available in revision 67. I want to add full move legality checking to roundrobin.py and then I'll probably do another full release.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by jdb on Jul 12^th, 2010, 10:06am

Janzert,

I made some changes to roundrobin.py to create a PGN log file that is compatible with BayesElo. Is there some way I can send them to you?

Jeff

Title: Re: Arimaa Engine Interface (AEI)
Post by doublep on Jul 12^th, 2010, 11:42am

Janzert, I actually made a merge proposal through Launchpad, but given you already implemented this yourself, it probably doesn't matter.

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Jul 12^th, 2010, 1:21pm

on 07/12/10 at 10:06:19, jdb wrote:

Janzert,

I made some changes to roundrobin.py to create a PGN log file that is compatible with BayesElo. Is there some way I can send them to you?

Jeff

That sounds great. It's something I've been meaning to do for quite some time but have never gotten around to.

The "best" way to submit changes should* be through a launchpad merge proposal, but if you don't use bzr just emailing me (janzert@janzert.com) a diff or even the whole script would be fine.

* see below :/

Quote:

Janzert, I actually made a merge proposal through Launchpad, but given you already implemented this yourself, it probably doesn't matter.

Argh sorry and thanks for doing that. I thought launchpad was suppose to notify me about merge proposals with an email. I'll have to see if there is a setting to do that. In any case it may be best if people can drop me an email or note here on the forums when sending a proposal since I don't check the launchpad pages directly very often.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by jdb on Jul 12^th, 2010, 5:35pm

I emailed you the file with the BayesElo changes.

Title: Re: Arimaa Engine Interface (AEI)
Post by rbarreira on Aug 1^st, 2010, 2:15pm

I think I'm going to update the roundrobin script to write the moves to the PGN file, instead of just the game score. Unless someone is already doing it, that is.

If anyone is interested in the changes let me know.

edit - it's done and it seems to be working

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Aug 1^st, 2010, 8:00pm

Sorry, maybe I should have said something here.

I'm currently working on refactoring the roundrobin script with the plan to add writing the full moves to the pgn file and allowing for at least limited ability for the individual engines to be given different time controls.

If there is anything else that people really would like to see it able to do let me know and I'll see if I can add it in while I'm doing the rewrite or at the very least make sure new structure doesn't make it hard to add.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by jdb on Aug 1^st, 2010, 9:19pm

The roundrobin script uses strict checking on all the moves. In order to run handicap games, I disabled the strict checking. This let the bots send an incomplete setup move. As long as this stays simple to change, I'll be happy.

Very useful script. Thanks for making it available.

Title: Re: Arimaa Engine Interface (AEI)
Post by rbarreira on Aug 2^nd, 2010, 2:42am

on 08/01/10 at 20:00:47, Janzert wrote:

If there is anything else that people really would like to see it able to do let me know and I'll see if I can add it in while I'm doing the rewrite or at the very least make sure new structure doesn't make it hard to add.

As long as we're brainstorming, one feature that I was thinking about would be to have a bunch of starting positions to cycle through (probably a simple file with line pairs for the gold/silver setup), making sure that each bot gets to play every starting position against every other bot with both colors (as long as the number of rounds allows it of course). The starting positions could be illegal setups too, for example starting from a middlegame or even an endgame.

This may not be needed though. The objective would be to add more variety to the tournament, but maybe it's a moot point once the bots have randomized setups (though the feature above would have the advantage of removing luck out of the process).

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Aug 2^nd, 2010, 1:46pm

Having a list of starting positions is something I think would definitely be useful. The reason I haven't added it so far though is that it doesn't seem like a clean fit with a round robin tournament. Also for testing I think most of the time a one-vs-all tournament is generally what is wanted. The question I've been pondering is whether roundrobin.py should be expanded to run multiple tournament types or to create a separate script per type? Right now I'm leaning toward separate simple scripts.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Aug 2^nd, 2010, 1:46pm

As of revision 76, roundrobin.py will now write the full move list to the pgn file and has an option to disable strict legality checking for setup moves.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by rbarreira on Aug 2^nd, 2010, 2:15pm

Thanks for the new version, the move list in the pgn is better than what I had implemented as it has the move numbers as well. I suppose that makes it possible to use with this tool:

http://arimaa.com/arimaa/games/planGame.cgi

I haven't been able to try yet because for some reason the planGame tool doesn't work in Linux.

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Aug 2^nd, 2010, 2:37pm

yep, copying and pasting the moves into the plan game form works fine here.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by doublep on Sep 18^th, 2010, 5:26pm

Janzert, can you please add some functionality similar to 'botman' to 'gameroom.py'? I.e. I'd like to be able to launch the bot and have it continuosly playing games until stopped. As far as I see from the source code, 'break' in main() after a game is unconditional.

Also, 'gameroom.py' script shouldn't print a traceback on Ctrl-C at least unless there is a different clean way of stopping a bot. I guess just guarding main() call with try: ... except KeyboardInterrupt: pass should be enough.

Title: Re: Arimaa Engine Interface (AEI)
Post by rbarreira on Sep 19^th, 2010, 5:15am

on 09/18/10 at 17:26:39, doublep wrote:

I wrote this script to play games until stopped:

Code:

#!/bin/bash
while [ 1 ]
do
./gameroom.py s
./gameroom.py g
done

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Sep 19^th, 2010, 2:12pm

yeah, gameroom.py corresponds to the 'bot' script in the original bot interface so it doesn't have any facilities for running multiple games. Here's the simple python script I use to keep it running forever (forums are destroying the indentation so it's here (http://pastebin.org/962798) on pastebin). I suppose I could easily add this to the AEI package if desired.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by doublep on Sep 19^th, 2010, 3:18pm

Please add that to the package. While it is pretty trivial to use a shell-script around that (even a oneliner directly from command line), I'd like to be able to control e.g. whether it continues to play if the bot crashes or loses on time, things like that. Because in such situations bot probably needs to be stopped and fixed, while otherwise (normal losses etc.) it should keep playing.

Title: Re: Arimaa Engine Interface (AEI)
Post by doublep on Sep 21^st, 2010, 3:22pm

It seems AEI doesn't provide a lot of information about the game to the engine. As far as I understand, everything comes through 'setoption' command. Some things I miss:

* game identifier;
* move number; usually this would be possible to reconstruct counting 'makemove' (I think, didn't try), but this would be broken if 'setposition' is issued, e.g. if resuming a game;
* own name; not important for Badger, but I imagine some engines may choose to behave differently depending on their current nickname.

Title: Re: Arimaa Engine Interface (AEI)
Post by rbarreira on Sep 21^st, 2010, 4:34pm

Resuming a game with the current scripts is done with makemove commands if I understood correctly (which are necessary for reconstructing the repetition history anyway).

In my bot I don't support the setposition command, and I'm not planning to unless there's some advantage to it. I guess it could be useful if a GUI used it.

Title: Re: Arimaa Engine Interface (AEI)
Post by lavalamp on Dec 15^th, 2010, 1:07pm

I'm playing around with making a bot and I found a bug in roundrobin.py--

Due to a bug in my code, my bot tried to play 4 steps for itself and one for its opponent-- and roundrobin.py is perfectly happy to let it do so!

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Dec 15^th, 2010, 4:41pm

Heh, yep looks like the legality checking doesn't make sure you stop at the end of your turn.

If you want to try out a quick fix, in the file pyrimaa/board.py in the do_move method after line 659 "pos = self" add the following lines:

Code:

if len(steps) > 4:
raise IllegalMove("Too many steps in move")

That should catch a bot trying to extend control beyond its turn. ;)

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by doublep on Jan 29^th, 2011, 8:24am

Ping @reply 62. Can gameroom.py be improved to provide at least some of that information?

Title: Re: Arimaa Engine Interface (AEI)
Post by doublep on Jan 30^th, 2011, 11:06am

Code:

$ ./gameroom.py play b
$

No error given, yet no game is started. In the logs I see: ERROR:gameroom:Could not find game against b with side ''. It is not appropriate when a program fails without any message.

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Jan 30^th, 2011, 12:51pm

on 01/30/11 at 11:06:39, doublep wrote:

Code:

$ ./gameroom.py play b
$

No error given, yet no game is started. In the logs I see: ERROR:gameroom:Could not find game against b with side ''. It is not appropriate when a program fails without any message.

Hmm, here I get:

Code:

D:\MyStuff\Arimaa\aei_work>gameroom.py play b
Setting aei protocol to version 1
Setting bot option hash = 800
Setting bot option threads = 4
Could not find game against b with side ''

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by doublep on Jan 30^th, 2011, 12:58pm

I guess it depends on logging settings. I have this: http://pastebin.com/4LzQjww0

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Jan 30^th, 2011, 1:14pm

on 09/21/10 at 15:22:42, doublep wrote:

It seems AEI doesn't provide a lot of information about the game to the engine. As far as I understand, everything comes through 'setoption' command. Some things I miss:

* game identifier;

There is no permanent game identifier to give during the game. After the game is finished it does try and retrieve the game id and write it to the file but this doesn't seem to be completely dependable either.

What are you wanting it for? I think any database updating or learning should be done by a separate program, or at least run, rather than having the engine do it at the end of the game.

Quote:

* move number; usually this would be possible to reconstruct counting 'makemove' (I think, didn't try), but this would be broken if 'setposition' is issued, e.g. if resuming a game;

Yes, the correct way with the current protocol is to count the moves given to the engine. Even when resuming a game if there is a move history for the bot to take into account it will be caught up to the current position by having all the moves replayed to it.

In a future version of the protocol I could see changing the makemove command to including the move number and side information (e.g. 25s). I'm not completely convinced it should but I suppose it does provide an extra check that the controller and engine both have the same idea on the state of the game. Along that same line I could see adding a command for use in debugging to have the engine reply with what it thinks the current position is.

Quote:

* own name; not important for Badger, but I imagine some engines may choose to behave differently depending on their current nickname.

Since you can arbitrarily give a bot any option it wants to accept, I believe that is a better way to go about doing this. The options included in the protocol should just be a very basic set that every engine can handle. For sure one of the things that will be added in a future version of the protocol is a way for engines to let the controller know what (additional?) options it will accept. This is probably primarily useful for GUI controllers though.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Jan 30^th, 2011, 1:22pm

on 01/30/11 at 12:58:37, doublep wrote:

I guess it depends on logging settings. I have this: http://pastebin.com/4LzQjww0

I haven't looked at that code in quite a long time, but I would guess it is because you have the console logging (which I'm pretty sure is really console output) disabled. Try something like "console = True" and "console_level = Warn" (or Error). Completely disabling it is probably only useful when having gameroom.py run from another script and you don't want it writing to stdout at all.

One thing that should be improved is that I don't think it sets any error code in its exit status.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by doublep on Jan 31^st, 2011, 1:50pm

on 01/30/11 at 13:14:25, Janzert wrote:

For matching the logs created by the bot with game records on the server.

Quote:

Yes, the correct way with the current protocol is to count the moves given to the engine.

Can't AEI's gameroom just do this for me and send result as another 'setoption'? I mean, it's just another not-strictly-needed information like opponent's name, so 'setoption' seems like a good candidate for it. It could be used independently of number of 'makemove's (i.e. the real move number) to ask an engine to think as if it's late game already etc.

Quote:

Try something like "console = True" and "console_level = Warn" (or Error). Completely disabling it is probably only useful when having gameroom.py run from another script and you don't want it writing to stdout at all.

Sorry, but a logging level ought to have no effect on such things. It should not matter whether I disabled console output or not --- on such fatal errors the script should write to stderr unconditionally.

I generally have problems with AEI in that it tries to shoehorn engines into logging through it. This is absolutely not needed for communication with the server. I have some logging code in Badger and prefer to keep that rather than switching to AEI logging --- that's why I have console output from 'gameroom.py' turned off.

Title: Re: Arimaa Engine Interface (AEI)
Post by rbarreira on Feb 1^st, 2011, 4:07am

If a program has options for explicitly setting the console warning/error level, it seems pointless to argue that it's not outputting the errors you want to the console (unless of course you have set it to output errors and it's not doing so).

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Feb 2^nd, 2011, 10:00pm

on 01/31/11 at 13:50:46, doublep wrote:

I generally have problems with AEI in that it tries to shoehorn engines into logging through it. This is absolutely not needed for communication with the server. I have some logging code in Badger and prefer to keep that rather than switching to AEI logging --- that's why I have console output from 'gameroom.py' turned off.

AEI isn't meant just (or even necessarily primarily) for engines to communicate with the arimaa.com server. I hope someday that more developers will distribute their engines publically so that using desktop clients players can directly play those engines, use them for analysis, etc.. For this I think it would be quite frowned upon to log outside of the protocol. So in that sense yes for sure the AEI philosophy is for all engine output to be contained in the protocol.

But given the simplistic state of the current package scripts I can understand that people want to do their own logging. I'd rather not change the protocol to encourage it though. Much better would be to improve the current handling.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by rbarreira on Mar 16^th, 2011, 7:11pm

Since OpFor, clueless and briareus all use bot_threads to set the number of search threads, maybe it wouldn't be a bad idea to have this option name as an AEI standard (it seems as relevant to have a standard for it as it is to have the bot_hash standard).

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Mar 17^th, 2011, 1:38pm

Yep, this is probably a good idea. The only reason I didn't include it before was that it seems a fair number of chess engines use separate processes instead of threads. In which case it seemed like a bit of a misnomer.

But I guess it is the best common name, and I'll try to include a description in the spec that gives the more general usage (i.e. basically that it should set the limit on cores actively used).

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by rbarreira on Mar 28^th, 2011, 11:36am

When testing briareus with a postal game, for the first move (after the setup) I had to start it manually with the game id, the postal controller claimed that there was no game with the bot's turn to move.

I printed the game variables and it seemed that g['turn'] was an empty string which of course failed the "g['turn'] == g['side']" test.

In the next move everything was fine and the postal controller worked as expected.

I don't know if this was the server or some script's fault. Did you ever experience this problem in postal games?

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Mar 28^th, 2011, 12:07pm

Hmm, I don't recall ever seeing that before. I'm also surprised you were able to play a turn by running gameroom.py manually, since it also has a similar check*. I wonder if maybe the server happened to return the correct state when you ran gameroom.py directly?

Janzert

* line 278 of the latest gameroom.py

Title: Re: Arimaa Engine Interface (AEI)
Post by rbarreira on Mar 28^th, 2011, 1:44pm

I tried to run the postal controller many times so it didn't seem like a random bug.

I think I just managed to reproduce it:

1- Started a postal game myself as silver (bot will be gold)
2- Made bot join the game with ./gameroom.py play <game_id>
3- Bot sends gold setup.
4- I send silver setup.
5- Bot starts thinking on 2g.
6- CTRL+C to interrupt gameroom.py (checked that bot or gameroom.py wasn't running afterwards)
7- Started postal_controller.py several times, always got this output (including debug print I added):

Quote:

{'rated': '1', 'schts': '0000000000', 'createdts': '1301337301', 'turn': '', 'timecontrol': '1d/60d/100/0/300d/21d', 'player': 'rbarreira', 'plycount': '0', 'gid': '222602', 'turnts': '0000000000', 'postal': '1', 'side': 'w'}
Found 1 games with 1 postal games and 0 on my turn.
No postal games with a turn found, sleeping.

If, instead, step 6 is done before step 4, running postal_controller.py seems to work fine!

I just tried the steps above twice and they seem to reproduce the bug consistently. Running gameroom.py on the game manually gets the bot thinking on the move in any case.

Fortunately this use case is probably not very common since postal games are usually started asynchronously, so this is probably not a critical bug (I'm suspecting the server might be at fault though?).

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on Mar 28^th, 2011, 5:40pm

I think I've seen before that server seems to lag behind in updating the turn state for games listed in the postal games list (http://arimaa.com/arimaa/gameroom/postalgames.cgi). I have a feeling the game state the postal controller is looking at is from the same source on the server. I wonder if you leave the postal controller running does it see the turn after 10-15 minutes?

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by rbarreira on Mar 28^th, 2011, 6:12pm

on 03/28/11 at 17:40:49, Janzert wrote:

Yeah, on the second try (after 5 minutes) it started the bot, so no big deal especially since this is already rare enough...

I also noticed that I had to try twice this time to reproduce the bug, so there's probably some timing/scheduled job thing at play here.

Thanks for the help!

Title: Re: Arimaa Engine Interface (AEI)
Post by Hippo on May 16^th, 2011, 3:14am

I am planning to change gameroom.py script for myself to allow to connect to open games without need of explicit gamenumber or opponent name. Of course it would be annoing for other players not wanting to be played by the bot so there would be automatic white/black list of opponents to join this way.

I think about resignation on bot's turn signalling put player on the black list, while resignation on player turn means put player on white list (remove me from black list).

This allows even players on black list to set their own prefered time and be played by bot ... by joining the bot, resigning on players turn and starting open game on prefered time control.
Expecting the bot is run in loop it joins the game ... .

I expect this could be usefull not only for my bot.
I am new to python, but I hope it would not be too hard to incorporate it there.

@Janzert: I expect you are not planning to code such thing, but I hope you don't mind me to do such changes.

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on May 16^th, 2011, 4:01am

You're certainly allowed to change the scripts provided with AEI however you want. I just may or may not incorporate those changes back into the main distribution. :)

Btw, for the feature you're talking about above, I think it would be safest to just use a whitelist and then add or remove players to it with the method you mention. So the bot would never join an open game for a player that hadn't made an explicit action causing it. Although even then it might be a little risky that someone would still get upset if they didn't understand the resignation signalling method being used.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by Hippo on May 16^th, 2011, 5:21am

Of course I expect mentioning the resign signalling in the game chat.
But you are right white list seems better.

... and I must think about bots.

Black list strategy for bot's seems be better ... when no open game find during setup ... bot creates its own open game and waits.

There is low probability of colision with the game opened for human (if human connects in short term).

And this allows anybody to invite the botA for botB just by opening botB and (if botA game remains open ... himself joining botA and resigning).

There would be only problem to invite such bot when there is another open game so the bot could chose the other. So if more bot's use this "always ready" strategy one should join those which should not be connected to swith it temporarily off.

Hmmm, it may be difficult to stop pair of "always ready" bots from playing each other (from outside).

May joining white list humans first than join bots would be preferable to allow easy break for white listed humans.

Omar@ would you mind traffic caused by several "always ready" bots?

Title: Re: Arimaa Engine Interface (AEI)
Post by Hippo on Apr 13^th, 2012, 3:05am

I want to make my bot to connect to open bot game with bot closest in rankings to my bot as possible.
But I don't know how to get bot's rating before sitting to the table.
Does anybody know how to get the ratings ... preferably from python?
Thanks

Title: Re: Arimaa Engine Interface (AEI)
Post by Nombril on May 2^nd, 2012, 10:05pm

Just a shot in the dark to see if anyone has some suggestions for where to look. John and I will be trying again in the morning.

roundrobin.py is not communicating with our new bot.

We were able to run bot_hippo locally (windows environment) with roundrobin.py and play some games locally (against human and simple bot). So I'm (relatively) confident the infrastructure is sound.

We have written a simplified bot, to included only what we understand. We can run our bot directly (in Netbeans), typing the server commands, and getting correct echos, changes to board state, etc.) But when we update roundrobin.py to use this new bot, we get a timeout that aeiok wasn't received.

It is still very murky to me how to debug in this situation when one language is calling another. Any suggestions for a path we might have missed? Or some other settings?

Edit: Here are the roundrobin.py settings... Am I correct we can have multiple bots listed here like this? http://pastebin.com/xQCvUN5a

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on May 2^nd, 2012, 11:21pm

First thing that comes to mind is, make sure you're flushing stdout after sending the bot output.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by Nombril on May 3^rd, 2012, 8:23am

Thanks for the quick reply Janzert.

Starting fresh today I was able to find the problem.
I had been running via starting the class, I tried running with the jar file (netbean's "suggestion" to run the program) and it worked.

Quote:

#cd \Users\Eric\Dropbox\Arimaa\the-best-arimaa-bot\bot_jae\build\classes
#java bot_jae/Bot_jae
#above didn't run will try this:
java -jar "C:\Users\Eric\Dropbox\Arimaa\the-best-arimaa-bot\bot_jae\dist\bot_jae.jar"

I had a poor description of the error last night - running roundrobin from python was giving me the timeout error. This morning I ran it from compiled python/windows explorer... and it gave me different output - that it couldn't find the bot.

Title: Re: Arimaa Engine Interface (AEI)
Post by Hippo on May 6^th, 2012, 7:15am

on 05/02/12 at 22:05:30, Nombril wrote:

We were able to run bot_hippo locally (windows environment) with roundrobin.py.

Well :) so it works at least for one other people.
... I expect I could use the jar calling method as well ... .

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on May 6^th, 2012, 11:24pm

on 05/03/12 at 08:23:20, Nombril wrote:

Thanks for the quick reply Janzert.

Sorry, this one's a bit slower. ;)

Quote:

Starting fresh today I was able to find the problem.
I had been running via starting the class, I tried running with the jar file (netbean's "suggestion" to run the program) and it worked.

I have very limited experience with java so I'm probably wrong, but doesn't that first way of running it need to also modify the classpath in order to work?

Quote:

I'm not quite clear on how you're running roundrobin, either the first or second way. :) Normally I just type in "roundrobin.py" from a command line opened to the aei directory and windows with its extension association mechanism will run it with python. Double clicking on roundrobin.py from windows explorer works as well though, but is easier to lose the roundrobin console output.

It sounds like for some reason the second method you are using to run it isn't finding your s.bat file. Maybe the current working directory isn't set to the aei directory? Except I think that would cause an error where it can't read the config file. In any case maybe try supplying an absolute path for your bot command.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by Hippo on May 7^th, 2012, 6:58pm

I have added 3 fold repetition test to roundrobin.py. My current version of it is here (http://hrosi.dyndns.org/arimaa/roundrobin.py).
[edit]I was editting it as it was not prepared for the "resign" move. So it was added as well.

(Chatting was added earlier.)
[/edit]

[edit]
I forgot to mention ... I have corrected bug in reserve calculation for percent != 100. (When player uses reserve).
[/edit]

P.S.: I have not added the forced 3 times repetition test to immobilization condition, but I expect it could be done.

P.P.S.: Janzert ... should I try to modify https://github.com/Janzert/AEI? ... I have not experimented with it ... I expect your version will be maintained and another version will be added?
Have I rights to do that?

I should modify aei protocol specification ... as I have added some signalling ... I am sending sid ... but it is not that much important. I am sending chats. I would like to send "postal" flag as postal playing differs from on-line (the sid varies during the game in postal game).
I have used sid+oppname as a logfile name, but it is not practical in postals ... . Sending permanent game id at the end could be usefull as well ... to rename the logfile finally to match it easily with the arimaa.com database.

In future allowing the postgame chat could be usefull.
(But I have more important TODOs now ;))

Standard aei bot should be ready to ignore signals it does not understand ... to let the AEI open for further changes.

Title: Re: Arimaa Engine Interface (AEI)
Post by Hippo on May 7^th, 2012, 7:12pm

on 05/02/12 at 22:05:30, Nombril wrote:

And play some games locally (against human and simple bot).

Do you have problems with human (rabbits interface) I was mentioning? ... Freezing whenever you touch a key on keyboard (with the window active).

It is not that anoying whan you don't want chatting ... but I would prefere chatting interface :(.

Title: Re: Arimaa Engine Interface (AEI)
Post by Nombril on May 9^th, 2012, 10:28pm

on 05/06/12 at 23:24:31, Janzert wrote:

I have very limited experience with java so I'm probably wrong, but doesn't that first way of running it need to also modify the classpath in order to work?

You are correct. I modified the classpath to get bot_hippo to run, and I thought bot_jae had been running, I wonder now if I somehow changed the directory? Anyway, the jar seems better, so the professor can run it without changing classpaths :)

Quote:

The first way was when I was trying to debug, I had roundrobin.py already open in IDLE, and ran in debug mode there. I guess the IDLE debug only showed what was happening as a result of the python commands. So it didn't show the error from the command line part. I agree, both ways you mentioned would have given me the error saw in the morning. (But running from the command line is probably better - double clicking from windows explorer --> the error sometimes doesn't stay on the screen for long enough to read before the window automatically closes.

PS, Hippo, I'll check tomorrow (hopefully!) about the chat, I thought it worked.

Title: Re: Arimaa Engine Interface (AEI)
Post by Janzert on May 10^th, 2012, 1:40pm

on 05/07/12 at 18:58:32, Hippo wrote:

I have added 3 fold repetition test to roundrobin.py. My current version of it is here (http://hrosi.dyndns.org/arimaa/roundrobin.py).

Unfortunately the latest unreleased version has rearranged quite a bit of the code in roundrobin.py so I'm having some difficulty comparing what has changed in your version and what has already been fixed. Even if you don't setup git (see below) you can get the latest version from github by clicking the "download as zip" at https://github.com/Janzert/AEI/downloads.

Quote:

[edit]I was editting it as it was not prepared for the "resign" move. So it was added as well.

(Chatting was added earlier.)
[/edit]

Oh, is this actually allowed by the arimaa server (as in it results in a resignation not an illegal move ending)? I just thought with Omar's preference against resigning that bots were not given a way to resign.

Quote:

[edit]
I forgot to mention ... I have corrected bug in reserve calculation for percent != 100. (When player uses reserve).
[/edit]

Thanks, that is definitely wrong currently.

Quote:

P.S.: I have not added the forced 3 times repetition test to immobilization condition, but I expect it could be done.

Yep, certainly should be added.

Quote:

P.P.S.: Janzert ... should I try to modify https://github.com/Janzert/AEI? ... I have not experimented with it ... I expect your version will be maintained and another version will be added?
Have I rights to do that?

Basing changes off the github repo is certainly the way to go. The normal way to do it is by forking it on github, which anyone can do. I'm not sure exactly what you mean by "your version will be maintained and another version will be added", but yes another copy is created when you fork it that you can then apply your changes to. The github tutorial series (http://learn.github.com/p/intro.html) seems a fairly good introduction, although it looks to be a bit dense if you aren't already familiar with other version control systems.

Quote:

I should modify aei protocol specification ... as I have added some signalling ... I am sending sid ... but it is not that much important. I am sending chats. I would like to send "postal" flag as postal playing differs from on-line (the sid varies during the game in postal game).
I have used sid+oppname as a logfile name, but it is not practical in postals ... . Sending permanent game id at the end could be usefull as well ... to rename the logfile finally to match it easily with the arimaa.com database.

In future allowing the postgame chat could be usefull.
(But I have more important TODOs now ;))

Sound like useful things to add to the gameroom interface. I don't think any of the above should be done in a way that modifies the protocol itself though. Specifically "setoption" should be used for sending the extra information to the engine and "log" for the chat messages back from the engine.

Quote:

Standard aei bot should be ready to ignore signals it does not understand ... to let the AEI open for further changes.

One of the primary goals for AEI is ease of correct implementation by engine authors. To help this the number of message types has been kept to a minimum. Any added message is meant to require a protocol version change, with the protocol version in use for a session set by the engine. So the engine should never get a message it can't handle. Also AEI is a stateful protocol and an unexpected or unrecognized message means the engine and controller can no longer be sure of the appropriate state after an unrecognized message. Having an immediate error helps prevents more subtle bugs from persisting.

Initially AEI allowed unknown setoption messages to pass silently. After I was bitten by misspelling an option name in OpFor without noticing it since it was just silently ignored, this was changed to logging a warning message. :-/

The one major addition AEI still needs is the ability for the engine to tell the controller about the options it understands during the opening phase.

Janzert

Title: Re: Arimaa Engine Interface (AEI)
Post by Hippo on Oct 28^th, 2012, 9:25am

My current AEI version gameroom_tst.py is here ...
http://hrosi.dyndns.org/arimaa/AEI_eee.zip.