Welcome, Guest. Please Login or Register.
Mar 28th, 2024, 5:27pm

Home Home Help Help Search Search Members Members Login Login Register Register
Arimaa Forum « AI Challenge Contest »


   Arimaa Forum
   Arimaa
   Off Topic Discussion
(Moderators: christianF, supersamu)
   AI Challenge Contest
« Previous topic | Next topic »
Pages: 1 2 3  Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print
   Author  Topic: AI Challenge Contest  (Read 7888 times)
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: AI Challenge Contest
« Reply #15 on: Dec 25th, 2011, 6:09pm »
Quote Quote Modify Modify

Congratulations, Hippo, on your 23rd-place finish!
IP Logged

omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: AI Challenge Contest
« Reply #16 on: Dec 28th, 2011, 3:13pm »
Quote Quote Modify Modify

on Dec 21st, 2011, 8:19am, ingwa wrote:
Without having done the math, I'm pretty sure that a 4x4 playing field is too small.  It could be precalculated in a not-so-giant table. (Besides with 8 pieces per side there would be no room for movement, the board would be full.)
 
I think that 6x6 could work, though, for instance using EMHHDD+6R.

 
Sorry I should have been more clear about this. The board would still be the usual 8x8 Arimaa board, only the number of pieces used in the starting would be less. Fewer pieces in the starting tends to make the games go faster, but I've never tried it with EMHDRRRR per side. So it would need to be play tested a bit. I proposed this based on Karl's guess that having one of each non-rabbit piece would make the game more sharp. Perhaps this could be play tested by bots. A bot playing about 100 games against itself thinking for just 2 ply should provide enough data to give us an idea of how many moves a typical game takes using this set of pieces.
IP Logged
omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: AI Challenge Contest
« Reply #17 on: Dec 28th, 2011, 3:20pm »
Quote Quote Modify Modify

on Dec 24th, 2011, 9:20am, Hippo wrote:
So great success at the end. Finished at 23rd place, being on 20th at the start of the last game. A bit of unluck seccond to last game finishing last while dominating board 5 turns to finish 2nd, around 30 turns to finish first.  
 
I lost at least twice due to using default parser which was not able to finish on server input parse first turn in time.

 
Amazing finish. It's good to know we have such great AI researchers working on the Arimaa challenge.
IP Logged
Janzert
Forum Guru
*****



Arimaa player #247

   


Gender: male
Posts: 1016
Re: AI Challenge Contest
« Reply #18 on: Dec 28th, 2011, 7:11pm »
Quote Quote Modify Modify

on Dec 20th, 2011, 10:56pm, omar wrote:
If you or Janzert are in touch with the organizers can you suggest a simplified version of Arimaa for a future contest. Maybe with just EMHDRRRR per side to make it go faster.

 
I guess you could say I'm in touch with the organizers since I now am one of the primary organizers. Wink
 
Unfortunately, as much as I like Arimaa, I don't think it's a very good fit for the challenge. Unlike Arimaa where strategic depth is a primary concern for its design, we need just enough to keep the game from being "solved" in a few months time. Not that extra depth would hurt, it's just not a primary constraint. A few of the primary constraints for a challenge game are:
 

  • Visually interesting and exciting replays with only minimal explanation of the game
  • Very easy for a new programmer to write a basic bot and obvious directions to improve it
  • A full game in no more than 10 minutes of time for each player and the average game in less than 5, preferably less than 2
« Last Edit: Dec 28th, 2011, 7:12pm by Janzert » IP Logged
omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: AI Challenge Contest
« Reply #19 on: Jan 7th, 2012, 7:15pm »
Quote Quote Modify Modify

Cool, I didn't know you were that involved with organizing it. Unfortunately none of the primary constraints have much to do with AI. But I understand why they are still important for attracting a large number of participants. Never the less, for such constraints I would suggest CCRRRR with 2 seconds per move. If you try such games with your own bot you see they go very fast, are easy to understand and visually exciting to watch. If a sample bot is provided where the participants just need to change the eval function, I think it could work.
 
IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: AI Challenge Contest
« Reply #20 on: Jan 8th, 2012, 11:23am »
Quote Quote Modify Modify

I was with Janzert in thinking Arimaa didn't fit the AI contest very well.  It's too slow, too opaque, and relies too heavily on a body of strategic knowledge.
 
Reduced-material Arimaa, however, is a great idea; The replays are visual, and can be understood with as little explanation as ants, or less.  (Ants combat rules are more opaque than Arimaa freezing and trapping, and Ants scoring is more opaque than getting a rabbit across the board).  The time control can be easily set to keep the games down to a couple of minutes.  And a basic bot would be only slightly fussier protocol than the other AI challenges.
 
However, I would strongly recommend ECRRRR or even EDCRRR for the material instead of CCRRRR.  The elephant increases the variety of piece interactions, makes the endgame even sharper, and (importantly for us) uses the iconic Arimaa piece in the promotion of Arimaa.  Keeping the same ratio of half-rabbits, half non-rabbits also has an attraction in my mind, not only to echo the full game, but to keep elimination on the table as a victory criterion.  3&3 would keep the capture side of Arimaa alive, whereas 2&4 would make it all about the goals.  And finally, more pieces would slow the game down, giving more time and more moves for superior strategy to manifest.  (Are my intuitions correct?  I haven't played a lot of reduced-material Arimaa.)
 
Also we would have to revisit the rule for deciding games that aren't over after 120 moves.  The easiest idea would be to accept draws.  Second easiest would be furthest-advanced rabbit, ties broken by next furthest, etc., with silver winning if both sides' rabbits are equally far advanced.  I would not like the current material rule for deciding draws, because I expect most draws in this format will not have any captures.  Not that I would expect many draws in such a sparse endgame, but bots will find ways to be clueless, and rabbit advancement is a way to reward a bot that is even minimally less clueless than its opponent.
 
A possible objection would be "why Arimaa" rather than another abstract strategy board game.  Plenty of people will be able to think of obscure abstract strategy games that look cooler (just from the rules) and haven't been deeply studied yet.  Last month I got an e-mail from boardspace.net announcing Volo: "Strategy and tactics for Volo are completely unknown - there are some obvious hex-like connections, but the rearrangement of the "birds" makes chains of connections difficult to maintain.  It's fair to say that no one plays it well yet, and it's hopelessly difficult for the robots."  So why Arimaa and not Volo?
 
One differentiator is that the $10,000 Arimaa Challenge lends the introductory challenge some lustre.  Participants can think, "After I win this, I will win the big bucks."  Another bonus is that Arimaa is sufficiently well studied that it is unlikely to break in the middle of the challenge.  Brand new games that haven't been beaten on as heavily as Arimaa are more likely to break under scrutiny.  It would be very awkward for a game to exhibit a strong drawing tendency or a strong player advantage for one side midway through the contest; there are few games with a legitimate claim to hold up on both counts given a rush of attention.
 
One of the charms of reduced-material Arimaa as far as I am concerned is that it will look simpler than it is.  There won't be hundreds of pieces flying around like Planet Wars and Ants.  Some people will even be tempted to think that they can achieve perfect play.  (12-piece tablebases?  Ha!)  But as experimentation progresses, folks will discover that correct play can be very subtle and visually non-obvious.  There will be plenty of discrimination between the best and second-best bot.
 
Another attraction is the elimination of randomized playing fields.  All previous contests needed an unsightly element of luck to keep the battles fresh, and participants legitimately complained that their bots did better on some maps but worse on others, so the map selection influenced their chances of winning.  Ugly.  Arimaa needs no such thing, thanks to the free setup and huge branching factor.
 
The only downside I can think of is one that Janzert didn't mention in his list, but which I bet the organizers will weigh heavily: with Arimaa people will be starting further from scratch.  For "new" problems, human intuition is more important than effectively implementing what it already known to work.  Boardgames in general (and Arimaa in particular), are too well studied for anyone to start from nothing and have a fighting chance.  In particular, static goal detection will be essential, and anyone without it won't be able to compete.  To level the playing field, one would almost have to release a static goal detector for everyone to use.
 
But with that possible objection aside, I think reduced material would make a fine AI challenge.  There is no great body of "strategic knowledge" of how to play Arimaa endgames.  Nobody will have to read books to figure our current theory.  Our existing material evaluators probably stink when it is down to EDCRRR.  We humans are clueless, so clueless developers starting out might have intuitions just as good as any in the Arimaa community today.
 
I am quite curious, Janzert, as to how you see reduced-material Arimaa meshing with the Google AI challenge.
« Last Edit: Jan 8th, 2012, 11:37am by Fritzlein » IP Logged

clyring
Forum Guru
*****



Arimaa player #6218

   


Gender: female
Posts: 359
Re: AI Challenge Contest
« Reply #21 on: Jan 8th, 2012, 12:21pm »
Quote Quote Modify Modify

on Jan 8th, 2012, 11:23am, Fritzlein wrote:
One differentiator is that the $10,000 Arimaa Challenge lends the introductory challenge some lustre.  Participants can think, "After I win this, I will win the big bucks."  Another bonus is that Arimaa is sufficiently well studied that it is unlikely to break in the middle of the challenge.  Brand new games that haven't been beaten on as heavily as Arimaa are more likely to break under scrutiny.  It would be very awkward for a game to exhibit a strong drawing tendency or a strong player advantage for one side midway through the contest; there are few games with a legitimate claim to hold up on both counts given a rush of attention.
With only 6 pieces in a multiple-rabbit endgame, it would surprise me if there wasn't a significant first-player advantage. That's hardly enough to close every file and leaves a defender without much leverage with which to resist once their opponent has advanced rabbits on both wings. IMO in order to make a more seriously playable variant without multiple minor pieces, there need to be either few enough rabbits that the elephant and minor pieces alone can hold them back or enough rabbits to serve as a serious defense on their own.
IP Logged

I administer the Endless Endgame Event (EEE). Players welcome!
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: AI Challenge Contest
« Reply #22 on: Jan 8th, 2012, 2:29pm »
Quote Quote Modify Modify

on Jan 8th, 2012, 12:21pm, clyring wrote:
With only 6 pieces in a multiple-rabbit endgame, it would surprise me if there wasn't a significant first-player advantage.

I agree that tempo is more important in an endgame than when the board is full.  Jdb verified that Gold has a forced win in ER vs ER, as explained in this thread.   On the other hand, with EDCRRR on each side,  Silver can align e vs D, d vs. C, and c vs. E.  The ability to be winning two of three fights might well compensate Silver for moving second.  Indeed, I would not be surprised if it was a forced win for Silver.
 
But of course the point is to have the game be so balanced and so complicated that we can't tell which side has the advantage.  I would be surprised if EDCRRR Arimaa showed a 50 Elo advantage for either side at the level that bots are able to play it.  
 
Arimaa has a natural "pie rule" to balance winning chances: swap sides after the setup.  At present, however, we can't implement the pie rule, because we don't know whether the option to swap should occur after the setup of Gold or the setup of Silver! Cheesy
« Last Edit: Jan 8th, 2012, 2:35pm by Fritzlein » IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: AI Challenge Contest
« Reply #23 on: Jan 8th, 2012, 3:08pm »
Quote Quote Modify Modify

on Dec 28th, 2011, 7:11pm, Janzert wrote:
Unlike Arimaa where strategic depth is a primary concern for its design, we need just enough to keep the game from being "solved" in a few months time.

It is interesting that Tron, with its super-simple rules, was nowhere close to being solved after a few months, although draws became enough of an issue to force many maps out of rotation.  With Planet Wars and Ants, it seems the game was prevented from being solved by having lots and lots of units.  The only downside of that as far as I can see is that humans can't play games once the number of units gets too large.
 
Is there any yearning for simplicity and elegance on AI contests?  In particular, is it a plus for the battleground to be a game that humans could compete at, if not outright vanquish the computers?  Perhaps coders don't care at all what un-assisted humans can do.
IP Logged

christianF
Forum Moderator
Forum Guru
*****



Arimaa player #4019

   


Gender: male
Posts: 804
Re: AI Challenge Contest
« Reply #24 on: Jan 9th, 2012, 4:19am »
Quote Quote Modify Modify

on Jan 8th, 2012, 3:08pm, Fritzlein wrote:
It is interesting that Tron, with its super-simple rules, was nowhere close to being solved after a few months, although draws became enough of an issue to force many maps out of rotation.
...
Is there any yearning for simplicity and elegance on AI contests?

What about Havannah?
Of the six or seven articles and theses listed there, I'll only mention Playing and Solving Havannah, an MSc thesis presentation by Timo Ewalds, M.Sc. Student, Department of Computing Science - University of Alberta, with Jonathan Schaeffer as supervisor. Timo is the creator of Castro, as far as I know the strongest Havannah bot to date.
 
Havannah is uniform and doesn't need loads of different pieces and an exploding branch density to be extremely hard to solve.
 
Never mind Symple (ah yes, you already do that Wink ), that will be the featured game at the 2013 CodeCup Challenge.  
 
You ask for simplicity ...  Huh
IP Logged
omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: AI Challenge Contest
« Reply #25 on: Jan 9th, 2012, 9:55am »
Quote Quote Modify Modify

Actually Havannah would also be a great game for the AI-challenge. Especially this year it would be perfect since the Havannah challenge match is coming up; which we've been anticipating for 10 years since Christian announced the challenge in 2002. Even on a base 5 board Havannah is not yet solved; and the game is gaurenteed to finish in 30 moves. I think the base 4 board was solved last year to be a win for the first player. The base 5 game should be sufficiently hard that it can't be solved during the contest and even if it is the organizers could easily change the contest to base 6 midway. The person who solved it would have achieved something they can write an academic paper about.
 
As Fritzlein mentioned, the nice thing about using games like Arimaa for the AI-challenge is that after the contest is over there is still a bigger challenge waiting. This applies for Havannah too.
IP Logged
rabbits
Forum Guru
*****




Arimaa player #1337

   
WWW

Gender: male
Posts: 108
Re: AI Challenge Contest
« Reply #26 on: Jan 9th, 2012, 3:03pm »
Quote Quote Modify Modify

on Jan 8th, 2012, 11:23am, Fritzlein wrote:
Another attraction is the elimination of randomized playing fields.  All previous contests needed an unsightly element of luck to keep the battles fresh, and participants legitimately complained that their bots did better on some maps but worse on others, so the map selection influenced their chances of winning.  Ugly.  Arimaa needs no such thing, thanks to the free setup and huge branching factor.

 
I actually like the idea of randomized playing fields in reduced-material Arimaa!  It would reduce the advantage that current bot developers have over people who had never heard of Arimaa, and the maps would be interesting!  The grid wouldn't have to be 8x8.  Figuring out how to represent the board would be an interesting problem in itself, since bitboards would not always be easily applicable.  Traps could be in different places.  Some squares could be designated as walls, where no piece may step.  If you want to be creative, you could introduce "sticky squares," on which a piece cannot step on and off in the same turn.  There are probably other interesting tweaks.
IP Logged

christianF
Forum Moderator
Forum Guru
*****



Arimaa player #4019

   


Gender: male
Posts: 804
Re: AI Challenge Contest
« Reply #27 on: Jan 9th, 2012, 3:22pm »
Quote Quote Modify Modify

on Jan 9th, 2012, 3:03pm, rabbits wrote:
Traps could be in different places.  Some squares could be designated as walls, where no piece may step.  If you want to be creative, you could introduce "sticky squares," on which a piece cannot step on and off in the same turn.  There are probably other interesting tweaks.

Loads of them, that's how we came to 2000+ chess variants. But bot performance is only meaningfull within the context of strong human opposition, so imo. deviating into forests full of variants isn't the way to go. Small Arimaa ok, if players and developers can agree on that, but I fear 'arimaaish' will open the floodgates. Wink
« Last Edit: Jan 9th, 2012, 3:49pm by christianF » IP Logged
Tuks
Forum Guru
*****



Arimaa player #2626

   


Gender: male
Posts: 203
Re: AI Challenge Contest
« Reply #28 on: Jan 11th, 2012, 5:38am »
Quote Quote Modify Modify

if the games are fast enough you could just have each bot play as both gold and silver against a specific opponent before going to the next one then people won't ever be able to use the excuse of he played gold and i had a much better chance of winning if i had played with gold based on my 80% win ratio as gold.
« Last Edit: Jan 11th, 2012, 5:38am by Tuks » IP Logged
Migi
Forum Senior Member
****



Arimaa player #4643

   


Gender: male
Posts: 26
Re: AI Challenge Contest
« Reply #29 on: Jan 11th, 2012, 8:32am »
Quote Quote Modify Modify

Congratulations Hippo on your 23rd place! My bot only made it to 31st place. I don't really know why it even got that high, though, because I only worked on it for like 3 days. Near the end of the contest, I made a second version of my bot, starting (almost) from scratch, which was mostly the same but with some small improvements here and there. I worked about a week on that, and locally it could beat my first version hands down, but to my surprise it performed pretty badly against other bots.
 
So in the end I just decided not to upload my second version.
 
I think the main reason my 2nd version is worse is the code I added to keep ants closer together and closer to my hills. The first version spreads out its ants a lot more, and often some ants randomly walk into an undefended hill. Also some of its ants end up behind enemy lines, which could be confusing the battle logic of some bots, and also allows me to surround enemy ants better. I also spent very little time on combat logic, and focused mostly on exploring and food gathering. I don't really know why though, better combat could have given me some easy ELO points.
 
About Arimaa (or a variant) being the next AI challenge game, I wouldn't get my hopes up. These are the criteria they are using to select the next game. I'm afraid Arimaa falls a bit short for the "easy" criterium. A lot of people in this challenge are new to programming and just want to use the challenge as a fun way to learn programming. I think they would be scared off by game like Arimaa. Also, I'm not sure many people would find it "familiar" and "fun to watch".
 
A lot of people also liked the multi-player aspect of Ants. I personally didn't, but it looks like the two games that are receiving most consideration right now (Risk and multiplayer Asteroids) are also multi-player. So all in all I don't think we'll be seeing Arimaa as the next Google AI challenge game, but I don't think that's bad thing, it's a completely different challenge with a completely different audience. And there already is an Arimaa challenge, why would we need another one? Wink
« Last Edit: Jan 11th, 2012, 6:43pm by Migi » IP Logged
Pages: 1 2 3  Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print

« Previous topic | Next topic »

Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB © 2000-2003. All Rights Reserved.