Welcome, Guest. Please Login or Register.
Sep 17th, 2019, 2:40pm

Home Home Help Help Search Search Members Members Login Login Register Register
Arimaa Forum Suite of test positions


   Arimaa Forum
   Arimaa
   Bot Development
(Moderator: supersamu)
   Suite of test positions
« Previous topic | Next topic »
Pages: 1 2 3  Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print
   Author  Topic: Suite of test positions  (Read 5318 times)
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Suite of test positions
« Reply #15 on: Feb 28th, 2008, 11:30pm »
Quote Quote Modify Modify

Now that is just scary, Jean.  How many games do you have your own analysis notes on?  It's no wonder you are the reigning champ if you are constantly analyzing games, even games in which you didn't participate.
 
But what is even scarier is that your un-aided analysis was better than my analysis with Bomb's help.  Upon further review, I think you are right that there is one and only one move to stop goal, even though neither I nor Bomb found it.  I am officially demoralized in advance of our next encounter.  Sad
 
Knowing that there is only one right answer re-instates this position as a good candidate for a test suite.  It is unlikely that a program will find the right move for the wrong reason, because Gold can't defend goal without giving up a horse.
IP Logged

chessandgo
Forum Guru
*****



Arimaa player #1889

   


Gender: male
Posts: 1244
Re: Suite of test positions
« Reply #16 on: Mar 1st, 2008, 4:32am »
Quote Quote Modify Modify

on Feb 28th, 2008, 11:30pm, Fritzlein wrote:
I am officially demoralized in advance of our next encounter. Sad

 
... says the man with an 8-0 record Smiley
IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Suite of test positions
« Reply #17 on: Mar 1st, 2008, 7:35am »
Quote Quote Modify Modify

Heheh.  I have a special ability to feel that all my wins are lucky flukes I didn't deserve.
IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Suite of test positions
« Reply #18 on: Oct 11th, 2009, 9:13am »
Quote Quote Modify Modify

Is anyone still interested in compiling public test positions?  The position before 60w in marwin vs. arimaa_master seems to be a good candidate for reasons discussed here:
http://arimaa.com/arimaa/gameroom/comments.cgi?gid=120131
 
Apparently the is an obvious move (taking a camel hostage) that loses the game, and otherwise one clear move candidate (pushing the horse into g3) to keep the game in balance.  The test question is how long it takes the bot to prefer the better move of the two.
IP Logged

aaaa
Forum Guru
*****



Arimaa player #958

   


Posts: 758
Re: Suite of test positions
« Reply #19 on: Oct 11th, 2009, 10:34am »
Quote Quote Modify Modify

It takes my bot five minutes to find 60g He2e Hf2n rg3n Hf3e.
IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Suite of test positions
« Reply #20 on: Oct 11th, 2009, 12:33pm »
Quote Quote Modify Modify

on Oct 11th, 2009, 10:34am, aaaa wrote:
It takes my bot five minutes to find 60g He2e Hf2n rg3n Hf3e.

Bomb2005, on my old, slow computer, needed an hour.
IP Logged

Arimabuff
Forum Guru
*****



Arimaa player #2764

   


Gender: male
Posts: 587
Re: Suite of test positions
« Reply #21 on: Oct 11th, 2009, 2:01pm »
Quote Quote Modify Modify

on Oct 11th, 2009, 12:33pm, Fritzlein wrote:

Bomb2005, on my old, slow computer, needed an hour.

Which one is at fault? Bomb or your computer?
IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Suite of test positions
« Reply #22 on: Oct 11th, 2009, 2:43pm »
Quote Quote Modify Modify

on Oct 11th, 2009, 2:01pm, Arimabuff wrote:
Which one is at fault? Bomb or your computer?

That's hard to know without having my computer run a different bot, or running Bomb on a different computer.  Tize said that Marwin found it at depth 17, while Bomb found it at depth 16, so that makes Bomb look good, but maybe Bomb has so many extensions it takes longer to get to depth 16 than marwin take to get to depth 17.  Presumably quad found it at a lower depth, but doesn't search as deeply due to using tons of extensions.
 
I salute Omar for providing the Arimaa Challenge hardware, so that we are truly testing which software is better.  Taking one variable out of the equation gives us more insight into the other.
IP Logged

jdb
Forum Guru
*****



Arimaa player #214

   


Gender: male
Posts: 682
Re: Suite of test positions
« Reply #23 on: Oct 11th, 2009, 4:18pm »
Quote Quote Modify Modify

It takes clueless 5 seconds to find this move, at depth 9. It stays with this move up to at least depth 18, in ten minutes.
IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Suite of test positions
« Reply #24 on: Oct 12th, 2009, 4:35am »
Quote Quote Modify Modify

Wow.
IP Logged

aaaa
Forum Guru
*****



Arimaa player #958

   


Posts: 758
Re: Suite of test positions
« Reply #25 on: Oct 12th, 2009, 6:05am »
Quote Quote Modify Modify

on Oct 11th, 2009, 2:43pm, Fritzlein wrote:
Presumably quad found it at a lower depth, but doesn't search as deeply due to using tons of extensions.

Just barely, at 15 steps deep. Maybe that's because the null-move pruning is applied conservatively, allowing a full four-step move for the beneficiary of the null move to refute it.
IP Logged
tize
Forum Guru
*****



Arimaa player #3121

   


Gender: male
Posts: 118
Re: Suite of test positions
« Reply #26 on: Oct 14th, 2009, 1:22pm »
Quote Quote Modify Modify

Hats off to Clueless!
 
Quote:

Tize said that Marwin found it at depth 17, while Bomb found it at depth 16...

[pathetic resque attempt]
You said, in a game comment, that Bomb didn't stick with the move when a depth 17 search was done. In that case Marwin finds the right move at depth 6. Smiley
[\pathetic resque attempt]
 
It takes about 11 minutes for Marwin to stick with the right move.
 
It's interesting to see that Clueless needed 5 seconds to reach depth 9 while Marwin needs less than a second on this position, but after 10 minutes Clueless has made it through depth 18 while Marwin still struggles with depth 17.
IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Suite of test positions
« Reply #27 on: Oct 14th, 2009, 6:12pm »
Quote Quote Modify Modify

on Oct 14th, 2009, 1:22pm, tize wrote:
You said, in a game comment, that Bomb didn't stick with the move when a depth 17 search was done. In that case Marwin finds the right move at depth 6. Smiley

Good point.  Finding the right move doesn't count if one doesn't stick with it.  I wonder why Bomb realizes the camel hostage is no good at 16 steps but likes it again at 17.  Maybe curiosity will overcome my impatience, and I will let Bomb run overnight on this position.
IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Suite of test positions
« Reply #28 on: Jan 18th, 2010, 3:24pm »
Quote Quote Modify Modify

I'm not sure if this link was posted long ago, and by now perhaps developers have each made their own, more extensive goal-finding suite, but for what it's worth Fotland made his suite public here: http://www.smart-games.com/mate12.ZIP
IP Logged

doublep
Forum Guru
*****



Badger author

   


Gender: male
Posts: 82
Re: Suite of test positions
« Reply #29 on: Jan 20th, 2010, 2:28pm »
Quote Quote Modify Modify

I have an around 100 positions goal test suite (positions + correct/incorrect solutions).  Most tests were collected with automated tools against errors of Badger.  However, all tests are written in a slightly modified GTP, like:
 
setup_board r-dr------r---r----------M-----rRhH-----H---C-----D--rEeRR-----R
620 find_one_move_win black
#? [true eh2n-eh3w-eg3w-rf2s-ef3x]
 
Is there any interest in it?
IP Logged
Pages: 1 2 3  Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print

« Previous topic | Next topic »

Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB 2000-2003. All Rights Reserved.