Author |
Topic: Suite of test positions (Read 7529 times) |
|
Fritzlein
Forum Guru
Arimaa player #706
Gender: 
Posts: 5928
|
 |
Re: Suite of test positions
« Reply #15 on: Feb 28th, 2008, 11:30pm » |
 Quote  Modify
|
Now that is just scary, Jean. How many games do you have your own analysis notes on? It's no wonder you are the reigning champ if you are constantly analyzing games, even games in which you didn't participate.
But what is even scarier is that your un-aided analysis was better than my analysis with Bomb's help. Upon further review, I think you are right that there is one and only one move to stop goal, even though neither I nor Bomb found it. I am officially demoralized in advance of our next encounter. 
Knowing that there is only one right answer re-instates this position as a good candidate for a test suite. It is unlikely that a program will find the right move for the wrong reason, because Gold can't defend goal without giving up a horse.
|
|
 IP Logged |
|
|
|
chessandgo
Forum Guru
Arimaa player #1889
Gender:
Posts: 1244
|
|
Re: Suite of test positions
« Reply #16 on: Mar 1st, 2008, 4:32am » |
Quote Modify
|
on Feb 28th, 2008, 11:30pm, Fritzlein wrote: I am officially demoralized in advance of our next encounter.
|
|
... says the man with an 8-0 record 
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Suite of test positions
« Reply #17 on: Mar 1st, 2008, 7:35am » |
Quote Modify
|
Heheh. I have a special ability to feel that all my wins are lucky flukes I didn't deserve.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Suite of test positions
« Reply #18 on: Oct 11th, 2009, 9:13am » |
Quote Modify
|
Is anyone still interested in compiling public test positions? The position before 60w in marwin vs. arimaa_master seems to be a good candidate for reasons discussed here:
http://arimaa.com/arimaa/gameroom/comments.cgi?gid=120131
Apparently the is an obvious move (taking a camel hostage) that loses the game, and otherwise one clear move candidate (pushing the horse into g3) to keep the game in balance. The test question is how long it takes the bot to prefer the better move of the two.
|
|
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: Suite of test positions
« Reply #19 on: Oct 11th, 2009, 10:34am » |
Quote Modify
|
It takes my bot five minutes to find 60g He2e Hf2n rg3n Hf3e.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Suite of test positions
« Reply #20 on: Oct 11th, 2009, 12:33pm » |
Quote Modify
|
on Oct 11th, 2009, 10:34am, aaaa wrote:| It takes my bot five minutes to find 60g He2e Hf2n rg3n Hf3e. |
|
Bomb2005, on my old, slow computer, needed an hour.
|
|
IP Logged |
|
|
|
Arimabuff
Forum Guru
Arimaa player #2764
Gender:
Posts: 589
|
|
Re: Suite of test positions
« Reply #21 on: Oct 11th, 2009, 2:01pm » |
Quote Modify
|
on Oct 11th, 2009, 12:33pm, Fritzlein wrote:
Bomb2005, on my old, slow computer, needed an hour. |
|
Which one is at fault? Bomb or your computer?
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Suite of test positions
« Reply #22 on: Oct 11th, 2009, 2:43pm » |
Quote Modify
|
on Oct 11th, 2009, 2:01pm, Arimabuff wrote:| Which one is at fault? Bomb or your computer? |
|
That's hard to know without having my computer run a different bot, or running Bomb on a different computer. Tize said that Marwin found it at depth 17, while Bomb found it at depth 16, so that makes Bomb look good, but maybe Bomb has so many extensions it takes longer to get to depth 16 than marwin take to get to depth 17. Presumably quad found it at a lower depth, but doesn't search as deeply due to using tons of extensions.
I salute Omar for providing the Arimaa Challenge hardware, so that we are truly testing which software is better. Taking one variable out of the equation gives us more insight into the other.
|
|
IP Logged |
|
|
|
jdb
Forum Guru
Arimaa player #214
Gender:
Posts: 682
|
|
Re: Suite of test positions
« Reply #23 on: Oct 11th, 2009, 4:18pm » |
Quote Modify
|
It takes clueless 5 seconds to find this move, at depth 9. It stays with this move up to at least depth 18, in ten minutes.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Suite of test positions
« Reply #24 on: Oct 12th, 2009, 4:35am » |
Quote Modify
|
Wow.
|
|
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: Suite of test positions
« Reply #25 on: Oct 12th, 2009, 6:05am » |
Quote Modify
|
on Oct 11th, 2009, 2:43pm, Fritzlein wrote:| Presumably quad found it at a lower depth, but doesn't search as deeply due to using tons of extensions. |
|
Just barely, at 15 steps deep. Maybe that's because the null-move pruning is applied conservatively, allowing a full four-step move for the beneficiary of the null move to refute it.
|
|
IP Logged |
|
|
|
tize
Forum Guru
Arimaa player #3121
Gender:
Posts: 118
|
|
Re: Suite of test positions
« Reply #26 on: Oct 14th, 2009, 1:22pm » |
Quote Modify
|
Hats off to Clueless!
Quote:
Tize said that Marwin found it at depth 17, while Bomb found it at depth 16...
|
|
[pathetic resque attempt]
You said, in a game comment, that Bomb didn't stick with the move when a depth 17 search was done. In that case Marwin finds the right move at depth 6.
[\pathetic resque attempt]
It takes about 11 minutes for Marwin to stick with the right move.
It's interesting to see that Clueless needed 5 seconds to reach depth 9 while Marwin needs less than a second on this position, but after 10 minutes Clueless has made it through depth 18 while Marwin still struggles with depth 17.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Suite of test positions
« Reply #27 on: Oct 14th, 2009, 6:12pm » |
Quote Modify
|
on Oct 14th, 2009, 1:22pm, tize wrote:| You said, in a game comment, that Bomb didn't stick with the move when a depth 17 search was done. In that case Marwin finds the right move at depth 6. |
|
Good point. Finding the right move doesn't count if one doesn't stick with it. I wonder why Bomb realizes the camel hostage is no good at 16 steps but likes it again at 17. Maybe curiosity will overcome my impatience, and I will let Bomb run overnight on this position.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Suite of test positions
« Reply #28 on: Jan 18th, 2010, 3:24pm » |
Quote Modify
|
I'm not sure if this link was posted long ago, and by now perhaps developers have each made their own, more extensive goal-finding suite, but for what it's worth Fotland made his suite public here: http://www.smart-games.com/mate12.ZIP
|
|
IP Logged |
|
|
|
doublep
Forum Guru
Badger author
Gender:
Posts: 82
|
|
Re: Suite of test positions
« Reply #29 on: Jan 20th, 2010, 2:28pm » |
Quote Modify
|
I have an around 100 positions goal test suite (positions + correct/incorrect solutions). Most tests were collected with automated tools against errors of Badger. However, all tests are written in a slightly modified GTP, like:
setup_board r-dr------r---r----------M-----rRhH-----H---C-----D--rEeRR-----R
620 find_one_move_win black
#? [true eh2n-eh3w-eg3w-rf2s-ef3x]
Is there any interest in it?
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print