Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> General Discussion >> Re: About the P3 version of Bomb
        
(Message started by: PMertens on Jul 2nd, 2005, 12:15pm)
Title: Re: About the P3 version of Bomb
Post by PMertens on Jul 2nd, 2005, 12:15pm
I agree.

There is no need for setting a timelimit on the bot side since it is fixed in search depth and humans need to sleep once in a while.

I would propose rather generous settings like 1d/10d since this experiment might take time and sometimes you cant play for even a week.
Maybe you can limit the load this bot can put on the server if you dont want it to run full power all day.
Title: Re: About the P3 version of Bomb
Post by omar on Jul 3rd, 2005, 10:15am
I was in the middle of setting it up, but then got inturrupted, so that timecontrol is not the final one. I have no idea about how long it will take for P3 to complete the move without being rushed by time. So I need to experiment with it a bit and then I'll set the timecontrol either to make it interactive or postal.
Title: Re: About the P3 version of Bomb
Post by PMertens on Jul 6th, 2005, 12:12am
uh ... there is a tiny little problem  :-/

If only one session of P3 is possible then someone will block the bot for everybody else for days/weeks.
Title: Re: About the P3 version of Bomb
Post by omar on Jul 6th, 2005, 6:47pm
I'll have to setup P3 to play postal games and rotate through multiple games.

Im finding that it takes about 30 minutes to make a move. P2 used to take only about 15 seconds per move. So it got slower by a factor of 120. If P4 is slower than P3 by a even a factor of 100 it will take around 3000 minutes to make a move; or about 50 hours; or a little over two days. Still possible to play it as a postal game; although it will only play a few games per year. P5 will have to wait till the hardware gets 100 times faster.
Title: Re: About the P3 version of Bomb
Post by PMertens on Jul 6th, 2005, 11:37pm
true, it would be a very slow experiment but I think it might be worth it (if you can spare the load on your server ;-) )

It would tell us wether or not we have to fear deeper search bots anytime soon or if we should concentrate on evaluation only.

If you can dominate a P4 Bomb the same boring way you can a P2 then the answer is clear to me.
If P4 rips me to pieces ... well then ... its only a question of time ...
Title: Re: About the P3 version of Bomb
Post by 99of9 on Jul 7th, 2005, 12:01am
I think we know the answer to that... remember bomb played 10 games at 3 hours each move in the postal.  That means it was between P3 and P4 on average... and lost a fair few.
Title: Re: About the P3 version of Bomb
Post by PMertens on Jul 7th, 2005, 12:57am
Thats not exactly correct.

P2 might use an AVERAGE of 15s but in complex situations its far more.
In my last game it had an Average of 24s and played several moves at far over a minute.
Its extremely complexity dependant.

Same with P3
I have seen a 6h move !

So actually the Postal-Tournament-Version was sometimes between P3 and P4 (in simple situations) and sometimes between P2 and P3 (in complex situations)

Since I consider the complex situations more important its not even P3.
Added to that I think a 12ply is MUCH stronger than an 11ply - while I believe a 13ply to be only a little stronger than a 12 ply.

And while bomb lost a few its even more surprising that it won any ;-)
At it current level of ply a P2 should not be able to win (unless the human is toying around like I tend to do)

Conclusion:

I dont think we know the answer yet
Title: Re: About the P3 version of Bomb
Post by 99of9 on Jul 7th, 2005, 4:51am

on 07/07/05 at 00:57:42, PMertens wrote:
P2 might use an AVERAGE of 15s but in complex situations its far more.
In my last game it had an Average of 24s and played several moves at far over a minute.
Its extremely complexity dependant.

I think that's because for bomb P2 is not strictly 8 steps.  Bomb has quite a lot of search extensions etc.  For gnobby P2 = exactly 8 steps, always.  Even in complex situations this never takes over 15s.
Title: Re: About the P3 version of Bomb
Post by RonWeasley on Jul 7th, 2005, 7:53am

Quote:
Omar, I believe that thanks to him any record bashing against Bomb is rendered moot by the fact that if you possess the home version you can play back and forth and therefore sooner or later you will find the series of moves that will win no matter how mediocre your personal talents.


The current handicap records against the Bomb variants have all been about discovering and applying a pattern that exploits a weakness in Bomb.  Once the best pattern has been discovered, the record goes to the one who can apply the pattern most effeciently.  And get some luck from the random effects in Bomb.

Training on the pattern, on- or off-line, is something all the bot bashers are doing.  I don't see the logic in objecting to a specific method of training.  Am I missing something?

If logic is not his motivation, another possibility is that Arimanator enjoys PMertens bashing just as much as bot bashing.  Or maybe he just likes "trash talk" and the attention it generates like we see in many professional sports.  The optimistic view is that this is more evidence of arimaa becoming a mainstream passtime.

Anyway, I'm giving Arimanator some attention here and I think that should make him happy.
Title: Re: About the P3 version of Bomb
Post by 99of9 on Jul 7th, 2005, 10:43am

on 07/07/05 at 10:18:46, Arimanator wrote:
Good question. With all due respect my dear RON I believe you are missing something.

Once you can play a bot by making him give a response to any position you want as the home version permits, it's no longer a question of finding a pattern but it's more like craking a code when you can check each number one after the other. even if the code contains dozens of numbers you intuitively guess that the number of hit and miss will be a lot fewer than it should be. And that it no longer matters whether you can play or not when you can check moves that makes no sense at all but to whom for some reason the bot gives a stupid answer that'll make him lose.

It makes Arimaa the stupidest of games where  you check each possibiltity after each other until you stumble on the right one.

DO YOU GET MY POINT THIS TIME?

because I can't make it any clearer, sorry.  


And your specific allegation is that this technique was what allowed PMertens to win the bomb_BLITZ handicap record?  If you think about it you'll realise that that is not as straightforward as you describe it for a system-dependent bot.

Even if it was, I, like Ron, believe that all is fair game.  If it's so easy to improve the record, go right ahead (it only costs $20 to prove your case!).  I thought you were going to do this by Sunday anyway?

By the way - there are plenty of other records out there and I'm still looking for volunteers to help me fill them.
Title: Re: About the P3 version of Bomb
Post by PMertens on Jul 7th, 2005, 11:02am
since its meant as a personal attack I would l like to mention the fact that I have no offline version of bomb.

Anyway ... I believe this to be the wrong topic for that discussion.
Title: Re: About the P3 version of Bomb
Post by RonWeasley on Jul 7th, 2005, 11:34am

Quote:
Once you can play a bot by making him give a response to any position you want as the home version permits, it's no longer a question of finding a pattern but it's more like craking a code when you can check each number one after the other. even if the code contains dozens of numbers you intuitively guess that the number of hit and miss will be a lot fewer than it should be. And that it no longer matters whether you can play or not when you can check moves that makes no sense at all but to whom for some reason the bot gives a stupid answer that'll make him lose.


While it's still not clear, Arimanator may be making the distinction between pattern and sequence.  With no randomization, it is possible to determine a sequence of moves, perhaps by trial and error, that wins.  It's possible to do this without any understanding of the game.  However, even if Bomb's randomization is restricted to the initial setup, and I don't think it is, it's unlikely a person without an understanding of the game could find a successful sequence in only a few weeks.  Even a good GA algorithm could take much longer due the number of combinations.  This was one of Omar's intentions and it appears to be holding up.  So I don't think any of the successful bot bashers could possibly be using that method.

A subtler difference is whether a bot basher understands why the pattern works.  That's an interesting topic, especially for bot developers who can then address the specific weaknesses.  To me, that is the greatest benefit of bot bashing.  Still, practicing off-line or on-line isn't relevant.
 

Quote:
DO YOU GET MY POINT THIS TIME?

because I can't make it any clearer, sorry.


Since we still disagree, I guess not.  Sending a howler was a good try, though!
Title: Re: About the P3 version of Bomb
Post by omar on Jul 9th, 2005, 6:59am
I've setup Bomb2005P3 so that it rotates through multiple postal games. Haven't tested it much though. Give it a try. When you start the game it might take a minute or so to show up and make it's move; so don't worry if it looks like it's not there.
Title: Re: About the P3 version of Bomb
Post by PMertens on Jul 9th, 2005, 2:22pm
hmmmm - that move is over 2 days already.
What was the longest move in your game, Omar ?

Any problem with the rotating ?


... ok, seems to work now ... thanks Omar :-)
Title: Re: About the P3 version of Bomb
Post by PMertens on Jul 12th, 2005, 1:51am
What surprises me most is that P3 seems not to be stronger than P2.
I managed to make some mistakes in the opening but bomb did not even try to take advantage (Game 23157 most impressive move 16b, but also those lovely Null-moves at 17 and 19)

I also got the impression that there are still lots of moves to prune (like using 3 steps to do st that can be undone in one ...) that should speed up things considerably.

Would it be possible to set up P2 vs P3 ?
In theorie P3 should clearly win.
Title: Re: About the P3 version of Bomb
Post by omar on Jul 14th, 2005, 6:37pm

on 07/09/05 at 14:22:03, PMertens wrote:
hmmmm - that move is over 2 days already.
What was the longest move in your game, Omar ?

Any problem with the rotating ?


... ok, seems to work now ... thanks Omar :-)


The process that was supposed to start the bot to make
the move for each postal game had died; Im not sure
what happened to it. I had to restart it again.
Title: Re: About the P3 version of Bomb
Post by omar on Jul 14th, 2005, 6:46pm

on 07/12/05 at 01:51:51, PMertens wrote:
What surprises me most is that P3 seems not to be stronger than P2.
I managed to make some mistakes in the opening but bomb did not even try to take advantage (Game 23157 most impressive move 16b, but also those lovely Null-moves at 17 and 19)

I also got the impression that there are still lots of moves to prune (like using 3 steps to do st that can be undone in one ...) that should speed up things considerably.

Would it be possible to set up P2 vs P3 ?
In theorie P3 should clearly win.


Im sure P3 is better than P2, but it's probably not that much better.  I could run P2 against P3, but it would take many games to determine exactly how much better P3 is than P2.
Title: Re: About the P3 version of Bomb
Post by omar on Jul 14th, 2005, 7:02pm

on 07/13/05 at 17:41:51, Arimanator wrote:
If P3 takes an average of 30+ min. per move does that mean that the championship Bomb, namely, the one Belbo had to contend against, was a P2+ one?

Or was the computer utilized back then a lot faster one?


The computer used for the challenge match was probably twice as fast as the one running P3.

The bots can usually be set to limit their search either based on time or depth of search. If limited by time they will search deeper in some areas of the search tree and hardly look at other areas. So the depth of the search can be quite variable. If the bot is limited by depth of search it will basically look at all postions to that depth before deciding on a final move; regardless of how long it takes.

So the version of Bomb that Frank played against in the challenge match was looking deeper than the P3 version in some areas of the tree. But we can say for sure now that it was not looking at all areas of the tree down to a depth of 3 plys (i.e. 12 steps).

Title: Re: About the P3 version of Bomb
Post by RonWeasley on Jul 16th, 2005, 4:06pm

Quote:
The program never gave me an answer to my move while I was connected.

For instance on move 6 I waited 2 hours only to discover that Bomb's response came after a mere 10 min.

Even Bomb's piece setting didn't come while I was connected.

I can't do that tiresome manoeuver 50 times in a row.

I have the lastest update of XP, in case you were wondering.


This experience may help me.  I also have XP (at home).  I can't get a game started as silver because I don't get gold's setup move.  Disconnecting and joining again might be the way to get a game going.

I wish I could help make the problem go away.  But I have tried JRE updates and security settings and have not found a combination that works.  Anybody else with XP have some insight?
Title: Re: About the P3 version of Bomb
Post by omar on Jul 16th, 2005, 11:51pm
You might want to make sure you have a current version of Flash player.

Go to http://macromedia.com

Near the bottom of the page there is a button to download the Flash player.

Also I would suggest using to the Firefox browser if possible. You can get it from:

http://mozilla.org/

If you have this setup already and the problem still presists, then it's problem with my Flash client. I already know of some things that I need to fix, but I would like to know about any other problems that are encountered. Thanks.
Title: Re: About the P3 version of Bomb
Post by Fritzlein on Jul 17th, 2005, 10:01pm

on 07/06/05 at 18:47:25, omar wrote:
If P4 is slower than P3 by a even a factor of 100 it will take around 3000 minutes to make a move; or about 50 hours; or a little over two days. Still possible to play it as a postal game; although it will only play a few games per year.


I'll play BombP4 if you set it up.  I didn't understand that BombP3 was only going to take ten minutes or so per move.  I'm up for a nice, slow postal game, say at 4 days per move.
Title: Re: About the P3 version of Bomb
Post by PMertens on Jul 17th, 2005, 10:58pm

on 07/07/05 at 00:57:42, PMertens wrote:
Thats not exactly correct.

P2 might use an AVERAGE of 15s but in complex situations its far more.
In my last game it had an Average of 24s and played several moves at far over a minute.
Its extremely complexity dependant.

Same with P3
I have seen a 6h move !


that would be up to 25 days for P4 :-(
(not in the opening)

the good thing is:
it will speed up in the endgame due to hardware changes Omar is likely to do in a few years ;-)
Title: Re: About the P3 version of Bomb
Post by omar on Jul 18th, 2005, 6:45pm
I've setup the P4 version of Bomb if anyone wants to try it. But only one person will be able to play it at a time.

I've set it up with a 10 day per move timecontrol and max reserve of 30 days. Hope that will be enough time for it.

Have fun.
Title: Re: About the P3 version of Bomb
Post by Fritzlein on Jul 18th, 2005, 7:00pm

on 07/18/05 at 18:45:06, omar wrote:
I've setup the P4 version of Bomb if anyone wants to try it. But only one person will be able to play it at a time.

I've set it up with a 10 day per move timecontrol and max reserve of 30 days. Hope that will be enough time for it.

Have fun.


Thanks, Omar.  I'll be greedy and take the first shot at it.  I'll report back how long it seems to be taking per move.

1w: 00:00:00:00 :-)
2w: 00:00:07:51
3w: 00:00:27:04
4w: 07:14:39:20 (process died)
5w: 00:02:06:30
6w: 01:17:30:36
7w: 02:16:16:04
8w: 00:13:34:38
9w: 00:18:10:52
10w: 00:14:14:27
11w: forgot to record
12w: 00:18:26:31
13w: 00:18:59:07
14w: 01:02:28:57
15w: 00:08:56:01
16w: 00:18:58:36
17w: 01:11:57:06
Title: Re: About the P3 version of Bomb
Post by 99of9 on Jul 18th, 2005, 11:51pm

on 07/18/05 at 19:00:29, Fritzlein wrote:
1w: 00:00:00 :-)
2w: 00:07:51
3w: 00:27:04

The trend is looking worryingly superlinear!! :-)
Title: Re: About the P3 version of Bomb
Post by Fritzlein on Jul 19th, 2005, 10:28pm

on 07/19/05 at 05:16:56, Arimanator wrote:
What happened to P4 ? did it crash? If so too bad  :(. I was looking forward to Fritz game against it.

Fritz, if you try this again, you have at least one avid spectator here.  ;)

I can see it now in the list, so maybe it was a dysfunction on my end.


I'm not sure how I would know if the bot crashes, unless it runs out of time, and even then it might be hard for me to discern the cause of the timeout.  Perhaps in a couple of days Omar can check if P4 is still thinking.

Having spectators puts on the pressure!  In my game against P3 I'm playing very haphazardly, but I guess since I'm the only human who currently has a shot at P4, I shouldn't blow it for humanity.  :-/
Title: Re: About the P3 version of Bomb
Post by Fritzlein on Jul 20th, 2005, 8:15am

on 07/07/05 at 00:57:42, PMertens wrote:
Same with P3
I have seen a 6h move !


Interesting... The longest move I've seen so far from P3 (up through move 14b) was 31 minutes.  Most moves have been around 7 minutes.  I guess I don't play very complex stuff...

Now if P3 can swing by a factor of 20 or more on the time taken per move, then probably P4 will someday take more than 40 days on a move and time out on the spot.
Title: Re: About the P3 version of Bomb
Post by omar on Jul 20th, 2005, 11:16am
Yes, it would be nice if we could set different time controls for the two players. Claude had suggested this as well (a while ago).

This would allow us to set a virtually unlimited time control for the Px bots and a resonable time control for the humans.

Unfortunately I didn't think of this when I was developing the site and changing it now is not going to be easy. But I'll keep this in mind for the future.
Title: Re: About the P3 version of Bomb
Post by Fritzlein on Jul 26th, 2005, 4:40pm

on 07/19/05 at 22:28:34, Fritzlein wrote:
I'm not sure how I would know if the bot crashes, unless it runs out of time, and even then it might be hard for me to discern the cause of the timeout.  Perhaps in a couple of days Omar can check if P4 is still thinking.


Omar, Bomb2005P4 hasn't moved for over a week now.  Would you be willing to make sure it is actually thinking, rather than having crashed?  Possibly Bomb simply can't calculate that deeply in a reasonable amount of time, but it would be a shame to have P4 lose on time due to a process dying.
Title: Re: About the P3 version of Bomb
Post by omar on Jul 27th, 2005, 8:08am
Yes, it had stopped. I've restarted it.

I made a change which might prevent this from happening, but let see if it really works. Continue to let me know if it seems to have stopped.
Title: Re: About the P3 version of Bomb
Post by Fritzlein on Aug 10th, 2005, 9:32am

on 08/10/05 at 06:01:57, Arimanator wrote:
So Fritz, I am curious, what is p4's mean reflection time so far (not counting the move during which it crashed, if possible).


I have continued to update the times on my post on the second page of this thread.  The maximum was over 2 days and the average under one day.  I should point out, however, that the position has not yet gotten as complex as it might, with multiple capture and goal threats.  I wouldn't be surprised if the position gets messy and Bomb starts pushing its ten day allotment.
Title: Re: About the P3 version of Bomb
Post by Fritzlein on Aug 18th, 2005, 11:28am
Say, Omar, has BombP4 died again?  It hasn't moved in over five and a half days, which is twice as long as any other move apart from the time it last died.  No hurry on checking this out: Bomb still has a few weeks in reserve.
Title: Re: About the P3 version of Bomb
Post by omar on Aug 18th, 2005, 10:22pm
No, looks like the process is running:


  PID TTY      STAT   TIME COMMAND

 2581 ?        S           0:00 /usr/bin/perl botman start
12709 ?        S           0:00 /usr/bin/perl ./bot move 23397 w
12720 ?        R   6486:25 ./getMove ./running/pos12709 ./running/move12709



Title: Re: About the P3 version of Bomb
Post by 99of9 on Aug 18th, 2005, 11:55pm
Remember that bomb_P2 does an extra deep search if it thinks it's in trouble.  You must have finally scared the bot!
Title: Re: About the P3 version of Bomb
Post by Fritzlein on Aug 20th, 2005, 9:46am
Thanks for checking that the process is live, Omar.  I do think BombP4 is in some trouble in our game, but the position isn't as complicated as it might get later.  I was starting to think we were safe from a timeout, but I am no longer sure.  What will happen if the position gets truly sharp?
Title: Re: About the P3 version of Bomb
Post by Arimanator on Aug 20th, 2005, 10:29am

on 08/20/05 at 09:46:35, Fritzlein wrote:
Thanks for checking that the process is live, Omar.  I do think BombP4 is in some trouble in our game, but the position isn't as complicated as it might get later.  I was starting to think we were safe from a timeout, but I am no longer sure.  What will happen if the position gets truly sharp?


The question is : Is Bomb's perception of complexity parallel to ours.

I am sometimes surprised how long Bomb takes in some positions and how quick it is in some others when their relative intricacy looked pretty much the same to me.
Title: Re: About the P3 version of Bomb
Post by Fritzlein on Aug 20th, 2005, 4:57pm

on 08/20/05 at 10:29:25, Arimanator wrote:
The question is : Is Bomb's perception of complexity parallel to ours.


Good question.  One stab at a partial answer: Bomb possibly thinks longer when there isn't one move that is clearly better than the others, independent of how "sharp" the position is, whereas humans are possibly more able to choose between moves of similar value but need time to work out all of the most critical variations when pieces are hanging.
Title: Re: About the P3 version of Bomb
Post by Fritzlein on Aug 21st, 2005, 9:40am
Omar, I am beginning to think that the BombP4 experiment isn't worth it.  When P4 was moving in a day or so, it seemed like the game might complete in a reasonable amount of time, but now it looks like it might take months and months.  The current move is a week and counting, and the position is not as complex as it might get later.  Given that P4 is probably negatively impacting the performance of lots of the other bots, and given that I am probably beating P4 already, and given that P4 might time out somewhat down the road for a lame ending anyway, I think it may be wise to abandon the game now and free up the system resources.

Would you be willing to abort the game and make it unrated?
Title: Re: About the P3 version of Bomb
Post by omar on Aug 22nd, 2005, 9:45pm
I think you are right. The process time keeps going up so it's not as if the process died. But it is taking a long time to move.

I've reset this to be an unrated game. So you can just resign it and add comments explaining what happened.
Title: Re: About the P3 version of Bomb
Post by Fritzlein on Aug 24th, 2005, 10:37am
I resigned against P4.  Do you want to take it off-line so nobody else starts another game against it?
Title: Re: About the P3 version of Bomb
Post by omar on Aug 27th, 2005, 11:13am

on 08/24/05 at 10:37:47, Fritzlein wrote:
I resigned against P4.  Do you want to take it off-line so nobody else starts another game against it?


Thanks for the reminder. I just took it off the Bots page.


Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB © 2000-2003. All Rights Reserved.