Welcome, Guest. Please Login or Register.
Mar 28th, 2024, 10:22am

Home Home Help Help Search Search Members Members Login Login Register Register
Arimaa Forum « Arimaa Challenge format for 2006 »


   Arimaa Forum
   Arimaa
   Events
(Moderator: supersamu)
   Arimaa Challenge format for 2006
« Previous topic | Next topic »
Pages: 1 2 3 4  Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print
   Author  Topic: Arimaa Challenge format for 2006  (Read 4612 times)
omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: Arimaa Challenge format for 2006
« Reply #30 on: Aug 23rd, 2005, 1:03am »
Quote Quote Modify Modify

I was thinking again about just how much harder the new format is for the computers compared to the old format.
 
Karl had pointed out in another thread that if a bot had a 50% chance of winning with the old format then it would only have a 6.25% chance of winning with the new format if there were 4 human players. Likewise the chance of winning would be 12.5% if there were 3 human players.
 
I started wondering how much higher a bots rating would have to be above the humans rating to bring the bots  chance of winning back up to 50%. If this number turned out to be like a 1000 or 2000 points then I might have made the challenge impossible to acheive and should reconsider the format.
 
But my calculations are saying that a bot needs to only be about 155 rating points above the human to bring it's chance of winning back up to 50%. If that's the case then the new format is very reasonable. There is pleanty of room between the best player and perfect play for a bot to demonstrate that it is better than the best player. Can someone else try to see what number they come up with so we can verify this. Thanks.
 
« Last Edit: Aug 23rd, 2005, 9:11am by omar » IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Arimaa Challenge format for 2006
« Reply #31 on: Aug 23rd, 2005, 6:06am »
Quote Quote Modify Modify

If we assume that all 3 players are rated X, then by my calculation, the computer needs a rating of about  X+153.7 to have 50% chance of sweeping the humans as three best-of-three matches.
 
I can see why you say that 154 rating points is not very much, but the difference seems pretty significant to me.  For chess we had approximate parity in in 1999 and now six years later computers might be getting to 154 points better with Hydra.  For a challenge with only fifteen years left, six years is a fairly hefty chunk of time.  Smiley   I understand, though, that such a gap in strength might mean more in chess where draws are more frequent.
« Last Edit: Aug 24th, 2005, 6:10am by Fritzlein » IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Arimaa Challenge format for 2006
« Reply #32 on: Aug 23rd, 2005, 6:22am »
Quote Quote Modify Modify

on Aug 22nd, 2005, 10:45pm, omar wrote:

Im pretty sure that a supercomputer running an Arimaa program could definitely beat todays players.

 
Are you talking about today's software as well as today's human players?  I'm not as confident about that increase in play strength from faster hardware as you are.  Remember how easy it was to beat Bomb in the postal giving it maybe a 20 to 1 time advantage?  The weakness of strategic judgement in Bomb may cripple it even at great search depth.  However, I concede a difference between a factor of 20 and a factor of 10,000.  Smiley
 
I read that for chess there seems to be a rough correspondence between a doubling of think time and gaining 100 rating points.  Now if Arimaa is as computer-resistant as it was designed to be, for Arimaa a doubling of hardware speed might result in a gain of only 20 rating points.  Consider the difference in rating between Arimaazilla and Arimaazon, where the latter has 3 or 4 doublings of think time.  Now 10,000 is about 13 doublings, so maybe 260 rating points?  I still have more than that large a cushion above the current Bomb, and thus might withstand a 10,000-fold speed increase.
 
I guess, given that we don't have a supercomputer, it is idle speculation, but interesting nonetheless.  Smiley
 
IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Arimaa Challenge format for 2006
« Reply #33 on: Aug 23rd, 2005, 7:33am »
Quote Quote Modify Modify

on Aug 22nd, 2005, 10:05pm, omar wrote:
I actually did call Intel and ADM and talked to the heads of their marketing departments shortly after releasing Arimaa to see if they might be interested in sponoring the challenge.
 
I found out that big companies don't like to sponsor things that aren't already well known and established.

 
Oh, that's too bad.  I shouldn't be surprised, but I admit I am disappointed.  Omar, I apologize if my previous posts implied that you weren't thinking of the obvious sponsors and the things you could offer them.  I don't know much about marketing, and I was just speaking off the top of my head.
 
You've done a pretty darn good job of marketing Arimaa all by yourself, without the help of corporate sponsorship.  I don't think you have to do anything different for Arimaa to grow apace.  The friendly server and terrific game you have created are already enough of a draw to keep the community growing steadily.
IP Logged

omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: Arimaa Challenge format for 2006
« Reply #34 on: Aug 23rd, 2005, 9:54am »
Quote Quote Modify Modify

on Aug 23rd, 2005, 6:06am, Fritzlein wrote:
If we assume that all 3 players are rated X, then by my calculation, the computer needs a rating of about  X+153.7 to have 50% chance of sweeping the humans as three best-of-three matches.
 
I can see why you say that 154 rating points is not very much, but the difference seems pretty significant to me.  For chess we had approximate parity in in 1999 and now six years later computers might be getting to 154 points better with Hydra.  For a challenge with only fifteen years left, six years is a fairly hefty chuck of time.  Smiley   I understand, though, that such a gap in strength might mean more in chess where draws are more frequent.

 
Thanks for double checking that Karl.
 
Also with Chess being such a well studied game with so many good player, every rating point in the space above the best human players is very hard to gain. So 154 points above the the best players in Chess is very significant compared to acheiving the same in Arimaa where the knowledge of the game is relatively unexplored and the level of play is probably not even at a Masters level yet.
 
IP Logged
omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: Arimaa Challenge format for 2006
« Reply #35 on: Aug 23rd, 2005, 10:30am »
Quote Quote Modify Modify

on Aug 23rd, 2005, 6:22am, Fritzlein wrote:

Are you talking about today's software as well as today's human players?  I'm not as confident about that increase in play strength from faster hardware as you are.  Remember how easy it was to beat Bomb in the postal giving it maybe a 20 to 1 time advantage?  The weakness of strategic judgement in Bomb may cripple it even at great search depth.  However, I concede a difference between a factor of 20 and a factor of 10,000.  Smiley

 
Probably even with today's software.
 
Although the recent advances made in bot bashing within the last week or so by Naveed and Pat seem to be turning everything upside down. It's going to be interesting to see if the bots will be able to fix those holes.
 
IP Logged
omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: Arimaa Challenge format for 2006
« Reply #36 on: Aug 23rd, 2005, 11:08am »
Quote Quote Modify Modify

on Aug 23rd, 2005, 7:33am, Fritzlein wrote:

 
Oh, that's too bad.  I shouldn't be surprised, but I admit I am disappointed.  Omar, I apologize if my previous posts implied that you weren't thinking of the obvious sponsors and the things you could offer them.  I don't know much about marketing, and I was just speaking off the top of my head.
 
You've done a pretty darn good job of marketing Arimaa all by yourself, without the help of corporate sponsorship.  I don't think you have to do anything different for Arimaa to grow apace.  The friendly server and terrific game you have created are already enough of a draw to keep the community growing steadily.

 
Oh don't worry about it; the thought never even crossed my mind.
 
I would much rather have Arimaa be a game that grows slowly and steadily than have it suddenly explode into popularity. Remember it took the Internet almost 20 years before it reached a thousand nodes. But that slow early growth allowed it to build a solid infrastructure for the future.
 
  http://www.zakon.org/robert/internet/timeline/
 
Scroll down to the bottom to see the growth charts.
 
Of course Arimaa will never see that kind of growth, but what I mean to say here is that I think its better to focus on building a solid infrastructure and not worry about growth so much.
 
IP Logged
omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: Arimaa Challenge format for 2006
« Reply #37 on: Aug 24th, 2005, 5:36am »
Quote Quote Modify Modify

The new agressive techniques for winning against the bots are working great. But a side effect of that is that the selection process I was planning to use becomes sort of useless because a person who learns those techniques can pretty much go on winning as much as they want against the *2005CC bots. So I guess I might have to go back to just selecting the players directly. I would have liked the selection process to be directly related to how a player performs against the bots. Anyone have any suggestions for this situation.
« Last Edit: Aug 24th, 2005, 5:38am by omar » IP Logged
Arimanator
Forum Guru
*****



Arimaa player #1064

   


Gender: male
Posts: 158
Re: Arimaa Challenge format for 2006
« Reply #38 on: Aug 24th, 2005, 6:18am »
Quote Quote Modify Modify

on Aug 24th, 2005, 5:36am, omar wrote:
The new agressive techniques for winning against the bots are working great. But a side effect of that is that the selection process I was planning to use becomes sort of useless because a person who learns those techniques can pretty much go on winning as much as they want against the *2005CC bots. So I guess I might have to go back to just selecting the players directly. I would have liked the selection process to be directly related to how a player performs against the bots. Anyone have any suggestions for this situation.

 
One solution to avoid people rehearsing a recipe to beat one particular bot would be to impose an (unusual) piece setting for the human player, one different for each player (to avoid copy cats  Wink ) , with no time to prepare and see how each of us manages with it.
 
It would have the advantage of selecting a person on his/er ability to beat bots more than humans and as I said one can't resort to recipes, at least not easily, when one doesn't know where the main pieces will be at the start of the game, let alone how the bot will place its pieces in response to that.
 
 
« Last Edit: Aug 24th, 2005, 6:24am by Arimanator » IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Arimaa Challenge format for 2006
« Reply #39 on: Aug 24th, 2005, 7:08am »
Quote Quote Modify Modify

on Aug 24th, 2005, 6:18am, Arimanator wrote:

One solution to avoid people rehearsing a recipe to beat one particular bot would be to impose an (unusual) piece setting for the human player, one different for each player (to avoid copy cats  Wink ) , with no time to prepare and see how each of us manages with it.

 
That's an interesting idea.  To make it simple one could impose the condition that the setup of the pieces for the human player is completely scrambled by the server before each game.  Then every player would have to fight through terrible initial setups as well as the reasonable ones.
 
Omar, I never had any objection to your original method of choosing the human defenders, i.e. arbitrarily choosing whomever you want.  I still think that may be the best plan.  I don't see why there has to be any more structure than that.
 
But if you do want some objective criteria that have something to do with how well a human is likely to perform against an unknown bot next February, then it seems relevant to be able to beat all of last year's bots from a randomized starting position.  There may still be methods that reliably beat each bot, but at least they can't possibly be move-for-move copies: the winning methods will then indisputably deserve to be called strategies.
 
Perhaps, though, this criterion will still be too easy.  Whatever criteria you settle on, if it involves beating bots repeatedly, might be something that too many humans can accomplish.  That, in turn, might indicate that many folks are capable of defending the challenge at present, and humanity is not in any danger from the bots for the foreseeable future, so it really doesn't matter who defends the challenge.  Smiley
IP Logged

jdb
Forum Guru
*****



Arimaa player #214

   


Gender: male
Posts: 682
Re: Arimaa Challenge format for 2006
« Reply #40 on: Aug 24th, 2005, 8:11am »
Quote Quote Modify Modify

Quote:
I would have liked the selection process to be directly related to how a player performs against the bots. Anyone have any suggestions for this situation.  

 
Maybe something that involves the bots that attempt to qualify for the 2006 Computer Championship?
 
IP Logged
Arimanator
Forum Guru
*****



Arimaa player #1064

   


Gender: male
Posts: 158
Re: Arimaa Challenge format for 2006
« Reply #41 on: Aug 24th, 2005, 8:21am »
Quote Quote Modify Modify

on Aug 24th, 2005, 8:11am, jdb wrote:

 
Maybe something that involves the bots that attempt to qualify for the 2006 Computer Championship?
 

 
 
I am afraid that that would let "the cat out of the bag" prematurely and some might say that the world official contestant had got some tips out of these games and that the deck was stacked in favor of the human player.  
« Last Edit: Aug 24th, 2005, 8:21am by Arimanator » IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: Arimaa Challenge format for 2006
« Reply #42 on: Aug 24th, 2005, 6:21pm »
Quote Quote Modify Modify

Just to see what it would be like, I tried playing Bomb2005CC with my own pieces randomized.  You can see the result in game 18622.  Based on this one experience, I would say that randomizing the opening position definitely gets around the problem of formulaic wins, and requires one to show real strategic superiority over the bot to win.
 
Nothing went as smoothly as planned for me.  I didn't have time to launch the E+H attack before Bomb had attacked one of my home traps with E+D.  Such an attack from Bomb was feasible because my trap had only dogs and rabbits next to it.  Then I wanted to take the attacking dog hostage, but since I had rabbits on the second row, Bomb delayed the hostage-taking by pulling a rabbit into the trap.  In order to finally get the hostage, I had to use my camel instead of a horse, because my random setup had put both horses on the other side, but this exposed my camel more than I would have liked.
 
Even once I had gotten the upper hand (HDC for M), my further progress was hindered by the lingering effects of the original setup.  My second-rank rabbits on the east flank made it much more difficult for me to get Bomb's camel hostage into position, and by the time the rabbits had cleared out, my two stupid corner cats (left there from the start) weren't enough to hold up the camel on h6.  Eventually, after capturing the hostage camel for HR, my mass of central rabbits was a liability as much as an asset.  And finally, in the endgame it hurt my goal defense and made things scarier to have my cats on the same side instead of split.
 
On the other hand, Bomb still over-valued the camel and the camel hostage, still made reversible moves, still played too passively, and still showed a poor understanding of trap control at times, e.g. around f3 late.  These are strategic weaknesses that undermine Bomb's play, and probably will continue to do so next year, since they are not easy to fix.
 
It is my guess that anyone who can consistently beat all the 2005CC bots with their own pieces randomized at the start will be qualified to defend the next challenge.
« Last Edit: Aug 24th, 2005, 7:16pm by Fritzlein » IP Logged

99of9
Forum Guru
*****




Gnobby's creator (player #314)

  toby_hudson  


Gender: male
Posts: 1413
Re: Arimaa Challenge format for 2006
« Reply #43 on: Aug 24th, 2005, 10:59pm »
Quote Quote Modify Modify

Interesting experiment - it definitely sounds harder.
 
I'm a bit worried because in some ways this is a different game (like Fisher random vs chess).  It might favour people who are not very good at traditional arimaa opening theory, which might mess them up when they get into the challenge against a good bot.
 
But anyway, I think this solution is better than the original metric that Omar proposed.  Of course I think it is also fine if Omar chooses participants directly.
IP Logged
Adanac
Forum Guru
*****



Arimaa player #892

   
Email

Gender: male
Posts: 635
Re: Arimaa Challenge format for 2006
« Reply #44 on: Aug 25th, 2005, 12:46pm »
Quote Quote Modify Modify

on Aug 24th, 2005, 10:59pm, 99of9 wrote:
Interesting experiment - it definitely sounds harder.
 
I'm a bit worried because in some ways this is a different game (like Fisher random vs chess).  It might favour people who are not very good at traditional arimaa opening theory, which might mess them up when they get into the challenge against a good bot.

 
One of the things that attracted me to Arimaa and away from chess is the idea that every opening is unique and that there's room for original thought from the very beginning of the game.  Now that most games more or less have the same opening setup and similar opening tactics, and bots can be defeated repeatedly using the same techniques every times, I think it would be great to mix things up a bit.  I'm speculating here, but perhaps these random opening positions would closer to the spirit that Omar had in mind when he created a game with a variable opening setup ??
 
By the way, I'm also I fan of Fischer Random Chess so I am a bit biased  Smiley
IP Logged


Pages: 1 2 3 4  Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print

« Previous topic | Next topic »

Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB © 2000-2003. All Rights Reserved.