Arimaa Forum - Print Page


    
      
        Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Site Discussion >> Bot Ladder feature
        
(Message started by: omar on Feb 11^th, 2006, 3:54pm)

Title: Bot Ladder feature
Post by omar on Feb 11^th, 2006, 3:54pm

Just added a "Bot Ladder" link in the gameroom.

I had been meaning to add this feature for awhile now. This will provide beginners with easy access to the various bots on the site.

It might still have some bugs, so post them here if you see any.

Title: Re: Bot Ladder feature
Post by PMertens on Feb 11^th, 2006, 7:24pm

ok ... I found plenty of bots ahead of me ... looks like a bug :-D
Anyway ... cute feature

Title: Re: Bot Ladder feature
Post by omar on Feb 11^th, 2006, 9:23pm

Those are probably bots which you haven't played yet. You have to defeat the bots two time to pass them. It probably won't take you long to get to the top of the list :-)

I hope this feature makes the site more interesting for the beginners.

Title: Re: Bot Ladder feature
Post by Fritzlein on Feb 12^th, 2006, 1:16pm

I love the new bot ladder! I think it makes sense to have the lowest-rated bots be at the bottom of the ladder (except for ShallowBlue getting an automatic bottom). That way, instead of us guessing which bots will be easier and harder for beginners to overcome, the bots will naturally sort out that way. Also beginners will be presented with a much greater variety of opponents.

At the moment, it is still possible to bypass the presented order of bots by going to the Arimaa Bots page. Do you intend to continue to allow people to skip over some bots if they know about the backdoor, or will you require everyone to work up through all the lower bots in the ladder before playing higher ones?

Title: Re: Bot Ladder feature
Post by Fritzlein on Feb 12^th, 2006, 4:49pm

I just thought of two additional refinements, if they seem worthy of incorporating:

1) Since beginners now have a whole slew of weak bots to work through, we might only require them to beat each bot once to be able to play up. Beginners will still be learning more by beating each of eight bots once than each of four bots twice.

2) Instead of having the first bot on the ladder automatically be ShallowBlue, let the bot with the lowest rating be the first bot on the ladder, whether or not it is ShallowBlue. I'm actually curious to see the relative ratings of LocP1, ShallowBlue, and Arimaalon if they are allowed to float on equal footing.

However, to preserve the nice feature of unrated practice games, automatically make games against the lowest ladder bot be unrated. If people want to play unrated games against a bot other than the lowest on the ladder, they could still set their flag to play unrated.

Giving ShallowBlue equal status with the other bots would remove the last vestiges of artificiality from the ladder of bots that newcomers face. The ladder would instead be a totally natural one.

Title: Re: Bot Ladder feature
Post by frostlad on Feb 13^th, 2006, 12:43am

I do have a couple of things that I wish were slightly different from a kid that is just starting.
1. I have been trying to beat arimaazon for a week or so now, but no longer have the option of playing that bot on the bot ladder. Where did arimaazon go to? I know I could probably get to it through the bot pages but that was a nice challenge for me and wouldn't mind it being in the bot pages.
2. It might be nice for a beginner to have a choice of the next two bots to play. That way if one bot's strategy is just difficult for a beginner to deal with at first then they can play try to play the next highest bot.

I do really like the feature though. I hadn't been playing anything but the 4 standard bots because I still really don't know how to start my own games very well. And, I don't know how I would start it against the different bots that are out there. ( I know I'm showing how new I am and how little I've explored around so far ) I've just wanted to play a lot and the bots have been plenty hard for me to explore and start playing other bots. It is nice to know what bots you've progressed beyond and what ones you still have to work on and have a variety of bots.

Title: Re: Bot Ladder feature
Post by Ryan_Cable on Feb 13^th, 2006, 4:18am

Congratulations Ryan_Cable. You've reached the top.

I agree with frostlad. The old ladder bots should be kept on the new ladder, if only so we can continue tracking their ratings. The idea of making the next 2 bots available sounds good as well.

Title: Re: Bot Ladder feature
Post by PMertens on Feb 13^th, 2006, 4:33am

The next 2 sounds better to me as well ...
Anyway ... if the order of bots can change than we could repeatedly run into the problem Frostlad described:
You suddenly cannot play against your current archrival-bot anymore ....
Therefore it would be nice to be always allowed to play against a bot if you where allowed to do so once.

Something else:
Do not emphasize to much you are a noob, Frosty ... we are afraid of noobs ... they have a tendency to win the next championship :-/

Title: Re: Bot Ladder feature
Post by RonWeasley on Feb 13^th, 2006, 11:51am

I like the Bot Ladder idea very much. For short, I will call it "The Bladder."

Title: Re: Bot Ladder feature
Post by frostlad on Feb 13^th, 2006, 12:25pm

on 02/13/06 at 04:33:16, PMertens wrote:

That actually did happen to me last night. I played bot_bomb2005P1 and lost, then I wasn't able to play it again because it jumped bot_gno2005P2 in rating.
On one hand it is sort of annoying for that to happen, but it also gives me a reason to want to beat gno2005P2 so that I can have another crack at bomb. So I'm sort of torn on whether I don't like that or not.
The time when it would get annoying is if every time I try to play a bot ahead of me it flip flops with another bot and I have to try and learn how to beat a different bot. I don't know the likelyhood of two bots having a rating that close to each other.

Title: Re: Bot Ladder feature
Post by PMertens on Feb 13^th, 2006, 12:59pm

Hmmm .... I would have be annoyed.

I believe that bots are primarily a good exercise for concentration ...
To play against the same bot over and over will make you win against this one faster because you learn the specific weakness of this one bot and you can concentrate on it.
Playing against alternated bots might give the same exercise or even more, but certainly less motivation because it will take you longer before you see your first success.

Also the difficulty of a bot cannot be measured in rating points anyway.
A bot has a certain behavior that might suit a human or go against his nature ... one more reason to allow more than one and avoid alternation.

What would happen if I for example lose against the worst bot and win against the third worst ?
Someone who just managed the first one ... will he have to play the former third worst that now became worst ? or the second worst since he won against the first already ?

Title: Re: Bot Ladder feature
Post by Fritzlein on Feb 13^th, 2006, 3:00pm

It's hard for me to guess what it would feel like to try to work through the ladder in various configurations, because I'm no longer new to Arimaa. Frostlad, it is great to have your perspective while you are still working on overcoming certain bots for the first time. The rest of us talking here are looking from the perspective of being able to beat all the bots.

I can understand chafing at the restrictions on which bot is available to play next. Maybe I want to play a certain bot that is my nemesis, or maybe I want to avoid it and try a different bot instead. Maybe I want to play the same bot over and over, but I can't because it bubbles up in rating. On the other hand, maybe I don't want to keep playing the same bot, and I would prefer some variety, but I have keep banging my head against the wall because one bot happens to be lowest rated.

Part of me thinks that there should be no restrictions at all. Why not let people play whichever bot they want to play? That would avoid several types of frustration.

On the other hand, part of me likes the idea of being forced to play the lowest-rated bot, even if there are rating flip-flops, and even if different bots are different difficulty for different people. The sense of accomplishment in completing a challenge is directly related to how difficult (and frustrating?) it was to do it. Video games don't let you jump to level 17 if you are having trouble getting through level 16, and it is well that they don't. Getting past the barriers is the fun part.

So I guess I would leave it pretty much as it is now.

One new concern of mine is that people with slow connections might not be able to beat the Blitz bots even if they are otherwise sufficiently skilled. Maybe we should stick with just P1, P2, and Fast, i.e. only 3 versions of each bot. When the 2006 versions come into the mix, that will be almost 30 bots. (And by the way, we should definitely reduce it to one win to pass a level when the 2006 bots are added, if not sooner.) But playing at Blitz speed is a different kind of challenge, and perhaps a fun type of challenge, so I don't really object to having 4 versions of each bot.

Title: Re: Bot Ladder feature
Post by Ryan_Cable on Feb 13^th, 2006, 4:12pm

Omar told me that there are two different ladders, one for beginners and an advanced one that top players are seeing. The beginner ladder does not have the Blitz bots.

Title: Re: Bot Ladder feature
Post by omar on Feb 13^th, 2006, 4:14pm

Thanks for the suggestions everyone.

I changed it so that if you played a bot before you can always play it again.

Also added a page to list all the bots that are available to play. This is like the back door if you really want to play a particular bot.

http://arimaa.com/arimaa/bots/index.cgi

The bots listed in the ladder are ones that move quickly. The slower moving bots can be accessed through the back door.

Ryan had asked me in the chat room why bot_Arimaanator is not listed. It's because this bot runs on a windows machine and logs in to the game room from my notebook PC. The bots in the list are all on the arimaa server which is a linux machine.

Also games won by timeout or repetition do not count; you must win by goal or immobilization.

Title: Re: Bot Ladder feature
Post by omar on Feb 13^th, 2006, 4:33pm

on 02/13/06 at 16:12:28, Ryan_Cable wrote:

Omar told me that there are two different ladders, one for beginners and an advanced one that top players are seeing. The beginner ladder does not have the Blitz bots.

The beginner ladder has:
bot_ShallowBlue bot_Arimaazilla
bot_Bomb2005P1 bot_Clueless2005P1 bot_GnoBot2005P1 bot_Loc2005P1
bot_Bomb2005P2 bot_GnoBot2005P2 bot_Loc2005P2

The advanced ladder also includes:
bot_Bomb2005Fast bot_Clueless2005Fast bot_GnoBot2005Fast bot_Loc2005Fast
bot_Bomb2005Blitz bot_Clueless2005Blitz bot_GnoBot2005Blitz bot_Loc2005Blitz

Title: Re: Bot Ladder feature
Post by omar on Feb 13^th, 2006, 5:33pm

Here's the feedback message that I got which convinced me to put just the fast bots on the ladder.

Quote:

Kal wrote:

The bots make playing this game so much fun because its exciting,
quick, and challenging. (Exciting because playing against a computer is
always a thrill. Quick because your bots are very responsive and most
people dont have lots of time. Challenging because its fun to see if we
can move up the levels from one bot to the next).

I am currently on bot_Arimaazon and I have lost interest in playing.
Its not because I havent beaten bot_Arimaazon (infact I have beaten it
but not twice in a row) I lost interest because of the time. This bot
takes the full one-minute on each all of its moves. If you look at the
average moves I make in a game, it comes to around 26 moves. Double
this number for the opponent bot and its a total of 52 moves. A minute
a move comes to almost an hour. It was exciting playing the other bots
but now its too time consuming.

Its a very fun game and my son Emaad (age 6) learned to play all by
himself just using the flash instructions. He even beat a few bots, but
I think it was by luck.

It's true that people especially those who are new to the game perfer to have the games finish quickly. Also we tend not to pay attention to time when the moves are happening quickly. Thus we are more likely to play several quick games rather than one slow game. Since the bot ladder will mostly be used by new players I think it makes sense to have just the fast bots on it. The advanced players will have reached the top and will not really use it after that.

Kal actually is my brother. He is also responsible for convincing me to not use a fixed setup in Arimaa and to let the players decide the setup.

Title: Re: Bot Ladder feature
Post by Fritzlein on Feb 13^th, 2006, 10:19pm

What triggers the advanced ladder to kick in? It can't be beating all the beginner bots, because I haven't done that yet.

It seems the bot ladder has triggered a bit of a boom in human vs bot activity. Of course part of it is established players realizing there are some bots they have never beaten, but I'm hoping part of it is that newcomers are also having more fun.

Well done, Omar!

Title: Re: Bot Ladder feature
Post by frostlad on Feb 14^th, 2006, 2:45pm

Just a thing to notice with the ratings. Shallowblue has already dropped to the bottom of the ratings on the ladder. I just thought that it was sort of interesting that the bots are already sorting themselves out.
I am also curious as to what triggers the advanced ladder to kick in. I still have a few bots to work through, but is it designed so that when I have worked past the top bot that the advanced ladder kicks in?

Title: Re: Bot Ladder feature
Post by omar on Feb 16^th, 2006, 7:33am

on 02/13/06 at 22:19:15, Fritzlein wrote:

What triggers the advanced ladder to kick in? It can't be beating all the beginner bots, because I haven't done that yet.

If a player has a rating below 1500 or a rating uncertainty above 60 then they get the beginner ladder, otherwise the advanced ladder. If a player has finished the beginner ladder then they can also move up to the advanced ladder.

Title: Re: Bot Ladder feature
Post by omar on Feb 17^th, 2006, 8:09am

I have setup a program to automatically have the bots play each other. Once an hour it checks to see how many bots are currently playing live games. If two or more bots are playing then it does not start a new game. If less than two bots are playing then it randomly selects two bots to play each other. The program is careful about what time control is used. For example BombP1 can play LocCC but only if the game is setup with the time control that LocCC wants. Also it will never pair BombBlitz against LocCC because both would try to use the full amount allowed by the time control.

The intent is to help stabilize the bot ladder. Also because some bots get played by humans much more than others, this will help distribute the rating points gained or lost by the system.

We will see a lot of BvB games in the listing of recent games, but I will soon add a switch to that page to show only human/bot/all games.

Title: Re: Bot Ladder feature
Post by PMertens on Feb 17^th, 2006, 11:15am

great feature ...

Title: Re: Bot Ladder feature
Post by Fritzlein on Feb 17^th, 2006, 2:36pm

on 02/17/06 at 08:09:24, omar wrote:

I like it: I'm curious to see how bots do against each other.

Quote:

The program is careful about what time control is used. For example BombP1 can play LocCC but only if the game is setup with the time control that LocCC wants. Also it will never pair BombBlitz against LocCC because both would try to use the full amount allowed by the time control.

There is a potential problem playing fixed-performance bots against variable-performance bots. Suppose that BombP1 plays regularly against LocCC and LocBlitz, long enough that their relative ratings get fairly accurate. BombP1 will play at a fixed skill, but LocCC will play better than LocBlitz, because it has more time to think. Therefore eventually LocCC will earn a higher rating than LocBlitz. But for humans it would make no sense that LocCC gets a higher rating. Even if LocCC objectively plays better moves than LocBlitz, it is harder to beat LocBlitz due to the time pressure.

We could address this problem by implementing a new rating system in which players have different ratings at different speeds, but in the mean time I suggest we limit bot games to Blitz vs. Blitz, Fast vs. Fast, Slow vs. Slow, and Fixed Performance vs. Fixed Performance. Otherwise these random games could make bot ratings less accurate, at least from a perspective of how easily humans can beat them.

Quote:

The intent is to help stabilize the bot ladder. Also because some bots get played by humans much more than others, this will help distribute the rating points gained or lost by the system.

The second benefit you tout (a better distribution of rating points) is the key benefit of extra bot vs. bot games. I love it. At present newbies inject points into bots at the bottom while experienced players take points away from bots at the top. This has created a compressed scale of bot ratings. When the bots play each other, the scale of bot ratings will expand and push outward on both ends, as it should be.

On the other hand, you should prepare for disappointment on you first point, namely stability. I'm almost positive random bot vs. bot play will not stabilize the bot ladder, except insofar as it gets some bot RU's down to 30 faster than they will get there from playing humans. When all the ladder bots have the minimum RU, a greater number of bot vs. bot games will actually destabilize the ladder, causing more flip-flops in rank and more frequent swings in rating.

It's like flipping a coin: You are less likely to have a run of ten heads in a row in 100 flips than you are in 1000. If bots play lots and lots of games, it becomes more likely that a given bot's rating will get totally out of whack sometime. True, playing lots of games will also serve to get the ratings of the bots back into line when they become temporarily inaccurate, but that's greater accuracy, not greater stability.

(Actually, I prefer accuracy to stability. I'd rather have a bot hovering within 200 points of its true rating than hovering within 50 points of a rating that is 200 points wrong.)

Title: Re: Bot Ladder feature
Post by 99of9 on Feb 17^th, 2006, 4:58pm

on 02/17/06 at 08:09:24, omar wrote:

If less than two bots are playing then it randomly selects two bots to play each other.

Why not wait until there are 0 bots playing? By your sheme it might start a game at the same time as one of the Challenge games!!! In principle I think it's better not to have any sharing of the CPU.

Title: Re: Bot Ladder feature
Post by Ryan_Cable on Feb 18^th, 2006, 6:55am

I think this is a pretty useful feature. Given enough time, it should get the bots reasonably close to an accurate ordering at least. Currently, the bots are ridiculously disordered.

on 02/17/06 at 14:36:49, Fritzlein wrote:

We could address this problem by implementing a new rating system in which players have different ratings at different speeds, but in the mean time I suggest we limit bot games to Blitz vs. Blitz, Fast vs. Fast, Slow vs. Slow, and Fixed Performance vs. Fixed Performance.

I don’t think this is necessary. I think the speed issue is vanishingly small in comparison to all of the other intransitivity that exists in our player pool.

on 02/17/06 at 14:36:49, Fritzlein wrote:

At present newbies inject points into bots at the bottom while experienced players take points away from bots at the top. This has created a compressed scale of bot ratings. When the bots play each other, the scale of bot ratings will expand and push outward on both ends, as it should be.

This should help a little, but I think we would have to have at least an order of magnitude more BvB games than HvB games to significantly spread the end points of the bot ratings. In particular, I doubt the bottom of the rating scale will drop below 1100 for any significant period of time. I think the true rating of the bottom of the ladder is <=800 points, but since noobs enter the pool ~500 points overrated they will always pull up the bottom of the ladder.

From playing them against each other, it appears that ShallowBlue, GnoBot2005P1, and Arimaalon are roughly tied for bottom of the ladder. (ShallowBlue is slightly more aggressive with its Rs, which lets it grab a few goals despite material disadvantage.) However, ShallowBlue has a 47% winning record, while Arimaalon has 24% and GnoBot2005P1 has 25%. I attribute this to the fact that ShallowBlue has been the very first bot noobs faced and that they would only play the other bots after they had beaten ShallowBlue and brought their true rating a little closer to their listed rating.

on 02/17/06 at 14:36:49, Fritzlein wrote:

True, playing lots of games will also serve to get the ratings of the bots back into line when they become temporarily inaccurate, but that's greater accuracy, not greater stability.

I think this is a rather silly definition of stability. I believe any increase in the rate of games being played would increase instability by this view. The only way to get stability would be to play no games at all. However, I believe if you randomly choose an instant to measure the difference in listed ratings and true ratings, the expected value of this measurement should decrease as the rate of game play increased. I guess this is close to what you mean by greater accuracy.

I think a G factor should be added to the time controls of all of the bots. bot_GnoBot2005P2 v bot_Bomb2005P1 lasted 195 moves. bot_Loc2005P2 v bot_GnoBot2005P2 looked like it would have gone on forever if Gnobby hadn’t mysteriously timed out at move 252.

Title: Re: Bot Ladder feature
Post by omar on Feb 18^th, 2006, 6:10pm

on 02/17/06 at 14:36:49, Fritzlein wrote:

I like it: I'm curious to see how bots do against each other.

There is a potential problem playing fixed-performance bots against variable-performance bots. Suppose that BombP1 plays regularly against LocCC and LocBlitz, long enough that their relative ratings get fairly accurate. BombP1 will play at a fixed skill, but LocCC will play better than LocBlitz, because it has more time to think. Therefore eventually LocCC will earn a higher rating than LocBlitz. But for humans it would make no sense that LocCC gets a higher rating. Even if LocCC objectively plays better moves than LocBlitz, it is harder to beat LocBlitz due to the time pressure.

We could address this problem by implementing a new rating system in which players have different ratings at different speeds, but in the mean time I suggest we limit bot games to Blitz vs. Blitz, Fast vs. Fast, Slow vs. Slow, and Fixed Performance vs. Fixed Performance. Otherwise these random games could make bot ratings less accurate, at least from a perspective of how easily humans can beat them.

Yes, I agree, but hopefully it won't be too bad. If the CC bots start getting a higher rating it will encourage the humans to play them to get some rating points. Im getting tempted to play CluelessCC now that it's at the top of the list.

I would eventually like to change the rating system to take game speed into consideration.

Quote:

On the other hand, you should prepare for disappointment on you first point, namely stability. I'm almost positive random bot vs. bot play will not stabilize the bot ladder, except insofar as it gets some bot RU's down to 30 faster than they will get there from playing humans. When all the ladder bots have the minimum RU, a greater number of bot vs. bot games will actually destabilize the ladder, causing more flip-flops in rank and more frequent swings in rating.

It's like flipping a coin: You are less likely to have a run of ten heads in a row in 100 flips than you are in 1000. If bots play lots and lots of games, it becomes more likely that a given bot's rating will get totally out of whack sometime. True, playing lots of games will also serve to get the ratings of the bots back into line when they become temporarily inaccurate, but that's greater accuracy, not greater stability.

I really wanted to just get the RU's down and let the ratings become a bit more accurate. I don't execpect them to ever stabilize. From simulations I had done a few years ago I know there is always a bit of drift in the ratings.

Title: Re: Bot Ladder feature
Post by omar on Feb 18^th, 2006, 6:15pm

on 02/17/06 at 16:58:46, 99of9 wrote:

Why not wait until there are 0 bots playing? By your sheme it might start a game at the same time as one of the Challenge games!!! In principle I think it's better not to have any sharing of the CPU.

Well since I only check once an hour there is a good chance that someone will be playing a bot when I check and so a lot less games would be arranged.

There isn't any impact on the challenge match games from having the ladder bots play each other. bot_Bomb is running on a seperate server and has the CPUs all to itself.

Title: Re: Bot Ladder feature
Post by frostlad on Feb 20^th, 2006, 5:34am

Just an idea, but would the bots stabilize better if they were only able to play bots within 2 spots of themself on the ladder? Then it would be similar to how humans have to move up the ladder if the bot is in the wrong spot on the ladder and hopefully the better bots would move up if it can beat the bot one or two ahead and consistently beats the bots right below it.
What do you guys think of that?

Title: Re: Bot Ladder feature
Post by Ryan_Cable on Feb 20^th, 2006, 8:46am

I think it is most efficient to have the bots play bots one class away. Certainly, it is the most efficient way to separate the compressed bot ratings. Playing more than one class away gains the stronger bot very few points; playing within the same class has too much randomness, minimizing the long run point transfer per game. Since most bots are more than one class away from a given bot, I think randomly playing bots would probably take at least 10 times more games to reach equilibrium. I think the following classes exist:

bot_GnoBot2005P1, bot_ShallowBlue, bot_Arimaalon
bot_Arimaazilla, bot_Loc2005P1
bot_Loc2005Fast, bot_Loc2005Blitz
bot_Bomb2005P1
bot_Clueless2005P1
bot_Clueless2005P2
bot_Bomb2005P2
bot_Clueless2005CC
bot_Bomb2005CC

Not all bots are sorted out, and at least one more class is likely to exist. In the absence of human players, these classes should be separated by 200 to 300 points, for a total of ~2000+ points between bottom and top. There is almost certainly some intransivity in these classes, but assuming there are no random timeouts, crashes, etc., I think it is perfectly reasonable to think that Bomb would win 10^5 in a row against ShallowBlue. In fact, there probably are at least 2 super-classes that theoretically should be sepreated by infinite points.

So long as HvB games are as frequent as BvB games and so long as there are much fewer classes of humans, the ladder is likely to stay much more compressed, but I think in the long run these classes should be separated by ~125+ points, for a total of 1000+ points between top and bottom. The bottom of the ladder is likely to be anchored at ~1200 due to the fact that most humans playing their first game are massively overrated at 1500. In my grand vision, Fritzlein would be pushed to ~2600+.

PS Instead of having the script run once per hour, you should have it run whenever a game ends, if that is possible. Games between the non-Clueless P1/P2/Blitz bots often only take a few minutes, so the server will often be sitting idle.

Title: Re: Bot Ladder feature
Post by nbarriga on Feb 21^st, 2006, 5:55pm

This bot ladder is really a great feature. I was loosing interest in playing the game because i couldn't beat arimaazon, and i didn't know what other bot to play. This ladder helped in two ways:

1) It boosts the competitive spirit by having a kind of personal tournament with a specific goal to achieve.
2) It ordered the ratings of the bots more accurately, so i can know which bot to play with.

Thanks for the renewed fun.

Title: Re: Bot Ladder feature
Post by omar on Feb 22^nd, 2006, 2:00am

on 02/20/06 at 05:34:13, frostlad wrote:

It would probably make the ratings stabilize faster. But as Karl mentioned there are two issues we are dealing with; stability and accuracy. A stable rating doesn't necessarily mean that it's an accurate rating. A rating is supposed to represent the players performance against the field. The more broad range of opponents a player has played the more meaningful the rating is. If a player plays only a few opponents the rating will not reflect the player's performance against the field and only represent the player's performance against the strategies and playing styles of the limited opponents.

Title: Re: Bot Ladder feature
Post by omar on Feb 22^nd, 2006, 2:09am

on 02/20/06 at 08:46:13, Ryan_Cable wrote:

PS Instead of having the script run once per hour, you should have it run whenever a game ends, if that is possible. Games between the non-Clueless P1/P2/Blitz bots often only take a few minutes, so the server will often be sitting idle.

I did consider this when I was writting that script, but I decided not to for a couple of reasons. I didn't want the recent games page to be filled with just the BB games and the recent HH games pushed off the page. This is not a problem now since we can select out the HH games. The other reason was so that the bot's opponents would not be too many other bots; it should be a fairly equal mix of humans and bots.

Title: Re: Bot Ladder feature
Post by Fritzlein on Feb 22^nd, 2006, 2:34pm

Factoid: Arimaalon is now the lowest-rated bot by 100 points, not because it is the worst, but because it isn't a part of the bot ladder, and therefore doesn't get the infusion of newbie inflationary points that ShallowBlue and GnobotP1 are getting. It gets beaten down by other bots, but doesn't get pumped up by the noobs. Another argument for Ryan's new rating model?

Title: Re: Bot Ladder feature
Post by omar on Feb 22^nd, 2006, 6:04pm

Thanks for pointing this out Karl; I had forgot to put Arimaalon on the bot ladder.

Title: Re: Bot Ladder feature
Post by frostlad on Feb 22^nd, 2006, 8:15pm

What about arimaazon? that was the bot that I was missing playing. I know that I can go play it in the bots page but it also isn't on the bot ladder. Just thought I'd point it out.

Title: Re: Bot Ladder feature
Post by Fritzlein on Feb 24^th, 2006, 7:44am

Thanks for adding Gnobot2006 to the ladder, Omar. You are really working overtime!

Title: Re: Bot Ladder feature
Post by 99of9 on Feb 24^th, 2006, 4:26pm

Don't forget to add your results to the botbashing table as they come. Otherwise we'll have to hunt through them later.

Title: Re: Bot Ladder feature
Post by omar on Feb 25^th, 2006, 8:43am

on 02/22/06 at 20:15:06, frostlad wrote:

What about arimaazon? that was the bot that I was missing playing. I know that I can go play it in the bots page but it also isn't on the bot ladder. Just thought I'd point it out.

Arimaalon takes more than 30 sec to move. I've tried to include only bots which move fast on the ladder. It seems that people perfer playing bots that move fast over ones that move slow. It makes sense since people usually don't have a lot of time for a game. For humans a fast moving bot can actually feel harder than a slow moving bot if the human is also forced to move fast. Although I would say it a different kind of hardness; it requires faster thinking and reaction time as opposed to deep, strategic thinking.

But you can always play any bot directly from the bot list page.

Title: Re: Bot Ladder feature
Post by omar on Feb 25^th, 2006, 8:46am

on 02/24/06 at 07:44:28, Fritzlein wrote:

Thanks for adding Gnobot2006 to the ladder, Omar. You are really working overtime!

Yes, and I just finished adding all the bots except Bomb.

You can thank Ryan for motivating me :-)

Title: Re: Bot Ladder feature
Post by IdahoEv on Feb 25^th, 2006, 4:31pm

I'll add another vote to the need for some mechanism to get past simple bots more quickly, at least in light of last night's addition of 6 or so more low-end bots to the ladder.

I had worked my way up to the 1500-level bots, but when I logged on last night I now had to fight my way past ArimaaScoreP1, ArimaaScoreP2, Arimaalon, Aamira2006P1, and GnoBot2006P1 to get back to anything interesting. They were all trivial to beat, but it took about three hours of time.

Possible ideas:
* Only one win required. (I disagreed before with this suggestion, but with as many bots as we have now, it seems more reasonable).
* An option to skip any bot whose rating is N points (100?) or more below your own.

The second one has an intriguing feature. If you jump ahead of bot A using it but then lose a couple of games against more difficult bot B, your rating will fall ... and you may find that you have to fight bot A after all. The system is self-correcting in that sense.

It would still make it possible to keep adding interesting bots to the ladder as they are created, without making the ladder too long and cumbersome.

Title: Re: Bot Ladder feature
Post by 99of9 on Feb 26^th, 2006, 1:57am

I like the rating skipping suggestion.

Title: Re: Bot Ladder feature
Post by Ryan_Cable on Feb 27^th, 2006, 3:01am

ShallowBlue just reappeared ahead of me in my ladder. Based on the Recent Games, it seems like it reappeared ahead of other people who have beaten it twice as well.

Title: Re: Bot Ladder feature
Post by Fritzlein on Feb 27^th, 2006, 12:12pm

I assumed it appeared ahead of me because all 12 of my wins against ShallowBlue were unrated.

Title: Re: Bot Ladder feature
Post by RonWeasley on Feb 27^th, 2006, 12:51pm

I assumed it appeared ahead of me because Malfoy was trying to insult me. As long as everyone else got equal treatment, I guess it's OK.

Title: Re: Bot Ladder feature
Post by omar on Feb 28^th, 2006, 1:54pm

Decided to change the bot ladder again. The ladder was getting too tall with all the new bots, so I broke it up into levels. Also changed it so that only one win is needed to move past the bot; with so many bots two wins was getting a bit execessive. Initially I was think of removing the 2005 bots and keeping it two wins, but now I think having more bots but only one win is better. Also decided to strictly require beating the lowest rated bot (within a level) to move up to the next one to preserve the concept of climbing the ladder one rung at a time.

The player info page now has a link to show you where the player is on their ladder.

Title: Re: Bot Ladder feature
Post by Ryan_Cable on Feb 28^th, 2006, 4:40pm

The Arimaa Bots Available to Play links for bot_Loc2006P2, bot_Loc2006Blitz, and bot_Loc2006Fast all bring up pages titled bot_Loc2006P1 Control Page (the URLs are correct though), which do in fact start bot_Loc2006P1. As a result, when trying to start bot_Loc2006P2 from the Bot Ladder page, I get an error message in the popup window and bot_Loc2006P1 appears in the Gameroom.

Also, I played bot_Clueless2006P1 as the last bot I needed to beat to advance past Level 2, but then it reappeared above me as the first bot of Level 3.

Otherwise, the new level system seems to be very good. I especially like the tips at the top of the page.

Title: Re: Bot Ladder feature
Post by frostlad on Mar 1^st, 2006, 12:05pm

loc2006P2 appears to be broken for me when I start a game.

I really like the new ladder with hints on how to beat each level of bots. Great job on that Omar.
What is the breakdown of the levels? Just curious to know.

Title: Re: Bot Ladder feature
Post by Fritzlein on Mar 1^st, 2006, 12:48pm

on 02/28/06 at 13:54:40, omar wrote:

Also changed it so that only one win is needed to move past the bot; with so many bots two wins was getting a bit execessive.

Thanks for all the work you put in on this great feature, Omar. Is it accidentally still set at two wins per bot? I just beat ShallowBlue, but it is still above me on my ladder.

Title: Re: Bot Ladder feature
Post by Ryan_Cable on Mar 2^nd, 2006, 4:16am

on 02/22/06 at 02:09:37, omar wrote:

It might be a slightly more efficient use of the server, if you have the script run every time a game ends, but only start a game if there are no bots playing. My guess is it would net a few more games most days, but not so many as to be overly disruptive to the ratings.

I think balancing the ratio of BvB to BvH games for individual bots is more important than limiting the gross number of BvB games. Bomb2005CC is on a 14 game winning streak, all against bots (more that 1/3 of the way to my prediction of 2000+ already :-)). In fact, all of the CC bots seem to be playing mostly bots. One possibility is to base a bot's probability of being picked to play a game on the number of human opponents it has had in its last ~20 games. A simpler approach is just to ban bots from playing bots if they haven't played a human in their last ~5 games. This might also allow you to safely turn the Join Games feature back on.

Title: Re: Bot Ladder feature
Post by frostlad on Mar 2^nd, 2006, 2:49pm

Just one thing to point out with loc2006P2 being broken...

I was looking through and all of the bots in level 3 above loc have mostly been playing other bots. All of the bots in level 2 have been playing a lot more humans. Personally, I am unable to move any farther up the ladder because loc is broken, I wonder if that is happening to other people.
I have enough to entertain me currently with the postal tournament and playing some of the level 2 bots, but I would imagine other newbies are in the same position as me right now, and might get frustrated at not being able to move up.

I really do like the leveled ladder idea and understand that bugs are bound to happen. And, I know you work really hard on this stuff omar, but if figuring out loc2006P2's bug is hard would it be possible to remove it from the ladder in the meantime? That way the people at that level would be able to progress.

Title: Re: Bot Ladder feature
Post by Fritzlein on Mar 3^rd, 2006, 7:20pm

on 03/02/06 at 14:49:59, frostlad wrote:

Personally, I am unable to move any farther up the ladder because loc is broken, I wonder if that is happening to other people.

You can skip ahead by following the lobby link "Arimaa Bots" and then "Bots Available to Play". However, most newcomers won't be reading the forum, so they will be stymied by this bug.

Title: Re: Bot Ladder feature
Post by IdahoEv on Mar 3^rd, 2006, 9:42pm

Heh. For those of us who STILL CAN'T BEAT the level two bots, it hardly matters. (sob).

Scuse me. Clueless2005P1 just got me again. Had to vent.

Title: Re: Bot Ladder feature
Post by omar on Mar 6^th, 2006, 9:04am

on 03/01/06 at 12:05:38, frostlad wrote:

Sorry I didn't see this earlier. Karl notified me about it by email.

While editing the config file of Loc2006P2 I forgot to change the username for the login. It was logging in as Loc2006P1, but playing like P2 :-) Thanks for notifying me.

Title: Re: Bot Ladder feature
Post by Ryan_Cable on Mar 6^th, 2006, 2:06pm

Thanks for fixing Loc2006P2 omar. However, Loc2006Fast and Loc2006Blitz still have the same problem.

Title: Re: Bot Ladder feature
Post by nbarriga on Mar 6^th, 2006, 4:24pm

bot_Loc2006P1 is playing at blitz speed, i think it should not. If it was an error, can i change the game to unrated?

Title: Re: Bot Ladder feature
Post by 99of9 on Mar 6^th, 2006, 4:47pm

That seems like an error to me too. We don't want to force newbies to play blitz in their first few games!!

Title: Re: Bot Ladder feature
Post by Fritzlein on Mar 7^th, 2006, 11:30am

For Omar's sanity, we should probably report bugs via the "Contact" link at http://arimaa.com/arimaa, instead of in the forum. Omar doesn't check the forum every day, and as active as it has been lately, there are probably more than a dozen new posts here every time he checks, so stuff can get lost in the shuffle. For example, Ryan's report that Loc2006Fast and Loc2006Blitz aren't working got scrolled up by Nbarriga's report that bot_Loc2006P1 is playing at blitz speed, so Omar might have seen the latter and not the former. (Say, maybe they're the same bug, i.e. trying to play LocFast starts the wrong bot at the right speed.)

Posting bugs to the forum is good in that other people can verify and discuss them, and sometimes they turn out to be features rather than bugs. Also feature requests are usually not very urgent and deserve to be aired out before implementation anyway. But if something is bugging you enough that you want it fixed soon, the forum may not be best place to mention it.

Title: Re: Bot Ladder feature
Post by PMertens on Mar 23^rd, 2006, 12:55pm

I believe that timeout victories should be made legal.
Several bots do have a time-problem and I would hate to see someone being frustrated because of that ...

Title: Re: Bot Ladder feature
Post by Fritzlein on Mar 23^rd, 2006, 5:20pm

For my part, I just gave up on the ladder when I ran into Clueless2006Blitz timing out every game. I guess some people have had fun solving that puzzle by learning how to win in under 30 moves, but winning as fast as possible has never been very interesting to me.

Anyway, as much as the bot ladder needs to be fixed, I think the bot needs to be fixed even more. It's getting a ridiculously low rating due to all of its timeouts.

Title: Re: Bot Ladder feature
Post by IdahoEv on Mar 23^rd, 2006, 7:33pm

I think Omar should simply disqualify any bot that times out more than some N% of the time as an invalid player for the ladder. I think 10% would be fair.

I keep forgetting about Loc2005 and phant barricades.

Title: Re: Bot Ladder feature
Post by PMertens on Mar 24^th, 2006, 10:13am

I mainly am for my proposal because I believe it the fastest solution ... not the best forever.
Some people have the problem now and may not want to wait for the bots to be fixed (but of course that would clearly the cleaner solution)

Title: Re: Bot Ladder feature
Post by PMertens on Mar 25^th, 2006, 11:43am

Another tiny little thing which is probably totally unimportant to most people at the moment :-[

When finishing the bot-ladder (yes, I had some time to kill the last 2 weeks) you are suddenly shown the Level 1 bots again with a NUL-bot above to play against ...

If I can wish for a page to show it would probably be the complete list of bots to play ... but as I said:
a) unimportant b) not urgent c) I guess the majority does not really care yet ;)

Title: Re: Bot Ladder feature
Post by omar on Mar 27^th, 2006, 11:08am

Thanks Paul. Actually that's what it was supposed to do. But I didn't get to test it out til someone reached the top :-)