Author |
Topic: ratings for 2010 CC bots (Read 3276 times) |
|
omar
Forum Guru
Arimaa player #2
Gender: 
Posts: 1003
|
 |
ratings for 2010 CC bots
« on: Jul 29th, 2010, 12:00am » |
 Quote  Modify
|
I just finished setting up the bots from the 2010 Computer Championship. Each bot has 5 versions:
CC - playing at time control of 2 minutes per move
P1 - search limited to 1 ply
P2 - search limited to 2 ply
Fast - playing at 30 seconds per move
Blitz - playing at 15 seconds per move
The ratings for these bots are currently set at 1400, but should be set something more reasonable based on ratings of the previous year bots or the developer bots ratings. Any suggestions on how to set the ratings?
|
|
 IP Logged |
|
|
|
rbarreira
Forum Guru
Arimaa player #1621
Gender:
Posts: 605
|
|
Re: ratings for 2010 CC bots
« Reply #1 on: Jul 29th, 2010, 1:50am » |
Quote Modify
|
Thanks a lot for the bots omar.
I was looking at the current ratings of bots from past years, here are my rough suggestions:
CC - Set them to the ratings at this page. Maybe minus 100 points for the bots which are parallel if you want to. I think all of them are parallel except for bomb and sharp.
P1 - CC minus 300 points.
P2 - CC minus 150 points.
Fast - equal to the CC? not sure about this one.
Blitz - CC plus 100 points.
|
| « Last Edit: Jul 29th, 2010, 3:06am by rbarreira » |
IP Logged |
|
|
|
arimaa_master
Forum Guru
Arimaa player #2010
Gender:
Posts: 358
|
|
Re: ratings for 2010 CC bots
« Reply #2 on: Jul 29th, 2010, 4:44am » |
Quote Modify
|
on Jul 29th, 2010, 12:00am, omar wrote:I just finished setting up the bots from the 2010 Computer Championship. Each bot has 5 versions:
CC - playing at time control of 2 minutes per move
P1 - search limited to 1 ply
P2 - search limited to 2 ply
Fast - playing at 30 seconds per move
Blitz - playing at 15 seconds per move
The ratings for these bots are currently set at 1400, but should be set something more reasonable based on ratings of the previous year bots or the developer bots ratings. Any suggestions on how to set the ratings?
|
|
Thanks Omar, that is great.
However bot_Clueless2010P1 doesn´t seem to start properly.
Maybe there is some info missing at Control Page - Bot info: "not available" and Max bots allowed is: where should be number I guess.
|
|
IP Logged |
|
|
|
rbarreira
Forum Guru
Arimaa player #1621
Gender:
Posts: 605
|
|
Re: ratings for 2010 CC bots
« Reply #3 on: Jul 29th, 2010, 4:53am » |
Quote Modify
|
Clueless2010P1 worked fine for me. Twice so far.
Maybe it only allows one instance of each bot?
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: ratings for 2010 CC bots
« Reply #5 on: Jul 29th, 2010, 8:10am » |
Quote Modify
|
rbarreira: thanks for the suggestion. Based on the time control most often played by the developer bot I guess the rating of the CC or Fast bot can be set. For example Marwin has earned its developer bot rating by playing mostly at CC speed whereas Clueless has played most of its games at the Fast speed. But how much to add or subtract to this base rating to get the ratings for the different versions is not clear. But then again, we just need to be in the ball park and not exact.
arimaa_master: I think I have all the bots set at only 1 instance right now. Will change the P1 and P2 bots to allow more instances.
Janzert: no, I had not seen that. I'll disable the P1 and P2 versions for now.
|
| « Last Edit: Jul 29th, 2010, 9:00am by omar » |
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: ratings for 2010 CC bots
« Reply #6 on: Jul 29th, 2010, 12:30pm » |
Quote Modify
|
I should also mention that I've set all the bots to use only 1 core. I was going to let the CC bots use 4 cores, but I noticed this morning that the load went up very high and the server became very slow when onigawara was playing Marwin2010CC; so I've set all bots to use only 1 core. GnoBot does not provide an option to limit the number of cores it uses; so I've disabled CC, Fast and Blitz version (since even the Blitz version can think for minutes if it is in trouble and has enough reserve); the P1 and P2 versions are still available, but set to allow only 1 instance. The P1 and P2 versions of all other bots are set to allow up to 5 instances.
|
|
IP Logged |
|
|
|
rbarreira
Forum Guru
Arimaa player #1621
Gender:
Posts: 605
|
|
Re: ratings for 2010 CC bots
« Reply #7 on: Jul 29th, 2010, 12:49pm » |
Quote Modify
|
I think it makes sense to set them to use one core, with 4 cores their performance would greatly vary according to the load on the server, which means their level of play would be quite unpredictable.
Of course the performance will still sometimes decrease even with just one core, but it's not that common for people to be playing four CC/Fast/Blitz bots simultaneously.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: ratings for 2010 CC bots
« Reply #8 on: Jul 29th, 2010, 5:41pm » |
Quote Modify
|
I agree with limiting bots to one core, and with disabling bots that don't give you the option to limit to one core. We should ask 99of9 about this; maybe there is a non-obvious flag in GnoBot 2010.
As for setting the ratings, I think you should just do it because people are already starting to play the bots. I suggest taking a base rating for a bot to be the rating of the developer bot. Then from the base rating.
bot_Blitz +100
bot_Fast +50
bot_CC -50
bot_P2 -100
bot_P1 -400
If the developer bot doesn't have a rating or you think it is inaccurate, key on one of last year's versions, with the relative gaps all the same. Obviously this won't work correctly, because different bots have different definitions of P1 and P2, depending on extensions. Still, as you say, we only need to get in the ballpark and then let the ratings float.
|
| « Last Edit: Jul 29th, 2010, 6:51pm by Fritzlein » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: ratings for 2010 CC bots
« Reply #9 on: Jul 29th, 2010, 5:42pm » |
Quote Modify
|
on Jul 29th, 2010, 12:49pm, rbarreira wrote:| IOf course the performance will still sometimes decrease even with just one core, but it's not that common for people to be playing four CC/Fast/Blitz bots simultaneously. |
|
Plus it is only the bot's move half of the time, so one should get approximately eight bots before the load starts to affect performance.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: ratings for 2010 CC bots
« Reply #10 on: Jul 30th, 2010, 7:59am » |
Quote Modify
|
Thanks for the input guys. I've reset the ratings for the 2010 bots. I basically used Karl's numbers except for P2 I used -150. Also I rounded the developer bot rating to the nearest 25; OpFor 1873 became 1875; Sharp 1958 became 1950, etc.
The bots can be started from this page:
http://arimaa.com/arimaa/bots/index.cgi
|
|
IP Logged |
|
|
|
Tuks
Forum Guru
Arimaa player #2626
Gender:
Posts: 203
|
|
Re: ratings for 2010 CC bots
« Reply #11 on: Jul 30th, 2010, 11:55am » |
Quote Modify
|
could you add them to the bot ladder?
not all of them or even most of them, just a couple of the highest level ones like marwin and clueless, i understand that the bot ladder is to get people interested but its also like a training and 3 years ago a player at bombs level was top notch, now its the norm for someone to eventually get to bombs level,
i liked the fact that the completion of the bot ladder was an accomplishment
|
|
IP Logged |
|
|
|
lostcalpolydude
Forum Junior Member
Arimaa player #4027
Gender:
Posts: 7
|
|
Re: ratings for 2010 CC bots
« Reply #12 on: Jul 30th, 2010, 4:17pm » |
Quote Modify
|
on Jul 30th, 2010, 11:55am, Tuks wrote:
i liked the fact that the completion of the bot ladder was an accomplishment |
|
It still is for the people that join the community (it was for me when I completed it a while ago, even getting to the point where I could handle fast and blitz games without panicking felt like an accomplishment). It's not like people just joining the community are going to be better in the same way that bots and humans that have been around for a while are improving.
Maybe there could be a second bot ladder if needed, but at that point it starts to seem more complicated.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: ratings for 2010 CC bots
« Reply #13 on: Jul 30th, 2010, 9:40pm » |
Quote Modify
|
I don't think the bot ladder is invalidated if it doesn't have the strongest version of the latest bot. It still is a significant accomplishment and a major benchmark.
On the other hand, I think it makes sense to overhaul the ladder every few years. Why not make the best ladder with the available material? In particular, instead of having four versions of every bot (P1, P2, Fast, Blitz), let's have fewer versions of each bot and more different developers/years. It's good for newcomers to be exposed to as great a variety of styles as possible.
For example, instead of SharpP1, SharpP2, SharpFast, and SharpBlitz, we could have PragmatictheoryP1, SharpP2, PragmatictheroyFast, and SharpBlitz. In addition to a wide variety of bot styles, it makes sense to have a wide range of ratings, from ArimaaScoreP1 to Marwin2010Blitz.
I think the ladder is OK as is, but would be improved by an overhaul.
|
| « Last Edit: Jul 30th, 2010, 9:40pm by Fritzlein » |
IP Logged |
|
|
|
Eltripas
Forum Guru
Meh-he-kah-naw
Gender:
Posts: 225
|
|
Re: ratings for 2010 CC bots
« Reply #14 on: Jul 30th, 2010, 9:55pm » |
Quote Modify
|
on Jul 30th, 2010, 11:55am, Tuks wrote:could you add them to the bot ladder?
not all of them or even most of them, just a couple of the highest level ones like marwin and clueless, i understand that the bot ladder is to get people interested but its also like a training and 3 years ago a player at bombs level was top notch, now its the norm for someone to eventually get to bombs level,
i liked the fact that the completion of the bot ladder was an accomplishment |
|
I agree with Tuks, I want some new bots on the ladder.
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print