Author |
Topic: Bot rating inflation (Read 1031 times) |
|
Fritzlein
Forum Guru
Arimaa player #706
Gender: 
Posts: 5928
|
 |
Bot rating inflation
« on: Sep 8th, 2007, 12:23pm » |
 Quote  Modify
|
on Sep 7th, 2007, 10:08pm, IdahoEv wrote:| (Because constant-performance bots are seeing increasing ratings with time). But I'll save those concepts for a separate post, because they're a bit off-topic here. |
|
Are you sure that constant-performance bots are seeing rating inflation? I graphed the rating of Arimaazilla, the bot with the most rated games, and I don't see an inflationary pattern.
Admittedly, the bottom bots in the ladder, particularly ArimaaScoreP1 and ArimaaScoreP2, are significantly overrated because they play primarily over-rated newcomers. Still, that doesn't mean that those bots are drifting upwards from where they are now; perhaps we are at some sort of equilibrium. Furthermore it isn't clear to me that the community as a whole is experiencing rating inflation.
One pattern I have noticed (and it might be just my imagination) is that when there is a surge of newcomers, the ratings of the bots seem to undergo a general upswing, whereas when there is a dearth of newcomers the players of moderate experience tend to consolidate their skills and drive down the average bot rating. Since we have been experiencing a surge of new players for the past five or six weeks, it isn't surprising if some bot ratings have spiked, but I would need to see more evidence before concluding that it is a long-term trend.
|
| « Last Edit: Sep 11th, 2007, 11:18am by Fritzlein » |
 IP Logged |
|
|
|
mistre
Forum Guru
Gender:
Posts: 553
|
|
Re: Bot rating inflation
« Reply #1 on: Sep 10th, 2007, 1:16pm » |
Quote Modify
|
It's interesting to see that Arimaazilla's rating was actually in the 1600's at the beginning of Arimaa and it looks like it peaked at about 1800. I always wondered why really strong players have several losses to this bot. I am assuming it was because it was one of the first bots anyone ever played against and at that time everyone was still learning basic strategy, and trying new ideas.
I would be interested in seeing a similar graph for Bomb2005p1.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Bot rating inflation
« Reply #2 on: Sep 11th, 2007, 11:40am » |
Quote Modify
|
on Sep 10th, 2007, 1:16pm, mistre wrote:| I would be interested in seeing a similar graph for Bomb2005p1. |
|
Your wish is my command. Let me know if you can't see the image; I'm experimenting with using Flickr to host my images for free.
We can discern two features on this graph: the date that the bot ladder was first created, and the date that botBombP1 was left alone as the only strong bot on Level 2 of the ladder.
|
|
IP Logged |
|
|
|
mistre
Forum Guru
Gender:
Posts: 553
|
|
Re: Bot rating inflation
« Reply #3 on: Sep 11th, 2007, 10:24pm » |
Quote Modify
|
Thanks for showing that. Bomb2005p1 is very tough on beginners and I had a rough go at it in the beginning starting just 6-13. I was actually doing much better versus Clueless2006p1 which is arguably the better bot.
But then something just clicked for me and now I have beaten this bot 30 straight times. Bot bashing? Not like syed. But someone has got to keep this bot's rating reasonable. No way it should be rated higher than Bomb2005p2, which it was recently.
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print