Author |
Topic: Ratings distribution (Read 1339 times) |
|
IdahoEv
Forum Guru
Arimaa player #1753
Gender: 
Posts: 405
|
 |
Ratings distribution
« on: Apr 19th, 2006, 1:05am » |
 Quote  Modify
|
I thought it would be interesting to see a little more detail about the ratings of bots and humans vs. time; other graphs have shown only the average and/or the extrema.
So I generated a plot that attempts to show something like ratings density over time. For each player or bot who played during a certain month, a point is generated denoting the average rating they had during that month.**
http://idahoev.com/arimaa/ratings_distribution.png
It's interesting to me that while the gap is opening, the density of bots inhabiting the upper-middle range (1600-1800) has been dramatically increasing the last few months. Now instead of one bot inhabiting a lonely space far above all the other bots, there is a whole population of bots thickly inhabiting the full range up to Bomb's best.
** Actually, there are two points for each player: one for their average rating when starting a game as a white player, one for black. Because the SQL for that was significantly easier to write. 
|
|
 IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Ratings distribution
« Reply #1 on: Apr 19th, 2006, 1:25am » |
Quote Modify
|
Cool graph, thanks. It is interesting to watch the extreme ratings spread out, and also to watch the ratings thicken in the neighborhood of 1500. The middle of the graph is fast approaching a solid green line. Also the BvB games are apparently having the anticipated effect of pushing the bot ratings outward at the extremes, widening the spread.
The bottom blue dot for most of the graph is ShallowBlue playing unrated games. The series of dots jumps once because ShallowBlue played a couple of rated games, scoring upset victories over Arimaalon. (Incidentally, it was probably taking the edge off of newcomer-driven inflation to have ShallowBlue play unrated, although I didn't realize this until after ShallowBlue started to play rated games.)
I made a SQL union query in which each game occurs twice, once with the players in each order. Also I reverse the rating, ratingk, and type fields. I've often found this "games doubled" query useful as a basis for generating graphs and stats like this one.
Thanks again for drawing us a pretty and informative picture.
|
| « Last Edit: Apr 19th, 2006, 1:29am by Fritzlein » |
IP Logged |
|
|
|
frostlad
Forum Senior Member
Arimaa player #1704
Gender:
Posts: 46
|
|
Re: Ratings distribution
« Reply #2 on: Apr 19th, 2006, 12:48pm » |
Quote Modify
|
It will be interesint to see if the ratings continue to push out more as time goes along.
What I mean by that is if we start to see more top humans break 2000 or be around that and have the absolute top tier of humans start to push up towards 2500. I forget who was talking about the different levels in chess but we could start to carve out a new ranking soon with that.
|
|
IP Logged |
|
|
|
IdahoEv
Forum Guru
Arimaa player #1753
Gender:
Posts: 405
|
|
Re: Ratings distribution
« Reply #3 on: Sep 4th, 2007, 7:35pm » |
Quote Modify
|
on Apr 19th, 2006, 12:48pm, frostlad wrote:It will be interesint to see if the ratings continue to push out more as time goes along.
What I mean by that is if we start to see more top humans break 2000 or be around that and have the absolute top tier of humans start to push up towards 2500. I forget who was talking about the different levels in chess but we could start to carve out a new ranking soon with that. |
|
Well, I just re-generated this graph a year and a half later, and frostlad's prediction came almost precisely true. The gap between the top bots and humans has stayed pretty steady at around 400 points but the rating of the best of both has increased steadily over the last two years. Humans are going to break 2500 quite soon.
Click for bigger.
Given that the bots themselves haven't improved, I wonder how much of this is simple ratings inflation from new members who lose the main points of their initial 1500 and then disappear. (as has been discussed elsewhere).
|
|
IP Logged |
|
|
|
mistre
Forum Guru
Gender:
Posts: 553
|
|
Re: Ratings distribution
« Reply #4 on: Sep 5th, 2007, 8:52am » |
Quote Modify
|
Thanks for the graph. It would be interesting to see the individual bot ratings over time and their all-time high and low ratings. Are bots closer to their lowest ratings or their highest ratings?
Bomb2005p1 is a very interesting case and probably has the highest standard deviation of any bot. It actually dropped to under 1500 a few months ago and then Omar moved the Cluelessp1 bots up to level 3, leaving Bomb as the only stronger bot on level 2 and its rating has since shot up to over 1800.
|
| « Last Edit: Sep 5th, 2007, 8:53am by mistre » |
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print