Author |
Topic: Widening gap? (Read 4994 times) |
|
omar
Forum Guru
Arimaa player #2
Gender: 
Posts: 1003
|
 |
Re: Widening gap?
« Reply #15 on: Oct 12th, 2005, 3:46pm » |
 Quote  Modify
|
Looks like the bots have stabalized around 1900's. It will be interesting to see this graph next year after the 2006 bots have been added.
Karl, perhaps you could create a "Arimaa Stats" page in the Wiki and have pages off of that page for the various stats you've looked at and post the graphs there. There's been times when I wanted to look at one of the graphs and hunting for it in the forum was not easy. Posting these in the Wiki would make it very organized and easy to find.
|
|
 IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Widening gap?
« Reply #16 on: Oct 12th, 2005, 7:05pm » |
Quote Modify
|
on Oct 12th, 2005, 3:46pm, omar wrote:| Looks like the bots have stabalized around 1900's. |
|
I doubt the bots will ever "stabilize" with our current rating system. BombFast and BombBlitz were both under 1800, and now they're both almost to 2000, but I'm sure that isn't the end of them alternately nosediving and rockecting upward.
Quote:Karl, perhaps you could create a "Arimaa Stats" page in the Wiki and have pages off of that page for the various stats you've looked at and post the graphs there. There's been times when I wanted to look at one of the graphs and hunting for it in the forum was not easy. Posting these in the Wiki would make it very organized and easy to find.
|
|
Thanks for that idea. That would be good especially if I can put the images directly onto your Wiki server, because I have no idea how long the University of Minnesota will let me have a Web page now that I'm a dropout. I'll go see what I can do.
[EDIT] Hey, it's really easy to upload images directly. I'll add more later, but I might as well update the ones I want to update before uploading them. Do you have any specific requests for graphs that I should generate?
|
| « Last Edit: Oct 12th, 2005, 8:01pm by Fritzlein » |
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Widening gap?
« Reply #17 on: Oct 15th, 2005, 9:47am » |
Quote Modify
|
The stats page in the wiki looks great. Thanks for creating that. Makes it very nice and easy to find the graphs now. Couple suggestions:
Maybe "discussion" links from the graph page back to the forum topic page(s) which discusses that partiular stats would be nice. Allows the discussion about the stats to be found easily.
Perhaps having seperate pages for each of the stat topics (Ratings, Server Activity, etc) would allow a lot more to be said with each topic and avoid one very long page as the number of topics grows. The main "Statistical Graphs" page would just link to these pages.
|
| « Last Edit: Oct 15th, 2005, 10:23am by omar » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Widening gap?
« Reply #18 on: Dec 10th, 2005, 3:34pm » |
Quote Modify
|
Just to update, there are now four humans rated over 2100 at the same time, for the first time ever. This is at the same time that the highest rating of an active bot is 1816. If we define "active" as having RU < 50, then there are still eleven active humans rated higher than the top active bot.
|
|
IP Logged |
|
|
|
Ryan_Cable
Forum Guru
Arimaa player #951
Gender:
Posts: 138
|
|
Re: Widening gap?
« Reply #19 on: Dec 11th, 2005, 5:51am » |
Quote Modify
|
You inspired me to make it 5 above 2100. 
I think all of the top 11 active players could beat any bot >90% of the time if they used all of the information available to humanity. Even as ridiculously overrated as I am, I think I am underrated relative to many of the bots. I don’t think there is any bot that deserves to be rated above 1700 right now. Bashing all of the bots to 1700- risks a repetitive stress injury to my brain though.  Maybe rick or some other low rated player will help finish the job.
|
|
IP Logged |
|
|
|
PMertens
Forum Guru
Arimaa player #692
Gender:
Posts: 437
|
|
Re: Widening gap?
« Reply #20 on: Dec 11th, 2005, 11:31am » |
Quote Modify
|
Quote:| I think all of the top 11 active players could beat any bot >90% of the time if they used all of the information available to humanity. |
|
Put in 100% if you are talking about all information ... but I guess 100% of the top 100 players would not like to simply replay an already played game.
As long as bot's do not play either "randomly" or change (or start to play much much better than they do now) it will always be a question of time to find just the right moves.
(Some of the rather extravagant botbashings are proof to that)
I am sure that botbashing did help us to understand the bots far better than we did just one year ago.
While it was possible to outsmart speedy we now are at a level where we actually know exactly what (for example) bomb will move even before we did press send.
I think that can be called a really wide gap
|
| « Last Edit: Dec 11th, 2005, 11:34am by PMertens » |
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender:
Posts: 1413
|
|
Re: Widening gap?
« Reply #21 on: Dec 11th, 2005, 5:18pm » |
Quote Modify
|
on Dec 11th, 2005, 11:31am, PMertens wrote:| As long as bot's do not play either "randomly" or change (or start to play much much better than they do now) it will always be a question of time to find just the right moves. |
|
Oh, but they do!
(Nevertheless, probably not enough to alter the truth of your other statements.)
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Widening gap?
« Reply #22 on: Dec 11th, 2005, 8:04pm » |
Quote Modify
|
It's true that we can beat most bots essentially formulaically, which could be exploited to make the ratings gap arbitrarily large. Once bots become more adaptable and/or better randomized, the ratings will be a better indication of the true gap in skill. Right now we can mostly just guess that the gap is large and growing, without being able to quantify it very accurately.
|
|
IP Logged |
|
|
|
Ryan_Cable
Forum Guru
Arimaa player #951
Gender:
Posts: 138
|
|
Re: Widening gap?
« Reply #23 on: Dec 31st, 2005, 9:56pm » |
Quote Modify
|
Fritzlein from http://arimaa.com/arimaa/gameroom/comments.cgi?gid=22958
Quote:I believe there is some rating deflation, maybe 100 points from a year ago and 200 points from two years ago. Even so, I'm not sure Speedy's true strength today is only 1750. I expect that a human who joined today and played only against other humans until attaining a rating of 1750 would lose a large majority to BombBlitz2005.
We have a strange situation (as you pointed out elsewhere) that there aren't many currently active players in the 1750 to 1950 rating range. If there were more such players, I believe the ratings scale would spread out more than it has at present, that top players would all gain 100 points from where they are now, and that BombBlitz's rating would bounce back unless it were determinedly bashed by known anti-bot methods.
On the other hand, I could be wrong about how much deflation is going on. Certainly we are learning tons about the game each year. I guess rating inflation/deflation is ultimately a matter of how fast we discover new ideas versus how fast new players join the pool.
|
|
I think that the ratings should be spread out more. I don't know how it is in chess, but with the number of new players Arimaa has coming in at 1500, it is almost inevitable that anyone that has been active for 6 months has a rating >1500. We seem to have three main groups of humans: noobs many of whom won't stay; those who can reliably beat noobs and most of the ladder bots but not the top bots/fast bots; and those who can reliably beat the top bots and people in the second group. If the second group sits ~1600, Bomb/Clueless Fast/Blitz then would go ~1800. Probably there is room for more spread in the ~1600 group, but usually people break out of this group at about the same time they get good enough to beat all the bots. Within the third group, I think there is enough skill space for at least three classes of humans at ~2000, ~2200, and ~2400.
I would put myself in the ~2000 group, I have a 20 game performance rating against humans of ~1825, and I think I can beat all of the Fast bots 80%+ using only E+H. Yet I have only won 1 game against a human rated >1750. Personally, I feel like I have advanced at least 3 classes from when I was ~1400, struggling to beat 'nator.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Widening gap?
« Reply #24 on: Jan 1st, 2006, 10:47am » |
Quote Modify
|
I expected that new players would start learning the game by reading and studying expert games, rather than by experimenting individually. If it had happened the way I imagined, there would have been much more serious rating deflation by now, with the lobby bots being driven well below their historical levels.
But instead almost everyone seems to prefer learing by doing, and the ratings of the lobby bots haven't changed at all (as far as I can tell). This provides a higher floor to keep the upper levels of ratings from deflating as much as they otherwise would. For every new player that works up the ladder, stealing points from everyone else on the way, there are half a dozen players that lose a few games and then never come back, injecting points into the system at the bottom.
Inflation comes in at the bottom of the system, but deflation usually comes closer to the top. It comes from players learning more about the game, and taking points away from players who have only stayed the same in skill. My hunch is that because inflation and deflation comes from different ends of the spectrum, the ratings scale has compressed somewhat.
So to tie this post back into the topic thread, the gap between humans in general and bots is general is probably a little bit wider than the ratings indicate, although specific bots (in particular BombBlitz2005) are way underrated IMHO.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Widening gap?
« Reply #25 on: Jan 20th, 2006, 11:14am » |
Quote Modify
|
Welcome Blue22 above the 2000 rating watermark. That makes eight humans (albeit just seven if Arimanator is considered inactive), while bots languish around 1800 or below. Unless the Bomb2006 was pumped up to master strength in secret, humanity has pulled away by roughly 200 points of playing strength between challenge matches.
|
|
IP Logged |
|
|
|
Ryan_Cable
Forum Guru
Arimaa player #951
Gender:
Posts: 138
|
|
Re: Widening gap?
« Reply #26 on: Jan 29th, 2006, 6:34pm » |
Quote Modify
|
There are now no bots rated >=1800 available to play. There are currently 9 humans with RU<=50 rated above the top available bot. There are only 6 available bots rated >=1700, all of which are versions of Bomb or Clueless. With enough effort it is definitely possible to knock all of these bots down to below 1700.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Widening gap?
« Reply #27 on: Jan 29th, 2006, 7:54pm » |
Quote Modify
|
It is interesting to me that ratings of the lobby bots haven't budged much over the same time period the CC bots have been beaten down. Perhaps new players will always perform about as well as they do now initially, no matter how much the state of the art advances, no matter how much we write about the game, or post puzzles in the the Wiki, or comment newbie games, or otherwise make help available. Maybe Arimaazilla will permanently bounce between 1400 and 1550, without ever being kept down by well-armed newbies.
The gap in strength between the lobby bots and the 2005CC bots is presumably as large as it always was, thus the CC bots now by and large have deflated ratings relative to anyone coming to play them after mastering the lobby. Yet those same CC bots will regularly bleed points to any of the top ten humans, so one could argue that the CC bots are overrated.
Before I gave this phenomenon as evidence that the scale of ratings is too compressed. Upon further reflection, however, I am not sure that a ladder of bots makes a good scale for ratings. The problem is that the HvB learning curve is much steeper than the HvH learning curve. If there is a bot you can't beat, you will hardly ever be able to beat it, and if there is a bot that you do know how to beat, you will beat it almost every game. The transition zone is very narrow. This is very different from, say, Robinson passing me in skill. There may be a longer time we are about 50-50, and even as he pulls away I'll probably be dangerous to him for a long time, and win a significant percentage as an underdog.
So maybe in purely human terms the current rating scale isn't so bad, and it just looks compressed because of all the HvB games.
|
|
IP Logged |
|
|
|
Ryan_Cable
Forum Guru
Arimaa player #951
Gender:
Posts: 138
|
|
Re: Widening gap?
« Reply #28 on: Mar 27th, 2006, 6:53pm » |
Quote Modify
|
There are currently 6 players 2100+, with another 2 or 3 active players 2000+. There are 11 humans with RU<=50 rated above the top available bot, and there are 3+ other active humans rated above the top active bot.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Widening gap?
« Reply #29 on: Mar 28th, 2006, 11:50am » |
Quote Modify
|
That reminds me that Bomb2006, presumably the strongest bot at present, isn't available for play. Of course, Bomb2006 isn't a whole lot better than Bomb2005, but there were a few tweaks and bugfixes.
Unless some bot makes some huge strides this year, we'll probably be able to have the next Challenge defended by the 20th-ranked active human.
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print