Arimaa Forum - Print Page


    
      
        Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Off Topic Discussion >> Working on research related to rating systems 
        
(Message started by: ginrunner on Aug 12^th, 2012, 5:03pm)

Title: Working on research related to rating systems
Post by ginrunner on Aug 12^th, 2012, 5:03pm

So let me start off by saying I work in the stock market and have been working on a theory that I got my inspiration from playing rated games like this. Also let me note that I have never played a game such as chess professionally and so my understanding of Elo ratings and such is limited. The general premise goes that here everyone starts off at 1500 (or some other static point in a game) and as expected when a person is new they tend to lose those points. Now many people quit without getting back to 1500 meaning that those points stay in the system and cause inflated ratings. Excluding advancements in general knowledge of strategy a 1900 rated player 3 years ago would be better than a 1900 rated person today. Is there any research that supports this or alternatively is there a system in place that prevents this? An example is Magnus Carlson's rating is currently 2837, would that have even been possible in 1980 when hypothetically there would be less points within the system?

Where I am eventually going with this is hypothetically the market will have a general average P/E ratio. This should be a zero sum number but the market P/E fluctuates and so I am betting that the increase or decrease is due to people that don't typically work in the market entering or exiting.

Title: Re: Working on research related to rating systems
Post by Fritzlein on Aug 13^th, 2012, 12:05am

on 08/12/12 at 17:03:12, ginrunner wrote:

The general premise goes that here everyone starts off at 1500 (or some other static point in a game) and as expected when a person is new they tend to lose those points. Now many people quit without getting back to 1500 meaning that those points stay in the system and cause inflated ratings. Excluding advancements in general knowledge of strategy a 1900 rated player 3 years ago would be better than a 1900 rated person today. Is there any research that supports this or alternatively is there a system in place that prevents this? An example is Magnus Carlson's rating is currently 2837, would that have even been possible in 1980 when hypothetically there would be less points within the system?

There are several ways for rating points to enter and/or leave the system. The biggest one, as you point out, is people entering the system at one rating and leaving with a different one. If they leave rated lower, it causes inflation, but if they leave rated higher, it causes deflation. A much smaller source of fluctuations is that games not zero-sum. One player may gain more points than the other loses, or vice-versa. On arimaa.com this is due to differing RU values; for FIDE ratings it is called differing K values.

There is another source of deflation/inflation in addition to the number of points in the system changing, and that is the players themselves getting better/worse at the game. Imagine a closed system where nobody enters or leaves, everyone stays active, and every game is zero-sum so that rating points can't be created or destroyed. What if everyone practiced so much that they gained 100 points in playing strength? Nothing. Nobody's rating would change, but that would effectively be deflation, because a rating of 1800 after would be as good a player as a rating of 1900 was before.

Both the USCF and FIDE observed that their ratings under a naive implementation were deflationary. Therefore both organizations intentionally introduced additional inflationary measures. They were aiming for stability, but how well they succeeded is up for debate.

In FIDE if you take the average rating of the top 100 players, it has steadily increased a few points per year since the inception of the ratings. Critics call this inflation. However, it arguably reflects the fact that players are getting better and better thanks to increased knowledge and training. We can't really test whether there is inflation unless we have a time machine to send today's top players back forty years for some test matches against the top players when FIDE ratings started. I personally believe that Magnus Carlsen would kick butt and take names in that situation, i.e. he would win enough games to get a 2837 rating against past grandmasters. "Fewer points in the system" notwithstanding, he would be better than them and thus be rated significantly higher than them. Others think differently, and suppose that top players of the past could keep up with top players of today.

Absent a time machine, the closest shot to objectivity would be to use a strong computer to critique the recorded games of past grandmasters. The only such study I know of concluded that grandmasters of one hundred years ago (long before FIDE ratings) were substantially weaker than modern grandmasters.

Arimaa.com gameroom ratings are roughly stable. Because most of the play is bot vs. human, the local effects of rating inflation/deflation are even more exaggerated than for chess ratings. Nevertheless, a 1900 rating today is (on average) probably within a hundred points of what a 1900 rating was (on average) in 2005.

Title: Re: Working on research related to rating systems
Post by ginrunner on Aug 14^th, 2012, 9:51pm

Thank you very much.