Arimaa Forum - Print Page


    
      
        Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Site Discussion >> Rating of the championship bots.
        
(Message started by: mouse on Feb 23^rd, 2005, 4:56am)

Title: Rating of the championship bots.
Post by mouse on Feb 23^rd, 2005, 4:56am

The 2005 championship bots is all started with a rating of 1500 and a RU of 120. This seems a little strange ??? given the championship bots have got established ratings and both are substantially higher than 1500. So why are the last rating from the last game of the championship and challenge game not transferred to the "new" bots? :P

I can see the point with the slower timecontroll where Clueless have not played lately but Speedy have played with a rating several 100's points higher than the 1500 mark of BOMB2005fast.

It may be too late to change it for the bots that has already played rated games. But it could be changed for the once which has not played rated games and for next years championship bots.

Title: Re: Rating of the championship bots.
Post by Fritzlein on Feb 23^rd, 2005, 7:36am

Omar, thanks for putting up the championship bots, and especially thanks for the fast versions!

I agree with mouse about the ratings, though. Even if the championship bots start off with an RU of 120, they should start with the ratings established by the respective bots. And in the case of Bomb2005Fast, it should start with Speedy's old rating. For the other bots maybe just add 200 points.

If these estimates are inaccurate, a high RU gives it a chance to sort itself out. But at least the first people to play shouldn't have their ratings clobbered when we know that the bots are much stronger than 1500.

Title: Re: Rating of the championship bots.
Post by mouse on Feb 23^rd, 2005, 8:01am

Hi Omar

I forgot to mention that I also very much appreciate the championship bot being online - especially the different timecontrols. I'm looking forward to playing them in the time to come.

Title: Re: Rating of the championship bots.
Post by Adanac on Feb 23^rd, 2005, 11:22am

I'm definitely going to play a few games against these championship bots. Unfortunately, there was a crash during my first attempt to play Clueless. By the time I was able to log back in the game was abandoned. Now, I'm unable to access my Game Record page. Does that normally happen after an abandoned game?

Title: Re: Rating of the championship bots.
Post by Fritzlein on Feb 24^th, 2005, 9:25am

It occurs to me that the championship bots might not be quite as strong when hosted by Omar as the originals when hosted by their developers. Maybe speedy and clueless were running on faster machines than Bomb2005Fast and Clueless2005CC. Even so I think they should start at the ratings of the respective bots rather than at 1500.

Title: Re: Rating of the championship bots.
Post by Adanac on Feb 24^th, 2005, 1:17pm

I also believe it would be more logical if bots had their rating set to their established level rather than 1500. Similarly, new human players should probably start at 1200 or perhaps 1400 rather than 1500. It seems that most new players go through a ratings nosedive during their first few rated games before eventually returning to the 1500 level after playing a few dozen games. In both cases, it would create a bit more fairness in the ratings structure if initial ratings were closer to real playing strength.

Title: Re: Rating of the championship bots.
Post by Fritzlein on Feb 24^th, 2005, 1:30pm

on 02/24/05 at 13:17:52, Adanac wrote:

Similarly, new human players should probably start at 1200 or perhaps 1400 rather than 1500.

Ah, but starting new players at 1200, if no other changes were made to the rating system, would have a tendency to deflate the ratings overall. The average rating would be pulled down. Specifically the beginning bots Arimaalon, Arimaazilla, and Arimaazon would probably lose about 300 points each, and then we would be back in the same boat as we are now, except that new players coming in at 1200 would be taking a nosedive in their first few games.

There has been a good deal of discussion about this in other threads, specifically "Arimaa ratings deflation" and "Ratings distortion due to selection of opponents". I will be curious to hear what you have to say if you read those threads and find anything interesting to comment on.

Title: Re: Rating of the championship bots.
Post by Adanac on Feb 24^th, 2005, 5:50pm

I hadn't considered the continuous deflationary spiral that could occur if new players' ratings are reduced. I suppose that without a ratings anchor, an idea that I noticed in the previously mentioned threads, it's difficult to control ratings deflation in a new game with a small group of players showing continuous improvement. Getting back to this thread topic, giving new CC bots their established ratings, rather than 1500, could partially offset the deflation :)

Title: Re: Rating of the championship bots.
Post by omar on Mar 3^rd, 2005, 2:47pm

Yeah I should have set the inital ratings to be closer to their established versions. Totally slipped my mind; sorry.

Im resetting the ratings as follows:

Bomb2005CC - 1800
Bomb2005Fast - 1900

Clueless2005CC - 1680
Clueless2005Fast - 1750

GnoBot2005CC - 1500
GnoBot2005Fast - 1550

Loc2005CC - 1380
Loc2005Fast - 1400

The CC ratings are slightly less than the rating of the established version and the Fast rating is slightly more than that.

I am also going to set the RU of these bots down to 50 so that its more stable.