Arimaa Forum - Print Page


    
      
        Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Site Discussion >> Making games unrated
        
(Message started by: Tachyon on Nov 3^rd, 2008, 11:06pm)

Title: Making games unrated
Post by Tachyon on Nov 3^rd, 2008, 11:06pm

I recently played a couple of games in short succession while experiencing client connection problems causing me to lose games on time. The system did not want to allow making the games unrated and give a message along the lines of .... "this game cannot be made unrated because you were not in a good position ( position score -2.03 )"

What does this mean ?

In the one game (#87446) I had a clear winning position

In the other game (#87447) I had a strong position that I expected to turn into a winning position

Title: Re: Making games unrated
Post by Fritzlein on Nov 4^th, 2008, 4:49am

I can understand your frustration in not being to unrate those two games, because it looks like you were on track to win both.

Some time ago, before you joined the server, there was substantial abuse of the unrate feature. It became clear that people were using it when they weren't actually disconnecting. For any individual game, one can never be sure whether a disconnection really occurred right when the human player got into trouble, but from observing a series of games it is easy to tell when someone is systematically timing out on purpose to use the unrate feature and avoid losing.

Full disclosure: in an earlier age, I had lost a camel for a cat in game 20776 (http://arimaa.com/arimaa/gameroom/comments.cgi?gid=20776), and unrated the game after I was disconnected, even though I was obviously losing on the board. I felt that I had winning chances that I had been deprived of executing, and that "no score" was a fair result. Since I wanted to unrate even from a losing position, I can obviously sympathize with your wanting to unrate from a winning position.

However, the rating abuse that went on via the unrate feature later convinced me that it was better to have the unfairness of losses due to bad Internet connection rather than the unfairness of inflated ratings due to intentional abuse of the unrate feature. I advocated for removing the unrate feature entirely, because there was no way to ever police it properly. It is impossible to tell involuntary disconnections from intentional disconnections.

Omar, however, suggested leaving the feature active in case the human was not losing. That was a good idea, except that it would be a hassle to judge whether the human was losing or not. The feature wouldn't work if a human judge had to be called every time. So Omar settled on the best automatic judge available, namely Bomb. Yes, Bomb will be wrong some of the time, but Bomb is as fair as we can get and still be automatic. Bomb is not perfect, but its judgment is generally pretty darn good.

An unforeseen consequence of having Bomb be the judge, however, is that Bomb is particularly bad at judging its own positions. In the two games you mention, material is equal and Bomb is in a strategic position that it was trying to get. Bomb didn't take your camel hostage in an attempt to lose, it took your camel hostage because it thought that was a good thing. It thinks it is beating you, which is exactly why the procedure you are executing works to beat Bomb in game after game after game. If Bomb didn't think it was beating you, it would not go for this losing line every time.

In other words, we are consulting Bomb for its judgment in precisely the type of position it consistently misjudges. I'm not sure what a good solution would be. Maybe have two automatic judges? Perhaps one could have Bomb be the judge in non-Bomb games, and OpFor be the judge of Bomb games?

Anyway, that's the history behind why you weren't allowed to unrate games that you were winning.

Title: Re: Making games unrated
Post by 99of9 on Nov 4^th, 2008, 5:04am

Good analysis Fritz. Extra judge bots is a good idea, but I think it would just be simpler to make the rule "if all judge bots agree that the human is losing by over X, then unrating is not allowed". That way, if the position is in any way unclear, the human is given the benefit of the doubt.

Of course this will not prevent all such problems... materialistic Gnobot agrees with bomb that Tachyon is behind in both games ;), in one case by more than one R, but less than 2R.

Title: Re: Making games unrated
Post by Tachyon on Nov 4^th, 2008, 2:59pm

Thanks for the explanation Fritz ... I am not terribly upset about not being able to unrate these particular games. Was just curious as to what the reasoning was.

However it does provide another reason why I believe that a review of the way games are rated / unrated on the Arimaa games platform is overdue.

I have made a long career of developing technology solutions and in general I believe that first prize is to fix the origin of the problem, not the symptom. In this case I am sure that it should be possible for the game server to tell whether the connection has broken down at some point during the game and flag the game accordingly. Then rather use this flag as the deciding factor as to whether the game can be made unrated or not.

Using the position on the board and bots for judges is unreliable at best e.g. I have played games where I suffered from a broken connection, managed to re-connect and had to make hasty moves to avoid losing on time and eventually loses despite finishing the game. Such a game should also be able to be made unrated and the board position will be of little help to the bots.

Title: Re: Making games unrated
Post by Tachyon on Nov 4^th, 2008, 3:11pm

Another possibility might be to record the time situation with every move and treat a disconnection as a game adjournment. The player can then continue the game from the same position and time settings where is was left of whenever he reconnects.

Title: Re: Making games unrated
Post by 99of9 on Nov 4^th, 2008, 4:14pm

on 11/04/08 at 14:59:10, Tachyon wrote:

In this case I am sure that it should be possible for the game server to tell whether the connection has broken down at some point during the game and flag the game accordingly. Then rather use this flag as the deciding factor as to whether the game can be made unrated or not.

I am sure that I would like your proposal less than the current method. It would be very easy for a ratings-cheat to pull the network cable out when they were about to lose, and cause a "broken connection" flag. In this kind of situation unrating should not be allowed.

Your adjournment suggestion is better, but that could still be exploited if someone was running short on time and wanted some thinking time. Technologically it seems like it would require quite a bit of extra work, because adjourning games is not yet a feature of the arimaa gameroom.

Title: Re: Making games unrated
Post by Tachyon on Nov 4^th, 2008, 10:35pm

I see your point 99of9. I suppose one has to bear in mind what the purpose of the rating is.

As long as it is intended to be a relative player strength indicator in a casual playing enivironment I suppose it does not warrant the additional effort to make it ultra robust.

If however Arimaa continues to grow in popularity and ratings starts to have a more formal use such as in official chess ratings these issues will have to be adressed one way or another.

Title: Re: Making games unrated
Post by Hirocon on Dec 3^rd, 2010, 7:44am

Sorry to necropost, but I got disconnected in my most recent game and I can't figure out how to unrate the game. Where is that option in the game room interface?

EDIT: Never mind, I found it. It's on the left-hand menu: Play Now-->Unrate Game

Title: Re: Making games unrated
Post by qswanger on Dec 3^rd, 2010, 3:11pm

on 11/04/08 at 15:11:25, Tachyon wrote:

You might want to check with Dave Dyer about this to be absolutely sure, but I believe that this is how his Java based Boardspace.net site handles such situations. Even though game time is tracked at Boardspace, I don't believe there is such a things as actually losing on time. However, I do know that when you play bots and there is a disconnection for ANY reason that when you attempt to play the same bot again you cannot start a new one and the game that got disconnected is loaded and re-continued instead. This sounds like a good solution to me.