Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Events >> 2014 World Championship Rules
        
(Message started by: Janzert on Sep 2nd, 2013, 5:30pm)
Title: 2014 World Championship Rules
Post by Janzert on Sep 2nd, 2013, 5:30pm
Well it's September and time to really get active on the next WC planning. Here is the page for the 2014 WC rules (http://arimaa.com/arimaa/mwiki/index.php/2014_World_Championship_Rules). I've listed the main changes I'm currently considering or have made in sections at the top of the page. Most of these are simply procedural changes or additions. The rest I consider fairly minor changes to the actual format. I believe the overall format from last year worked quite well and that it's important for Arimaa to bring year to year stability to the WC format.

I would love to hear any feedback on these changes or any additional changes that I missed. I do not expect all the changes currently under consideration to make it in, especially all of the ones requiring software change.

I'm expecting to have all substantial changes finalized by November 1st. With any software changes required for them to be in progress and hopefully mostly complete by that time as well. I do not want to make any changes after December 1st when registration opens.

Janzert
Title: Re: 2014 World Championship Rules
Post by aaaa on Sep 5th, 2013, 8:03am
I would like to get some feedback on this proposal (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi?action=display;board=events;num=1354379162;start=30#37).
Title: Re: 2014 World Championship Rules
Post by supersamu on Sep 5th, 2013, 10:38am
I like the idea. It took a while before I understood it, but in the end the process is pretty simple.

I don´t think I understand it. You want the players with less than 3 losses to be virtually paired against the dummy player and be given wins against this dummy player?
This only players in the Swiss Section (the players with 3 or more losses) get "real" byes.
This then influences the UTPR and the STPR of the human player and the dummy Player.

Were the rating(s) of the human Players influenced by byes before or why do you think this method is better?
I understand the process, but I don´t understand how this process lessens the influence of byes gained.

Or do byes affect the pairings so heavily that it is favorable to give players a higher performance rating and not giving them byes?
Title: Re: 2014 World Championship Rules
Post by Hippo on Sep 5th, 2013, 11:07am

on 09/05/13 at 08:03:49, aaaa wrote:
I would like to get some feedback on this proposal (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi?action=display;board=events;num=1354379162;start=30#37).


I have not studied the proposals deeply, but I would like to remember Boo's proposal (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi?action=display;board=events;num=1354379162;start=15#25).
Title: Re: 2014 World Championship Rules
Post by harvestsnow on Sep 5th, 2013, 12:05pm

Quote:
Seeding
Players will be seeded according to the WHRE ratings maintained by Hervé D'Hondt, as calculated 1/4/2013. Players with fewer than six WHRE games will be seeded according to their gameroom ratings, adjusted at the discretion of the Tournament Director to account for artificially inflated or deflated ratings, since gameroom ratings can be manipulated much more easily than WHRE ratings. Seeding has a large effect on early-round pairings, but rapidly decreases in importance; after about three rounds seeding becomes irrelevant as pairing is then done entirely on the basis of in-tournament results.


Which version of the WHRE will be used?
If it's the current version, there will be a seeding issue. woh tuned it so that ArimaascoreP1 would be rated 1000, anchoring it to the gameroom ratings. But since the Auto Postal games have been removed, ArimaascoreP1 isn't in the list anymore. A lot of low ranked players disappeared as well, resulting in a general deflation: WHRE ratings are now about 200 points lower than gameroom ratings. If we're going to use both, the WHRE needs to be rescaled.
Title: Re: 2014 World Championship Rules
Post by Janzert on Sep 5th, 2013, 10:40pm

on 09/05/13 at 08:03:49, aaaa wrote:
I would like to get some feedback on this proposal (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi?action=display;board=events;num=1354379162;start=30#37).


I would like to see some comparison between that and simply dropping UTPR from use, or doing both together. By default I would rather simplify the pairing system. Adding complexity should only be done if it can be shown to correspondingly improve the tournament. Where the amount of improvement is probably mostly a subjective judgement.

Hopefully I'll get time to dig into it in the next couple of months.

Janzert
Title: Re: 2014 World Championship Rules
Post by Janzert on Sep 5th, 2013, 10:55pm

on 09/05/13 at 11:07:46, Hippo wrote:
I have not studied the proposals deeply, but I would like to remember Boo's proposal (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi?action=display;board=events;num=1354379162;start=15#25).


Two things going against it unfortunately. First it's a fairly significant change and so I would want to see that it works significantly better than the current system before I would consider it. Second I still feel that it is rather ambiguously specified. While I could sit down and implement something that I think would fulfill the proposal, I'm not at all confident that it would be what was proposed. Actually I think I could probably sit and implement a half dozen different variations that would fit in the proposal.

I should probably also mention that if I don't get time to make any changes to the pairing system from last year, I won't be extremely disappointed. While there is certainly room for improvement I don't think it was terribly bad either. So I don't have much interest in tossing it completely or making major changes.

Janzert
Title: Re: 2014 World Championship Rules
Post by Janzert on Sep 5th, 2013, 11:26pm

on 09/05/13 at 12:05:16, harvestsnow wrote:
Which version of the WHRE will be used?
If it's the current version, there will be a seeding issue. woh tuned it so that ArimaascoreP1 would be rated 1000, anchoring it to the gameroom ratings. But since the Auto Postal games have been removed, ArimaascoreP1 isn't in the list anymore. A lot of low ranked players disappeared as well, resulting in a general deflation: WHRE ratings are now about 200 points lower than gameroom ratings. If we're going to use both, the WHRE needs to be rescaled.


WHRE is something that is bothering me for more reasons than just the skew from gameroom backup ratings. Of the top 20 players 11 have less than 50 games, and 5 less than 20. There are only ~200 players over ~2000 games ranked. It just doesn't seem to have enough players with enough games to give valid ratings for enough of the player base.

The solution of course is that we need more events. :) But as a short term solution I'm not sure what would be best. I'm currently leaning toward asking that auto-postal games be re-added. Even though the auto-postal system seems to be broken for the last few months and is open to manipulation regardless it may be the lesser of the evils available.

Janzert
Title: Re: 2014 World Championship Rules
Post by Janzert on Sep 23rd, 2013, 1:42pm
Given the state of WHRE (http://home.scarlet.be/~woh/whr/whre.htm) and gameroom rankings what do you think of this? Drop WHRE and gameroom ratings altogether. Use a weighted average between WHRH (http://home.scarlet.be/~woh/whr/whrh.htm) and a dummy 1400 rating. Specifically seed = 1400 + (WHRH * # of games ) / # of games + 1.

From what I see now pros are:
Uses a rating that has broad player and game input.
Does not need a second rating system to fall back on.
Smoothly handles the case of a player with very few games in the system.

Cons:
The rating is manipulable since a player can choose the opponents for rated games. I would probably leave a clause in to adjust manipulated ratings.
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Sep 25th, 2013, 11:28pm

on 09/23/13 at 13:42:07, Janzert wrote:
Specifically seed = 1400 + (WHRH * # of games ) / # of games + 1.

Did you leave off some parentheses there?

seed = (1400 + (WHRH * # of games )) / (# of games + 1)

I was applying the division before the first addition and getting something silly.  :P  Anyway, I would rather use the WHRE because it is less subject to manipulation.  It is problematic that many players will have no WHRE, given that substituting the gameroom rating makes it potentially more manipulable than even WHRH.

Here's an alternative proposal that is somewhere between "no WHRE = 1400", which leaves too many unseeded players, and "no WHRE = fairly scaled gameroom rating", which is too manipulable.

seed = (1400 + gameroom + (WHRE * # of games )) / (# of games + 2)

This mutes the effect of gameroom ratings, thereby providing no incentive to avoid getting a WHRE.  Some joker who inflates his gameroom rating to 2600 still only gets seeded at 2000, which he very likely could have earned by playing in events.  On the other hand, it does differentiate all the players with no WHRE.

That said, I'm sensitive to the fact that there have only been two events this year: the World Championship and the Postal Mixer.  Such a paucity of events undermines the value of WHRE.  Maybe until such time as we have more events year-round it would be just as well to use

seed = (1400 + gameroom + (WHRH * # of games )) / (# of games + 2)

I prefer this to your original proposal, because it differentiates players with no WHRH.
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Sep 26th, 2013, 12:44am
We have seen the 2013 WC pairing system in action exactly once.  I like it in all the priorities except one: how players are ranked within their score group.

We saw after two rounds that the players in the top score group were ranked almost exactly opposite their initial seeds.  This tells me that we overdid it on the principle of excluding pre-tournament information.

In my opinion there is a balance between using seed ratings to preserve climatic pairings for later in the tournament (a benefit) and rewarding rating manipulators (a detriment).  I simply don't buy into the idea that internal evidence of playing strength, however weak and tenuous, should override pre-tournament evidence of playing strength, however well-established.  We need to restore the balance so that 1 vs. 3 pairings don't happen in an early round of a 40-player tournament.

The next thing to try is exactly what Janzert suggested: eliminate the unseeded tournament performance rating (UTPR) and use only the seeded tournament performance rating (STPR) to rank within a score group.  Not only is it simpler, I expect it will strike a better balance between pre-tournament and in-tournament information.

For starters, an early-round bye won't drop the #1 seed to the bottom of the top score group.  The #1 seed might lose a place or two, or even stay #1 if the other top players had wins over noobs instead of a bye.  That's fine by me.  I don't think that the #10 seed trouncing the #31 seed is such a great accomplishment that it should catapult him above an idle #1 seed, unless their ratings were quite close to begin with.

And the improvement won't just be in the handling of byes.  More generally the STRP rankings will be likely to make only small changes in the seed ordering in early rounds, unlike the complete scrambling after the second round we had last year.  Only in later rounds will in-tournament strength of schedule completely dominate pre-tournament rating.  This is what I originally envisioned.  I hoped the UTPR would serve as a better formula for SoS than we had been using, but it wasn't.  Now I hope that STPR will be better than either.

Of course, just like using UTPR in 2013, the consequences of using STPR in 2014 could be surprising.  We could find out it lands us back in the situation of rewarding top seed(s) too much, thus encouraging rating manipulation too much.  I doubt it will happen, but you never know.  I think we have to try to it to find out, simply because UTPR was unsatisfactory in 2013.
Title: Re: 2014 World Championship Rules
Post by Janzert on Sep 26th, 2013, 11:13am

on 09/25/13 at 23:28:46, Fritzlein wrote:
Did you leave off some parentheses there?

seed = (1400 + (WHRH * # of games )) / (# of games + 1)


Sorry, I unthinkingly decided that all those parantheses weren't really needed when writing the post.


Quote:
seed = (1400 + gameroom + (WHRH * # of games )) / (# of games + 2)

I prefer this to your original proposal, because it differentiates players with no WHRH.


The starting gameroom rating is 1400 anyway so do you think the dummy rating is still worthwhile? It is essentially just reducing the magnitude of gameroom rating change and lowering the influence of WHRH on low number of game players. The former seems that it may be beneficial the latter less so. In which case maybe it should be:

seed = (( (1400 + gameroom) / 2 ) + (WHRH * # games)) / (# games + 1)

But I'm not sure the complication is worth it over simply:

seed = (gameroom + (WHRH * # games)) / (# games + 1)

Janzert
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Sep 26th, 2013, 12:18pm

on 09/26/13 at 11:13:30, Janzert wrote:
In which case maybe it should be:

seed = (( (1400 + gameroom) / 2 ) + (WHRH * # games)) / (# games + 1)

Agreed.


Quote:
But I'm not sure the complication is worth it

There will be players who have played against bots but not against other humans.  I do think it is worth using that information rather than seeding all such players at 1400.  Of course there could also be players without any games at all, either against bots or humans, who would still enter at 1400, but I can't recall any such registrants in the past.
Title: Re: 2014 World Championship Rules
Post by Janzert on Sep 26th, 2013, 1:41pm
I must not have been clear. The complication I'm not sure that is worth keeping is the 1400 dummy rating. I've come around to using the gameroom rating already. ;)

One of my original reasons for dropping the gameroom ratings was that for some reason I was thinking that they neither had the same anchor nor the same scale as WHR ratings. But that is incorrect, at least in theory the scale should be the same. Also in practice the anchoring seems to not be extremely different. I was further thinking that like last year it would be a binary choice of one or the other which seemed a mess to sort out.

Using a weighted average between gameroom and WHR seems a workable and practical solution.

Janzert
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Sep 26th, 2013, 11:25pm

on 09/26/13 at 13:41:09, Janzert wrote:
I must not have been clear. The complication I'm not sure that is worth keeping is the 1400 dummy rating. I've come around to using the gameroom rating already. ;)

Oh, sorry.  You were perfectly clear.  I just read too fast and misunderstood.


Quote:
seed = (gameroom + (WHRH * # games)) / (# games + 1)

So someone who has no WHRH rating would be seeded entirely by their gameroom rating?  Nightmare scenario there is someone creating a new account, bot-bashing up to 2800 rating, declining to play any humans, and getting the top seed.

My thought was that if someone can botbash up to 2800 but only gets seeded at [1400 + 2800]/2 = 2100 is someone who could legitimately get a 2100 WHRH or better, and thus would have an incentive to work on the WHRH rather than trying to game the system.
Title: Re: 2014 World Championship Rules
Post by Janzert on Sep 27th, 2013, 8:45pm
ok, I think I'm convinced. Seems like this is probably the best compromise for this year:

seed = (((1400 + gameroom) / 2) + (WHRH * # of WHRH games)) / (1 + # of WHRH games)

Here is a spreadsheet (https://docs.google.com/spreadsheet/ccc?key=0Aq1sLIA-T44zdGd4SGZrZlBZRzZfVHlBM0k0Q0NleEE&usp=sharing) with today's (9-27) WHRH with this seeding formula and a column to show the difference from straight WHRH.
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Sep 27th, 2013, 9:43pm

on 09/27/13 at 20:45:03, Janzert wrote:
Here is a spreadsheet (https://docs.google.com/spreadsheet/ccc?key=0Aq1sLIA-T44zdGd4SGZrZlBZRzZfVHlBM0k0Q0NleEE&usp=sharing) with today's (9-27) WHRH with this seeding formula and a column to show the difference from straight WHRH.

For players in the top 100 or 200, it is quite clearly advantageous to have more WHRH games, because the other component drags you down.  This is well and good as far as I am concerned.

Further down the spreadsheet the trends are less clear.  We'll just have to see whether the seeds in the actual tournament seem reasonable, and/or whether the seeding system induces weird behavior on the part of tournament entrants.
Title: Re: 2014 World Championship Rules
Post by aaaa on Sep 30th, 2013, 9:24am
No matter how slight, do we really want to allow even for the possibility that the same set of games plus outcomes can lead to a different final ranking depending on what the players' seed ratings were? And if an exception is made for the final ranking such that that is calculated based purely on what happened during the tournament, isn't it suspiciously unnatural if that isn't then just as well extended to the scheduling?
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Sep 30th, 2013, 12:53pm

on 09/30/13 at 09:24:48, aaaa wrote:
No matter how slight, do we really want to allow even for the possibility that the same set of games plus outcomes can lead to a different final ranking depending on what the players' seed ratings were?

Yes, I do want to allow for that possibility, because there is a positive benefit that outweighs the probable harm.  To me this isn't a case of absolutes, but rather one of relative advantages, relative disadvantages, and the probabilities of each.

If we seed a tournament in any way, i.e. if we don't place all entrants on an equal footing starting with the first round being paired randomly, then the players' seed ratings will affect the final ranking via their effect on pairings.  Having accepted that seed ratings already have an impact on the final standings, in exchange for the tangible benefit of delaying climactic pairings, improving ordering of players relative to performance, and increasing the chance that the best player is crowned champion, I don't have to reach very far to accept a small possibility of seed ratings affecting the final outcome beyond the way they determine pairings.

The tournament outcome under Janzert's proposal to use STPR and not use UTPR will still be determined overwhelmingly by wins and losses of players in their games.  The current discussion of how to pair and rank players is a relatively minor refinement to a system that is basically functioning well.  I won't be heartbroken if no changes are made from last year; things basically worked then.  The issues with pairing last year were drowned out by over-the-board performance.

I understand, of course, if others take a different stance towards the relative benefits and detriments of various pairing/ranking schemes.  I'm merely expressing the balance that I perceive.
Title: Re: 2014 World Championship Rules
Post by Janzert on Oct 3rd, 2013, 12:22am
I agree with what Fritzlein said, I just want to add that besides the pragmatism motive I also would like to make the tournament system as clear as possible. Part of that is reducing complexity where it's not giving an appropriate benefit.

Janzert
Title: Re: 2014 World Championship Rules
Post by Janzert on Oct 3rd, 2013, 1:18am
I'd like to hear any thoughts on the idea of making the last round of the preliminaries step up to the next time control. Last year the step up was done in the first round of the finals.

There are two primary motivations. First is that it gives the players on the cusp of making the finals a higher quality time control game to decide whether they make it in or not. Second is that it gives all the players in the tournament a taste of a longer time control.

The largest disadvantage I see is that it may be an overwhelming number of games to handle at the longer time control for that one week.

Janzert
Title: Re: 2014 World Championship Rules
Post by Hippo on Oct 3rd, 2013, 10:56am
I would prefere all games in preliminary with the same time controll.
Title: Re: 2014 World Championship Rules
Post by supersamu on Oct 3rd, 2013, 12:12pm
I also think that every round in the preliminaries should have the same time control.
What does it really matter for a game in what order it is played?
The first game is as important as the last game in the preliminaries, at least in the sense that you either lose a life or don´t lose a life.
The last round isn´t really special in any way.
(I know that the pairing algorithm changes things up and my statement is not entirely true (it would be true for random match-ups), but I still think that the importance of the game in the first round is almost as high as the importance of the game in the last round.
Having distinct time controls for different phases of the tournament makes it clear that the second part of the Championship has begun.

As for the Seeding algorithm:
If we want to include everybody and not sort possible participants out, because they haven´t played enough games against humans (and I think we should include everyone),
we either have to dilute the purity of the seeding algorithm for sake of simplicity or implement a "fair" seeding algorithm that is very complicated.
I think we are in a good spot with Janzert´s algorithm, although I have very little experience with Tournaments.
Title: Re: 2014 World Championship Rules
Post by Janzert on Oct 4th, 2013, 9:08pm
Ok sounds like there isn't any interest in the longer time control for a preliminary round, so I'll drop that idea.

Next topic, does anyone mind or see a problem if the time scheduling is done  2-3 hours earlier (e.g. Monday at 2100 or 2200UTC)? Of course it would be pushed back if the final game of the previous round was still in progress.

Janzert
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Oct 4th, 2013, 10:54pm

on 10/04/13 at 21:08:57, Janzert wrote:
Next topic, does anyone mind or see a problem if the time scheduling is done  2-3 hours earlier (e.g. Monday at 2100 or 2200UTC)? Of course it would be pushed back if the final game of the previous round was still in progress.

You say you have a schedule conflict every Monday evening.  Does that schedule conflict have a predictable end time?  I would slightly prefer having the pairing be done at the same time every week a few hours later, i.e. consistency to me would outweigh having more advance notice.

Conversely, the earlier pairing time would be just fine with me, except that it would mildly annoy me every time it got pushed later by an unpredictable amount.
Title: Re: 2014 World Championship Rules
Post by Janzert on Oct 5th, 2013, 2:12am

on 10/04/13 at 22:54:31, Fritzlein wrote:
You say you have a schedule conflict every Monday evening.  Does that schedule conflict have a predictable end time?


I have a training on Monday evenings 3 weeks a month starting at 24:00UTC. I usually arrive roughly 10 minutes early and it generally lasts 2-3 hours. A few times a year it may go as long as 3.5 hours. Travel time to and from is less than 10 minutes.


Quote:
Conversely, the earlier pairing time would be just fine with me, except that it would mildly annoy me every time it got pushed later by an unpredictable amount.


Looking at last year a 22:00 time would only have been interfered with one time when game 255808 (http://arimaa.com/arimaa/gameroom/opengamewin.cgi?role=v&side=w&gameid=255808) ended at 21:59. Interestingly I believe it's also the only game that could have even theoretically made it past 24:00.

Looking theoretically, a game in the last time slot could make it 2, 4 or 6 hours past the 24:00 mark depending on tournament stage.

Janzert
Title: Re: 2014 World Championship Rules
Post by Janzert on Oct 11th, 2013, 5:16pm
For now at least I went with scheduling 3 hours later at 03:00UTC.

I think I'm also going to leave the forfeit procedure alone for this year. I'd still like to try changing it eventually. But maybe it can be changed earlier in the year and tried in a few events before being used in the WC.

Janzert
Title: Re: 2014 World Championship Rules
Post by chessandgo on Oct 12th, 2013, 9:10am
I just became aware of the new (and lenghtier) Code of Conduct replacing the Cheating section. I'm not sure how to inquire about its intent / existence of discussion on this topic / etc without using unwelcome smileys :) oops.
Title: Re: 2014 World Championship Rules
Post by Janzert on Oct 12th, 2013, 12:59pm
Certainly I'd like to hear feedback on the new code of conduct. It is heavily based off of the US Chess Federation code of ethics, along with bits and pieces picked up from a few others that I'm familiar with.

There are two reasons I hadn't explicitly brought it up for discussion yet. First I'm not really happy with the TD being the primary arbiter. I'd much rather have an ethics committee like the USCF does. But given the difficulty of having the committee get together in a timely manner and performing a considered discussion over the internet at any given point throughout the length of the tournament, I don't think it's a practical option. Maybe I'm wrong on that. Either way I'd like to see any complaint resolved within a few days.

The second thing I haven't done yet is to check with Omar that he is willing to act as the final appeal. Although I'm not sure it will work well if he doesn't, given that he is the final person that will need to enforce any sanction.

Janzert
Title: Re: 2014 World Championship Rules
Post by chessandgo on Oct 13th, 2013, 6:18am
The rules on cheating used to very brief and as non-specific as possible in order to avoid unenforceability, which was even mentioned verbatim. This mention to enforceability has disappeared, everything became a lot more specific, and about 40% of the space is even dedicated to underlining the procedure for a player to file a complaint.

Any particular reason for this all? Was this new version written by someone else (Omar, or Fritz maybe?) if you don't like some of it Janzert?
Title: Re: 2014 World Championship Rules
Post by Janzert on Oct 14th, 2013, 12:49am

on 10/13/13 at 06:18:46, chessandgo wrote:
The rules on cheating used to very brief and as non-specific as possible in order to avoid unenforceability, which was even mentioned verbatim. This mention to enforceability has disappeared, everything became a lot more specific, and about 40% of the space is even dedicated to underlining the procedure for a player to file a complaint.


I'm confused by this, I believe the new code is in every way less specific and more general than the old cheating section at least as far as what it applies to. Certainly I suppose the procedures to be used have been spelled out much more specifically. But that was meant to improve the enforceability, at the very least by hopefully clarifying the process and thereby improving the perceived fairness.


Quote:
Any particular reason for this all?


I've experienced too many situations, particularly in online gaming contexts, where there has been completely unacceptable behavior exhibited toward others because of their race, religion, gender, etc.. We have been quite fortunate in the Arimaa community to have so far avoided major problems and I'm also sure Omar would take appropriate action if there ever were a problem. But I also want it to be explicit what the community standards are and what action to take if there is a problem.

Once there is a general code of conduct it seems logical to include cheating under it.


Quote:
Was this new version written by someone else (Omar, or Fritz maybe?) if you don't like some of it Janzert?


I hope I didn't give the impression that I was being forced, pressured, or even suggested into this. This was solely my idea to do this, so any blame should be thrown directly my way. As mentioned though, the general form and many specifics used come mostly from the USCF Code of Ethics; while also referencing and drawing from the PyCon Code of Conduct (https://us.pycon.org/2013/about/code-of-conduct/), the Guild Wars 2 Rules of Conduct (https://www.guildwars2.com/en/legal/guild-wars-2-rules-of-conduct/), and probably various other sources that I'm currently forgetting. So any good parts are certainly the result of blatantly stealing^wborrowing from them.

My primary dislike is having all the responsibility for a decision falling on me. I'd rather stick to the USCF method of using an established ethics committee to handle all complaints. But at this stage given the community size, tournament length and need for very quickly resolving any issue I don't have the confidence that will work quite yet.

One other part that still needs to be expanded are the specific examples. They should help clarify that it doesn't just apply to tournament altering actions.

In talking to Omar, he did agree to be the final point of appeal. He also reminded me that Tournament Organizer and Licensor need to also approve any sanction that extends outside of the bounds of the current tournament.

Eventually I think there should be a general Arimaa Code of Conduct that any tournament simply falls under. But using one within a tournament for a time or two and refining it a bit first is probably a good first step.

Janzert
Title: Re: 2014 World Championship Rules
Post by rbarreira on Oct 14th, 2013, 5:09am
I have some minor feedback regarding this:


Quote:
by reading or listening to live chatroom or audio commentary


There is no such thing as live chatroom or audio commentary of WC games, as the commentators are always seeing a delayed game.

So I'd suggest removing this part of the rule, or removing the word "live". Preferably the latter IMHO.
Title: Re: 2014 World Championship Rules
Post by chessandgo on Oct 14th, 2013, 8:07am

on 10/14/13 at 00:49:43, Janzert wrote:
I'm confused by this, I believe the new code is in every way less specific and more general than the old cheating section at least as far as what it applies to.


"tampering with the server" and "Harassment, intimidation or any other display of disrespect or unsportsmanlike conduct toward others in the community." are new, it seems. I'm favourable to the new, more detailed rules, but I understand Fritz and Omar have always been big on keeping the amount of forbidden stuff to a bare minimum. So I'm wondering about the policy change.

For example, the new paragraph on multiaccounting had been expanded with "Participating under a secondary account with the aim to conceal the relation to your primary account." I understand (bad-faithed rant alert) that multiaccounting 99% didn't happen in 2012, in which case I'd like to add that MAing 98.5% never happened in the history of the championships. Is it necessary to expand the rules on MAing to a very mild infraction (concealing the relation to your primary account won't hurt other people that much), when the main infraction is not expected to happen? (end rant).


on 10/14/13 at 00:49:43, Janzert wrote:
This was solely my idea to do this, so any blame should be thrown directly my way.


Oh I hope I didn't give the impression to blame anyone with anything, keep up the good work Janzert!
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Oct 14th, 2013, 8:51am

on 10/14/13 at 08:07:47, chessandgo wrote:
[...] but I understand Fritz and Omar have always been big on keeping the amount of forbidden stuff to a bare minimum. So I'm wondering about the policy change.

My recollection is that Omar and I have had quite different attitudes towards the code of conduct: he has always wanted to forbid anything he thought was unfair, whereas I have been reluctant to write unenforceable rules.  It seems Janzert has his own ideas different from both Omar and myself, but it wasn't a unified front in the past either.  :)  I don't know what to expect from Janzert's proposed changes, but I figure we should give it a chance and see how it works.

If we can foresee problems with the new code of conduct, by all means let's fix them before they happen, but I can't quite tell what problems you are expecting.  My recollection from the one proven case of cheating (722caasi playing moves taken from live commentary) was that you were strongly opposed to delaying commentary to prevent cheating.  I recall that you were not particularly disturbed by the possibility that cheating might happen again if we didn't make changes, i.e. you wanted live commentary to continue no matter who might use it to get advice about an ongoing game.

Is that your main concern today, i.e. are you less concerned about cheating than you are about over-zealous sanctions for minor offenses?  Is it that you expect cheating to happen and you can tolerate the cheating itself better than you can tolerate misguided responses of the community?  Or am I misinterpreting the thrust of your posts due to my memory of one past discussion?
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Oct 14th, 2013, 8:53am

on 10/14/13 at 05:09:53, rbarreira wrote:
So I'd suggest removing this part of the rule, or removing the word "live". Preferably the latter IMHO.

Yes, let's get specific.  Suppose a player does not log out of the chat room during a World Championship game.  Suppose the other community members in the chat room talk about the game, including commenting on the delayed moves as soon as they see the moves.  Is this an infraction by the player?  If so, what is the proposed sanction?
Title: Re: 2014 World Championship Rules
Post by chessandgo on Oct 14th, 2013, 9:00am
Oh ok, thanks for the correction Fritz. And no, I am not expecting any particular problem, and I usually have a high tolerence level. I am just fooling around :)
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Oct 14th, 2013, 9:13am
Just for fun: http://www.chessbase.com/Home/TabId/211/PostId/4011396/the-shoe-assistant--ivanov-forfeits-at-blagoevgrad-051013.aspx
Title: Re: 2014 World Championship Rules
Post by chessandgo on Oct 14th, 2013, 9:31am

on 10/14/13 at 08:53:13, Fritzlein wrote:
Yes, let's get specific.  Suppose a player does not log out of the chat room during a World Championship game.  Suppose the other community members in the chat room talk about the game, including commenting on the delayed moves as soon as they see the moves.  Is this an infraction by the player?  If so, what is the proposed sanction?


Indeed. Hence my asking what the new "philosophy" behind the rules are. If we keep the former "don't forbid what we can't prevent" we can just skip mention of the chatroom / live feed altogether (that's why the delay is in place after all). If we want to be more specific and extensive in what we forbid, we'll need to be very careful.

Wait, the old rules actually did forbid " including live chatroom and audio commentary." http://arimaa.com/arimaa/mwiki/index.php?title=2014_World_Championship_Rules&oldid=3755#Cheating

I must misremember then. I'm confused. What would have been the sanction as of last year to the situation you describe then Fritz?
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Oct 14th, 2013, 9:43am

on 10/14/13 at 09:31:29, chessandgo wrote:
What would have been the sanction as of last year to the situation you describe then Fritz?

I ruled as rbarreira suggested, namely that the commentary isn't "live", so there is no infraction.  I get the feeling, however, that Omar would have been of a different mindset, i.e. that even listening to the delayed commentary is cheating, and therefore subject to sanction.  This definitely needs to be clarified.


Quote:
If we keep the former "don't forbid what we can't prevent"

There is a middle ground here, because the TD can prevent a great deal of cheating by sending a live representative to be present during games.  If someone (like Ivanov) is suspected of cheating, perhaps we can't forfeit or ban them immediately just on suspicion, but we certainly can increase surveillance to prevent future cheating.  So there is a case to be made for keeping "unenforceable" rules on the books, namely that in extreme cases, we can increase enforcement.
Title: Re: 2014 World Championship Rules
Post by Hippo on Oct 14th, 2013, 9:49am
To cheating by listening delayed broadcast/chat ... I don't think it would help except in stalemated positions with both (weak) playes without a plan.

In all cases I would like radio disabled in play window for players of WC games till the game ends (and may be automatically opened when the game ends).

I would not like argumentation about missclicked radio button during the game.

I have not read the wording considering radio yet ...
Title: Re: 2014 World Championship Rules
Post by Janzert on Oct 14th, 2013, 10:54am

on 10/14/13 at 05:09:53, rbarreira wrote:
There is no such thing as live chatroom or audio commentary of WC games, as the commentators are always seeing a delayed game.

So I'd suggest removing this part of the rule, or removing the word "live". Preferably the latter IMHO.


First to clarify, I do not consider commentary on a delayed game to be "live". If it is still "live" then either the delay should be increased or dropped altogether, since that seems to be the only reason for it.

This part is basically just copied from last year. I thought about dropping it given the normal situation that the game is delayed for spectators. But should a game be run without the delay, say through a misconfigured game restart, it seemed relevant to still include. Having said that, even if the whole list of examples were removed it would not change what is covered by these rules. The examples should just clarify and make explicit some of the things that are covered.

Janzert
Title: Re: 2014 World Championship Rules
Post by rbarreira on Oct 14th, 2013, 12:23pm

on 10/14/13 at 08:53:13, Fritzlein wrote:
Yes, let's get specific.  Suppose a player does not log out of the chat room during a World Championship game.  Suppose the other community members in the chat room talk about the game, including commenting on the delayed moves as soon as they see the moves.  Is this an infraction by the player?  If so, what is the proposed sanction?


From past experience, when someone forgets to close the chatroom people tend to avoid commenting on the game. This tells me people feel some advantage could be gained from their comments even though the game is delayed. There are two obvious examples of where even delayed commentary can help:

- general strategy (saying something like "in the medium-long term, that rabbit could be advanced and create dangerous situations")

- situations where the board position repeats itself (this one is more rare but still possible).

This leads me to conclude that either:

a) we're OK with people gaining that advantage (so remove that text from the rules); or

b) we're not OK with it, in which case:
b1) increase the delay and remove the text; or
b2) keep the delay as it is (or increase it) and remove the word "live" from that part of the rules

If it's decided the rule should stay, the sanction could be anything from replaying/forfeiting that game to disqualification (I would go for forfeiting the game).

PS: If, on the other hand, we want to completely remove unenforceable rules, that also suggests we remove that rule, since it's practically impossible to guarantee someone isn't using another device to receive the commentary.
Title: Re: 2014 World Championship Rules
Post by Janzert on Oct 14th, 2013, 12:51pm

on 10/14/13 at 08:07:47, chessandgo wrote:
"tampering with the server" and "Harassment, intimidation or any other display of disrespect or unsportsmanlike conduct toward others in the community." are new, it seems. I'm favourable to the new, more detailed rules, but I understand Fritz and Omar have always been big on keeping the amount of forbidden stuff to a bare minimum. So I'm wondering about the policy change.


"tampering with the server" came from an example out of the USCF rules where the phrase is "tampering with the clocks". I don't believe we could prove every case, or even most cases, of server tampering. It does seem to very clearly fall outside the bounds of fair play and good sportsmanship though. :)

The last example "Harassment, intimidation... etc." was just my initial poor attempt to show that it applied outside of a specific game context and includes e.g. chatroom, forum and other means of communication. It really is much too vague and should be broken up into a few more specific examples.


Quote:
For example, the new paragraph on multiaccounting had been expanded with "Participating under a secondary account with the aim to conceal the relation to your primary account." I understand (bad-faithed rant alert) that multiaccounting 99% didn't happen in 2012, in which case I'd like to add that MAing 98.5% never happened in the history of the championships. Is it necessary to expand the rules on MAing to a very mild infraction (concealing the relation to your primary account won't hurt other people that much), when the main infraction is not expected to happen? (end rant).


You're right concealing the relation doesn't really matter. There are two separate issues with multiple accounts. First, entering the tournament multiple times is pretty clearly always wrong. Second, using a separate account to hide or otherwise misconstrue some information. In this case the only part I think we're really concerned about is the rating. There are also legitimate reasons to change accounts or even switch back and forth (e.g.  bieber4ever seemed really cool 3 years ago, to omar vs. omarFast). So a blanket ban on having multiple accounts is wrong as well. I've changed the example by splitting it in two to separately address the two concerns and modified the secondary account one to be specific about concealing the rating.

Janzert
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Oct 14th, 2013, 1:23pm

on 10/14/13 at 12:23:59, rbarreira wrote:
From past experience, when someone forgets to close the chatroom people tend to avoid commenting on the game. This tells me people feel some advantage could be gained from their comments even though the game is delayed.

Another explanation is that people thought that the player who inadvertently stayed in the chat room could be forfeited if there were chat comments, so they didn't comment so as not to jeopardize the status of someone who wasn't trying to cheat.

I feel that at least a small advantage can indeed be gained from delayed commentary, so listening to it is indeed unfair, but if we make no attempt to enforce the rule, it shouldn't be against the rules.  I don't see this as an absolute, i.e. there is a detriment from possible cheating but also a value to semi-live commentary, and we have found some balance.  Rbarriera, I am not OK with people gaining this advantage.  I don't want anyone to get outside assistance from any source during the game, but I see harm rather than benefit in an unenforced rule.  I know Omar (and others) feel differently, and want cheating to be 100% against the rules regardless of enforcement.

I'm not going to argue for my position, in part because I don't care that much, and in part because I'm not TD this year and I think the TD should run the tournament in a way that he is comfortable with.  Whatever the rules/procedures are, we want the TD to be 100% behind it so that he follows through.

I repeat that we can indeed enforce the "no assistance" rule by means of sending live representatives.  We may want to talk more about that possibility if we are at all serious about it.
Title: Re: 2014 World Championship Rules
Post by rbarreira on Oct 14th, 2013, 5:20pm

on 10/14/13 at 13:23:03, Fritzlein wrote:
I'm not TD this year and I think the TD should run the tournament in a way that he is comfortable with.  Whatever the rules/procedures are, we want the TD to be 100% behind it so that he follows through.


I completely agree, which is why I was talking about several possibilities. Apparently (and in hindsight, obviously) I did not exhaust all the scenarios, so I don't mean to box Janzert's or anyone's opinion in them.
Title: Re: 2014 World Championship Rules
Post by Janzert on Oct 14th, 2013, 9:26pm
Since the phrase is only there as an example and meant to help clarify the rules, but seems to only be confusing it more, I've dropped the phrase about chat and broadcasts. I also removed the word 'illegally' in the phrase about giving, receiving, asking or offering advice since it might imply that there is a legal way to do so. The example now reads:

Quote:
Cheating in a game by giving, receiving, offering or asking for advice; by consulting written sources or Arimaa engines (a.k.a. bots); by tampering with the server; or in any other manner.

Great discussion on it by the way, please keep it coming. Don't feel like you may hurt my feelings by voicing any concerns you have with any part of the rules.

Janzert
Title: Re: 2014 World Championship Rules
Post by Janzert on Oct 23rd, 2013, 10:28pm
Aside from the method of server monitoring, I think I've now made all the major changes from last year that I was considering. So if there are any other updates or additional situations that should still be covered, now would be a really good time to speak up.

Janzert
Title: Re: 2014 World Championship Rules
Post by clyring on Oct 30th, 2013, 11:26pm
Rereading the current draft, I noticed that finalists are ranked first by total wins excluding byes. I know this was done last year as well, but it seems strange: Why are they initially ranked by wins and not by round reached or wins including byes?
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Oct 31st, 2013, 10:37am

on 10/30/13 at 23:26:51, clyring wrote:
Rereading the current draft, I noticed that finalists are ranked first by total wins excluding byes. I know this was done last year as well, but it seems strange: Why are they initially ranked by wins and not by round reached or wins including byes?

Because if a bye counted as a win, that would make byes too beneficial.  The thought was that if you got a bye, your schedule was automatically weaker than some who went out in the same round but didn't get a bye.  Unfortunately, that logic doesn't hold true.  Therefore, with byes not counting as wins, they can be too harmful.

I'm looking in particular at the 2013 results with browni3141 placing 8th and omar placing 9th.  Comparing their respective strength of schedule, omar clearly faced tougher opposition despite getting a bye.  Omar should have finished ahead of browni3141 despite having fewer over-the-board wins.

On the other hand, I think counting byes as wins will even more likely result in the reverse injustice, i.e. someone with five "wins" (including a bye) might clearly have a weaker schedule than someone with four wins not including a bye, which would make it unfair to automatically place the player with five "wins" ahead of the player with four.  My hunch is that counting byes as wins is even worse than not counting them as wins.

Because I see clear cases where strength of schedule is more important in my mind than W-L record, I am currently leaning towards wanting the final ranking to go by seeded tournament performance rating, i.e. ignoring byes and W-L record.  But I am hesitant to recommend this because it is such a radical change, and radical changes often have unintended consequences.  So I'm not sure what to think.
Title: Re: 2014 World Championship Rules
Post by clyring on Oct 31st, 2013, 3:50pm
I considered the reverse injustice, but intuitively decided it was probably less likely in general by thinking about a pairwise comparison of the strengths of the opponents faced by two players with N wins and 3 losses, one receiving a bye and the other receiving no bye. It is possible, but not more likely than the same situation with no byes involved at all in my estimation. Just ordering by raw performance rating would also be perfectly acceptable to me, but is somewhat less transparent from the tournament results and can lead to some weirdness where non-finalists can sometimes be ranked above finalists when there is no specific clause to avoid this.
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Oct 31st, 2013, 5:26pm

on 10/31/13 at 15:50:00, clyring wrote:
I considered the reverse injustice, but intuitively decided it was probably less likely in general by thinking about a pairwise comparison of the strengths of the opponents faced by two players with N wins and 3 losses, one receiving a bye and the other receiving no bye. It is possible, but not more likely than the same situation with no byes involved at all in my estimation.

Perhaps my intuition is too influenced by 2007.  http://arimaa.com/arimaa/wc/2007/showGames.cgi
After four rounds, compare me to the other survivors: I had the bye and my three opponents were weaker than the weakest three of other players who made it to round 5.

If the byes-count-as-wins rule had been in effect, I would have taken second place ahead of PMertens automatically.  But we each won five over the board.  Discounting our two head-to-head games which we split, and our common opponent of chessandgo, my opponents were chessandgo, Chegorimaa, seanick, and IdahoEv, while his were robinson, 99of9, Adanac, and Tore.  Clearly he had the tougher opponents, and would have earned a higher tournament performance rating.

So this is one historical case where counting byes as wins would have caused the reverse injustice.  But the pairing rules are rather different now, so it is probably less likely than it was then.
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Oct 31st, 2013, 5:46pm

on 10/31/13 at 15:50:00, clyring wrote:
[...]can lead to some weirdness where non-finalists can sometimes be ranked above finalists when there is no specific clause to avoid this.

I'm completely in favor of a specific clause that all finalists finish ahead of all non-finalists.  A reversal in performance rating wouldn't bother me at all in that case.  You see it all the time in golf: someone misses the cut with 7 over par, and someone who makes the cut goes on to finish 19 over par, not only worse absolutely, but worse on average.  It's almost inevitable just due to random fluctuation that someone who did well in early rounds will do horribly in later rounds.  Still the guy who missed the cut gets no prize money and the guy who made the cut gets a payout.  That's the way a hard cutoff works: no matter how badly you do after making the cut, you still made the cut.
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Oct 31st, 2013, 5:56pm

on 09/30/13 at 09:24:48, aaaa wrote:
No matter how slight, do we really want to allow even for the possibility that the same set of games plus outcomes can lead to a different final ranking depending on what the players' seed ratings were?

By the way, this scenario occurred in 2013.  Supersamu finished 10th and Alfons 11th according to unseeded TPR which was in effect.  But Alfons's seed rating was 2430.7 to supersamu's seed rating of 1978.3.  That difference was large enough so that, even though game results have a heavier impact, it would have pulled Alfons past supersamu into 10th place had we used seeded TPR for rankings.  So the probability that final ranking will change between UTPR and STPR might not be so slight after all.
Title: Re: 2014 World Championship Rules
Post by Janzert on Oct 31st, 2013, 11:13pm

on 10/31/13 at 17:56:57, Fritzlein wrote:
By the way, this scenario occurred in 2013.  Supersamu finished 10th and Alfons 11th according to unseeded TPR which was in effect.  But Alfons's seed rating was 2430.7 to supersamu's seed rating of 1978.3.  That difference was large enough so that, even though game results have a heavier impact, it would have pulled Alfons past supersamu into 10th place had we used seeded TPR for rankings.  So the probability that final ranking will change between UTPR and STPR might not be so slight after all.


There are actually 2 pairs in the finalists last year that swap places between UTPR and STPR. Adanac and Rabbits being the other one.

Janzert
Title: Re: 2014 World Championship Rules
Post by Janzert on Oct 31st, 2013, 11:27pm
By the way, I'm not dedicated to any particular method for ranking the finalists beyond the top 3. However, I do pretty firmly believe the tournament constraints we have to live with just don't provide enough evidence to really give a completely accurate ranking. So any method used is likely to produce anomalies that seem wrong at times.

Wins without byes, wins with byes, straight STPR, all seem to be reasonable choices. If the community were to reach some sort of consensus to switch I would not mind doing so at all. Otherwise the default is to just carry on with last years choice (i.e. wins without byes).

Janzert
Title: Re: 2014 World Championship Rules
Post by aaaa on Nov 1st, 2013, 10:19am
At the very least, I oppose performance ratings ever overruling any "hard" counts (i.e. wins, losses, byes or combinations thereof) in any context, whether it's when determining a scheduling or the final ranking.
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Nov 1st, 2013, 10:39am

on 10/31/13 at 23:27:03, Janzert wrote:
However, I do pretty firmly believe the tournament constraints we have to live with just don't provide enough evidence to really give a completely accurate ranking. So any method used is likely to produce anomalies that seem wrong at times.

That's a very good point.  There just isn't enough information for a good ranking; we're going to have to live with "wrong" rankings.

As I revisit my logic, I realize that I want STPR for the pairings, particularly in early rounds, because I think UTPR has far too little information to work with to keep the pairings meaningful (as opposed to random) and to save climactic pairings for later rounds.  But when we get to the rankings of the finalists for prize distribution, my priorities change.  By then UTPR has six games to work with for all non-finalists and seven or more games to work with for all finalists, so I am less worried that UTPR contains too little information.  At that point I would be more worried about the unfairness of letting pre-tournament ratings affect the rankings too much.

So I guess I am now leaning towards wanting STPR to be used during pairing and UTPR for the final ranking, even though there is an apparent inconsistency there, as aaaa pointed out.  I would maintain that it is logical to use STPR for pairings and UTPR for prizes, though, because the quality of UTPR is so bad at the beginning of the tournament but arguably good enough by the end.
Title: Re: 2014 World Championship Rules
Post by aaaa on Nov 2nd, 2013, 2:44pm
Then any playoff game should also be based on the UTPR rather than the STPR.
Title: Re: 2014 World Championship Rules
Post by Janzert on Nov 7th, 2013, 2:48pm
I've been working on pairing programs and have found a discrepancy between the written rules and the implementation used on the server. The rules say that for the purposes of pairing players are ranked in ascending order of number of losses then by TPR (UTPR then STPR last year and just STPR this year). The implementation* though just ranks by TPR. For most tournaments (including last year's WC) this makes no difference.

I'm thinking that ranking first by losses is probably the way to go, but anyone have any thoughts on this?

Janzert

* There also seems to be a bug that rarely gets the rank order wrong when utpr is tied and it needs to tie break with stpr. Not sure what is triggering or going wrong with that yet.
Title: Re: 2014 World Championship Rules
Post by aaaa on Nov 7th, 2013, 9:32pm
Can you give a set of input that reliably reproduces these errors?
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Nov 8th, 2013, 10:23am

on 11/07/13 at 14:48:33, Janzert wrote:
I'm thinking that ranking first by losses is probably the way to go, but anyone have any thoughts on this?

Wait, I don't understand the description of the bug.  I thought that losses were used to determine what group of people were eligible to be paired against each other, and that then players were ranked within that group to bring about folding pairing. In that sense ranking first by losses is redundant because all players within a group have the same number of losses.

Oh, but just talking through it makes the issue clear.  We must use global rankings to determine the relative merit of cross-score-group pairings when such pairings must happen.  Off the top of my head I can't think of a scenario in which that would change the pairings from the intended folding pairing, so I'm not sure it's a big deal, but it could theoretically have an impact.  Now I understand the bug and agree that it ought to be fixed.  It was at least my intent that the global ranking be first by losses, in the spirit of Swiss pairings.

Thanks for your careful attention to detail, Janzert.
Title: Re: 2014 World Championship Rules
Post by Janzert on Nov 8th, 2013, 7:18pm
Here are the problem tournaments I've run into so far:
http://arimaa.janzert.com/bugged_pairings.zip

Also I've made some modifications to the pairing code here:
http://arimaa.janzert.com/fxe_aaaa_2014.zip

I've added a divider_stpr program to pair by stpr only. But more usefully right now, I've added a "make debug" target that will print the player ranks, losses and ratings when running either divider or divider_stpr. Also make clean will remove all binary files created.

All* of the above tournaments are using last years pairing rules, i.e. both utpr and stpr. STPR only tournaments are only minimally tested so far.

The way the above tournaments were found was by comparing random tournaments against my own FTE implementation. That can be found at:
https://github.com/Janzert/tournament_tools

Currently there is just a basic FTE implementation, a program to compare two pairing programs and a small script implementing the prize distribution system.

Janzert

Edit: Sorry one of the tournaments, rankbyrating_31, is using stpr only.
Title: Re: 2014 World Championship Rules
Post by Janzert on Nov 8th, 2013, 7:56pm

on 11/08/13 at 10:23:37, Fritzlein wrote:
Off the top of my head I can't think of a scenario in which that would change the pairings from the intended folding pairing, so I'm not sure it's a big deal, but it could theoretically have an impact.


It does happen very rarely.  Encountering a situation where their are multiple different valid pairings and my and aaaa's program pair them differently seems to be quite a bit more likely and that only happens in about 0.75% of the test tournaments I've been running.

Anyway here's the example I have involving 31 players after 3 rounds of play (with 4 players eliminated). Somewhat annotated to help clarify what is happening. For the full list of seeds and previous games see the above posted tournament zip.

[Edit: See the next few posts, this example at least is just two alternate valid pairings. Sum of square rank differences on the differing pairs, using the 'losses then ratings' rankings, have been added to show that both are valid.]

Rank and pair sorting first by losses
# 1 p11 (0, -2164.5815309724)
# 2 p18 (0, -2123.7112757105)
# 3 p31 (0, -2097.5760703336)
# 4 p3 (0, -2082.3834483196)
# 5 p4 (0, -2054.6735568018)
# 6 p5 (1, -1864.7656576815)
# 7 p20 (1, -1813.5911963568)
# 8 p26 (1, -1792.4069134981)
# 9 p16 (1, -1788.6860332727)
# 10 p9 (1, -1748.1031466139)
# 11 p8 (1, -1715.6621837132)
# 12 p15 (1, -1701.1271221016)
# 13 p28 (1, -1687.5644069241)
# 14 p23 (1, -1660.1837036798)
# 15 p7 (1, -1630.7647806119)   # by rating only
# 16 p21 (1, -1492.2566153833)  p12
# 17 p12 (2, -1575.1075213419)  p2
# 18 p2 (2, -1543.1262391929)   p21
# 19 p13 (2, -1406.0832056473)
# 20 p30 (2, -1397.4574912251)
# 21 p6 (2, -1382.615982871)
# 22 p24 (2, -1377.1948313299)
# 23 p14 (2, -1360.7427962124)
# 24 p1 (2, -1323.0008138238)
# 25 p22 (2, -1293.8579516968)
# 26 p10 (2, -1213.1360696216)
# 27 p27 (2, -1184.8649214225)
# Bye: p11
p18 p4  repeat_pairing 0 loss_difference 0
p31 p3  repeat_pairing 0 loss_difference 0
p5 p21  repeat_pairing 0 loss_difference 0 rank_difference 10 DIFFER
p20 p27 repeat_pairing 0 loss_difference 1 rank_difference 20 DIFFER
p26 p23 repeat_pairing 0 loss_difference 0
p16 p7  repeat_pairing 0 loss_difference 0
p9 p28  repeat_pairing 0 loss_difference 0
p8 p15  repeat_pairing 0 loss_difference 0
p12 p22 repeat_pairing 0 loss_difference 0 rank_difference 8 DIFFER
p2 p10  repeat_pairing 0 loss_difference 0 rank_difference 8 DIFFER
p13 p1  repeat_pairing 0 loss_difference 0 rank_difference 5 DIFFER
p30 p14 repeat_pairing 0 loss_difference 0
p6 p24  repeat_pairing 0 loss_difference 0

sum of square rank differences = 653

Rank and pair not counting losses
# 1 p11 (0, 44.1778, 44.1778)
# 2 p18 (0, 34.9163, 34.9163)
# 3 p31 (0, 30.0393, 30.0393)
# 4 p3 (0, 27.5238, 27.5238)
# 5 p4 (0, 23.4657, 23.4657)
# 6 p5 (1, 7.86437, 7.86437)
# 7 p20 (1, 5.85771, 5.85771)
# 8 p26 (1, 5.18522, 5.18522)
# 9 p16 (1, 5.07534, 5.07534)
# 10 p9 (1, 4.01798, 4.01798)
# 11 p8 (1, 3.33354, 3.33354)
# 12 p15 (1, 3.06597, 3.06597)
# 13 p28 (1, 2.8357, 2.8357)
# 14 p23 (1, 2.4222, 2.4222)
# 15 p7 (1, 2.04485, 2.04485)
# 16 p12 (2, 1.48429, 1.48429)
# 17 p2 (2, 1.23471, 1.23471)
# 18 p21 (1, 0.92128, 0.92128)      <-- by losses should be moved up 2
# 19 p13 (2, 0.560994, 0.560994)
# 20 p30 (2, 0.533819, 0.533819)
# 21 p6 (2, 0.490106, 0.490106)
# 22 p24 (2, 0.475048, 0.475048)
# 23 p14 (2, 0.432123, 0.432123)
# 24 p1 (2, 0.347738, 0.347738)
# 25 p22 (2, 0.294032, 0.294032)
# 26 p10 (2, 0.184752, 0.184752)
# 27 p27 (2, 0.157005, 0.157005)
p11     # the bye
p1      p2  repeat_pairing 0 loss_difference 0 rank_difference 6 DIFFER
p31      p3  repeat_pairing 0 loss_difference 0
p4      p18 repeat_pairing 0 loss_difference 0
p10      p5  repeat_pairing 0 loss_difference 1 rank_difference 20 DIFFER
p6      p24 repeat_pairing 0 loss_difference 0
p7      p16 repeat_pairing 0 loss_difference 0
p15      p8  repeat_pairing 0 loss_difference 0
p28      p9  repeat_pairing 0 loss_difference 0
p27      p12 repeat_pairing 0 loss_difference 0 rank_difference 10 DIFFER
p22      p13 repeat_pairing 0 loss_difference 0 rank_difference 6 DIFFER
p30      p14 repeat_pairing 0 loss_difference 0
p21      p20 repeat_pairing 0 loss_difference 0 rank_difference 9 DIFFER
p26      p23 repeat_pairing 0 loss_difference 0

sum of square rank differences = 653
Title: Re: 2014 World Championship Rules
Post by aaaa on Nov 8th, 2013, 10:31pm
Do the pairings themselves ever violate the rules, or is it just the rankings as generated by the wrapper code?
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Nov 9th, 2013, 12:19am
I infer from the fact that neither algorithm paired p5 against p27 that those two had previously played.  We are trying to maximize sum of squares of rank difference and that would be the biggie, the pairing that must be made to maximize, unless it violated a higher-priority constraint such as being a repeat pairing.

With p5 vs p27 out, the second-biggest pairings p5 vs p10 and p20 vs p27 are equally good.  But I see how the choice of which of these two to take would have cascading effects on the remaining pairings that don't cross score boundaries.

My question, which may be what aaaa was getting at, is whether the two different pairings remain tied in their badness function, or whether the different rankings make one distinctly better according to one ranking, and the other distinctly better according to the other ranking.

Both algorithms are clearly obeying the spirit of the law: Avoid repeat pairings and score group crossings.  When there must be a crossing between score groups, the top player of the higher group plays down and the bottom player of the lower group plays up.  Within the players of a group, use folding pairing.  Looking over the shoulder of the respective TD's, I say either TD came up with a fine pairing.

That said, I would definitely like the algorithm to match the wording: rank first by losses, then by TPR.
Title: Re: 2014 World Championship Rules
Post by Janzert on Nov 9th, 2013, 5:31pm

on 11/08/13 at 22:31:13, aaaa wrote:
Do the pairings themselves ever violate the rules, or is it just the rankings as generated by the wrapper code?


Just to be clear those ranks just by ratings are a straight print out of the internal ranks from the pairing program.

Having said that, after some thought I'm not sure if it's possible for the two methods to ever cause different pairings. Since this just tends to mix up the boundary between different loss groups and the across group pairing tends to go from opposite ends of the groups. So I'm thinking the best chance for this to actually make a difference is if the bottom player in one loss group rated completely below the next loss group down and I'm not at all sure that is possible.

Janzert
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Nov 9th, 2013, 6:12pm

on 11/09/13 at 17:31:01, Janzert wrote:
Having said that, after some thought I'm not sure if it's possible for the two methods to ever cause different pairings.

Wait, the two methods can certainly cause different pairings, and you gave an example of that, right?  I expect that all priorities except the lowest priority (namely folding pairing) will be respected by both algorithms.  If so, the remaining question is whether one algorithm gets "closer" to folding pairing than the other algorithm does, based on some mathematical definition of "closer".
Title: Re: 2014 World Championship Rules
Post by aaaa on Nov 9th, 2013, 7:37pm

on 11/07/13 at 14:48:33, Janzert wrote:
There also seems to be a bug that rarely gets the rank order wrong when utpr is tied and it needs to tie break with stpr. Not sure what is triggering or going wrong with that yet.

From the looks of it, this is caused by a minuscule difference in UTPR. I tried to prevent this from happening by having the iterative method that calculates the performance ratings enforce a uniform calculation for each player each round, but I guess this can go astray whenever any isomorphism is established over multiple rounds.


on 11/09/13 at 17:31:01, Janzert wrote:
Just to be clear those ranks just by ratings are a straight print out of the internal ranks from the pairing program.

Then this is just a misunderstanding. Responsibility for adhering to the pairing rules is mostly concentrated in the actual respective scheduling algorithms (functions bestScheduleFTE and bestScheduleSwiss), which already need the number of losses for the remaining players in their calculation and treat the order of the given list of players as a further tiebreaker for the ranking. The order, as externally given, is by UTPR, then STPR and finally by earlier occurrence in the seed file.
Title: Re: 2014 World Championship Rules
Post by Janzert on Nov 10th, 2013, 8:52am

on 11/09/13 at 18:12:37, Fritzlein wrote:
Wait, the two methods can certainly cause different pairings, and you gave an example of that, right?


It turns out that the example above is actually an example of two different valid pairings. I've now added the sum of squares of rank differences to the differing pairings to show that. I apparently calculated it wrong when initially finding this example.

Janzert
Title: Re: 2014 World Championship Rules
Post by Janzert on Nov 10th, 2013, 8:56am

on 11/09/13 at 19:37:54, aaaa wrote:
From the looks of it, this is caused by a minuscule difference in UTPR.


Ahh, I should have known to look for that. I had to deal with the same issue in my own implementation. The joys of floating point.

Janzert
Title: Re: 2014 World Championship Rules
Post by aaaa on Nov 10th, 2013, 2:02pm

on 11/09/13 at 19:37:54, I wrote:
I guess this can go astray whenever any isomorphism is established over multiple rounds.

After properly thinking about how the algorithm actually calculates the performance ratings, I realized that this statement above doesn't make sense, so I investigated what really makes these rating discrepancies occur. It turns out that these are the result of perfectly even performances occurring through differently structured "beat paths". Let me illustrate this with graphs denoting players beating each other in the two given scenarios. The colored nodes represent the "even" players, with their colors matching if and only if their performance ratings as calculated by the algorithm do.

wrongsort_22:
http://oi39.tinypic.com/359zog3.jpg

wrongsort_38:
http://oi40.tinypic.com/35k3eip.jpg

Given the asymmetry, it doesn't seem that unfair to tolerate these false assessments of unequal performances.
Title: Re: 2014 World Championship Rules
Post by clyring on Nov 10th, 2013, 9:00pm
Asymmetry? In my eyes, those are all examples of symmetry with respect to reversal of edge direction.
Title: Re: 2014 World Championship Rules
Post by Janzert on Nov 10th, 2013, 9:07pm

on 11/10/13 at 14:02:52, aaaa wrote:
Given the asymmetry, it doesn't seem that unfair to tolerate these false assessments of unequal performances.


In wrongsort_22 p19 & p11 are 1 least significant bit* above 1.0 while the rest of those 'same rating' players are 1 lsb below 1.0. In the wrongsort_38 example,  p37 is exactly 1.0, the 'red' group is 1 bit shy of 1.0, and p3 is off by it's 1st and 3rd least significant bits.  

Given that, I'm pretty hesitant to say it's an actual rating difference and not an artifact of floating point inaccuracies.

Janzert

* In the significand, the exponent is admittedly quite small still.
Title: Re: 2014 World Championship Rules
Post by aaaa on Nov 10th, 2013, 10:49pm
Of course all those ratings are really equal. My point concerned the fairness of letting it go unaddressed, as isomorphism isn't violated and players aren't, for example, treated differently based on where they are on the seed list.
Title: Re: 2014 World Championship Rules
Post by clyring on Nov 16th, 2013, 9:09pm
I doubt it will be an issue this year given the discontinuation of UTPR in pairings.
Title: Re: 2014 World Championship Rules
Post by Janzert on Nov 17th, 2013, 8:26pm
I've added some rounding code and fixed a bug where the ratings used were occasionally incorrect*. The latest version can be found at http://arimaa.janzert.com/fxe_aaaa_2014.zip.

Janzert

* The iterative algorithm keeps improving the ratings until the error given the current scores no longer goes down. It then used to return the final ratings produced, but occasionally the last iteration would have a larger error than the previous one. So the fix is just to use the previous version.
Title: Re: 2014 World Championship Rules
Post by chessandgo on Dec 23rd, 2013, 4:28am
I dont' know where the right place for that is, I'll write it here:

I'm very worried about connection problems for the coming event. My internet connection is fantastic (outside arimaa.com), but I'm having all sorts of trouble with the arimaa.com domain. Disconnections during games, moves taking a long time to go through, forum pages taking extremely long to load and usually needing one or several refreshes, chatroom disconnection / comments send but never recieved ...

Judging by the number of HvH games lost on time over the past few days, I guess I'm not the only one. I hope the championship won't be affected too severely :(

Edit: Right on cue, it took two refreshes and a one minute load to get to the write a reply page, and sending it got me: 400 - Bad Request
Your browser sent a request that this server could not understand. I've never had this error before :) I've spent 10 minutes of constant refreshing and I'm still unable to post the comment or get the full gameroom page. Posting the next day.
Title: Re: 2014 World Championship Rules
Post by Fritzlein on Dec 23rd, 2013, 9:01am
According to the pingdom monitoring, arimaa.com has been up for 80 hours, i.e. none of the once-per-minute pings from around the world timed out.  There have, however, been 19 slow pings (defined in the tournament rules as over 750ms) in those 80 hours since the last timeout.  Also there have been 16 missed pings in the last 30 days, most recently a spate on December 19 and 20.

I guess this concerns me on two fronts.  First, I am worried that our monitoring doesn't reflect the true user experience.  I am curious whether the game timeouts and your slow page loads correspond to the slow pings.  Also I am curious how whether Janzert's new monitoring bots would have detected any issues during these times.  Obviously, the better our monitoring corresponds to actual server problems, the better we can decide when to remediate tournament issues.

Second, I am concerned that no matter what procedures we have in place to restart timed-out games, connection issues will make things unfair.  What if poor connectivity costs someone three minutes of reserve when they had five minutes?  They didn't time out, so our remedies don't apply, but they still lost reserve and got a disadvantage.  No matter what our policies are, poor connectivity stinks.

Ultimately, the answer is going to be that life is unfair, and people who have connectivity problems just have to deal with that disadvantage, because there is nothing we can do to fix it.  The tournament has gotten too large for special exceptions and replays like those that occurred in the early days of the Arimaa World Championship.  That's not very satisfactory, but it seems like own of the downsides of growth.

The most optimistic spin I can put on it is that I recall some server issues popping up last December that had me worried, but the tournament itself ran very smoothly from the server side in January, February, and March.  Hopefully the same will happen this year.  But even with the server working well last year, browni and Hippo lost games that weren't over-the-board losses.  I can't think of anything to do about it but grin and bear it.
Title: Re: 2014 World Championship Rules
Post by browni3141 on Dec 23rd, 2013, 10:31am

on 12/23/13 at 04:28:12, chessandgo wrote:
I dont' know where the right place for that is, I'll write it here:

I'm very worried about connection problems for the coming event. My internet connection is fantastic (outside arimaa.com), but I'm having all sorts of trouble with the arimaa.com domain. Disconnections during games, moves taking a long time to go through, forum pages taking extremely long to load and usually needing one or several refreshes, chatroom disconnection / comments send but never recieved ...

Judging by the number of HvH games lost on time over the past few days, I guess I'm not the only one. I hope the championship won't be affected too severely :(

Edit: Right on cue, it took two refreshes and a one minute load to get to the write a reply page, and sending it got me: 400 - Bad Request
Your browser sent a request that this server could not understand. I've never had this error before :) I've spent 10 minutes of constant refreshing and I'm still unable to post the comment or get the full gameroom page. Posting the next day.

If it's any consolation, I think my time-out against you was a problem on my end.
However I have also had a recent problem with arimaa.com and only arimaa.com. A few days ago for a few hours I could not access any part of the site. All other sites worked correctly.  It didn't seem like anyone but me had a problem at that time though. It was Dec. 20, ~12:30 am to 3:30 am EST.


Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB © 2000-2003. All Rights Reserved.