Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Events >> 2014 World Championship Format
        
(Message started by: Fritzlein on Dec 1st, 2012, 10:26am)
Title: 2014 World Championship Format
Post by Fritzlein on Dec 1st, 2012, 10:26am
OK, the rules for 2013 have been set, which means it is time to start discussing the rules of 2014!  ;D  Seriously, if there are any problems with the rules of the current World Championship, those problems very likely won't get fixed.  There is a bias against changing rules once the registration has started, and this bias increases once the tournament has actually started.  Realizations of things that are broken, or things that could be done better, except in extreme cases, can only be considered for changes next year.

Let me kick off the discussion with the suggestion that the players should not be able to influence the starting of the clock.  Right now only the Gold player can start the clock, and there is no penalty for delaying up until 15 minutes after the scheduled game time.  Furthermore, Gold is not allowed to start the clock until Silver is seated, and there is no penalty to Silver for sitting down up to 15 minutes late.  Therefore we can consider starting the Gold clock at the scheduled game time, whether or not the players are seated.

Pros: Spectators, commentators, and the opponent will not have to wait for a tardy player.  Quoting from the 2013 rules thread:

on 11/30/12 at 22:42:39, browni3141 wrote:
Being late at all to any form of formal scheduled meeting such as a championship game of Arimaa is rude. I think that the clocks should start counting down the second the scheduled time is reached out of courtesy to the spectators and attending players. I do agree however that the game window should be allowed to be opened before the clocks start.
My opinion might be different if this were a casual game between two players, but this is the World Championship, and I personally might be slightly offended if my opponent arrived late to our game. Players who take this seriously will make a strong effort to be present when the game is supposed to start.


Cons: The starting reserve is not large, so after about five minutes a tardy player will lose.  This may be a rather draconian penalty for being late by only a short time.
Title: Re: 2014 World Championship Format
Post by ocmiente on Dec 2nd, 2012, 1:09pm
Starting the clock at the scheduled game time would be OK with me, provided that players can join the game before the starting time.

Another option might be to make the starting window narrower.  Rather than 15 minutes, reduce it to 5.

However, I'm perfectly happy with the way things are now.
Title: Re: 2014 World Championship Format
Post by Fritzlein on Dec 16th, 2012, 10:43am
The way the pairing algorithm works now, players are rewarded for tough pairings in early rounds with easier pairings later, and punished for easy pairings in early rounds with tough pairings later.  I like this feature a lot, as it prevents the top seed from reaping too many rewards just from pre-tournament rating.

However, I think we went overboard in one small detail.  If there are an odd number of players, the top player gets a bye in the first round, and therefore falls to the bottom of the winners' bracket.  This seems like too much punishment for the bye.  In the second round, the #2 seed gets the bye, and then by folding pairing, the #1 seed plays against the #3 or #4 seed (assuming all the top seeds won in the first round).

This happened in 2012, and it didn't work out too badly, but that is because there were only 11 players, and thus only 6 players in the winner's bracket in round two.  Since the bottom player in the winners' bracket played down, the widest the matchup could have been anyway is #1 vs. #5.  Furthermore, Nombril was sufficiently compensated later for getting smacked early by chessandgo.  All's well that ends well.

Unfortunately if we have an odd number of players in 2013, an early clash of the titans will probably raise more eyebrows than it did in 2012, because of the larger number of players.  I like the way we "stir the pot" by adjusting the seeds based on in-tournament ratings, but it seems like too much adjustment for the bye to drop the #1 seed all the way to the lowest winner.

An undesirable consequence of the current pairings is that it is better to be #2 seed than #1 if there are an odd number of players, because the #2 seed gets the second-round bye and therefore doesn't have to face a top player until the third round.  I would rather our tournament structure didn't give anyone an incentive to sandbag.

I'm not posting this for discussion in the 2013 thread, because it is too late to make changes for this year, and also it isn't clear what we should do to fix it.  A last-minute change might create an unintended consequence even worse than the current flaw.  If we want to make changes for 2014, we need to test them much earlier.  And maybe we don't mind an early battle between top seeds?  How does everyone (especially our top seed) feel about this?
Title: Re: 2014 World Championship Format
Post by chessandgo on Dec 17th, 2012, 2:21pm

on 12/02/12 at 13:09:03, ocmiente wrote:
However, I'm perfectly happy with the way things are now.

+1. Keeping in mind that being spectator friendly is one of the top cirteria, spectating a game delayed by 6 minutes is better than being treated with no game at all. Even in a major over the board event, I'm pretty sure a player wouldn't be declared forfeit by showing up 6 mn late.

Tossing possible n-minutes disconnections in the mix, I have a feeling that a lot of players would rather have the option to wait for their opponent to be seated before starting the clock.


on 12/16/12 at 10:43:48, Fritzlein wrote:
 How does everyone (especially our top seed) feel about this?


Is that me? I'm pretty much indifferent. There will always be "counter-intuitive" matchups, no matter what. So long as the pairing algorithm prioritizes relevant things, we'll be fine. After all, having a bye *is* the easiest schedule possible.

Has the possibility to randomize the bye been discussed (says uniformly among the top bracket)? That way there'll be no way to anticipate early match-ups and try to sandbag in order to avoid certain players.
Title: Re: 2014 World Championship Format
Post by Fritzlein on Jan 13th, 2013, 5:28pm
Note to future organizer:  The problem with someone being in a different time zone than the gameroom thinks they are in happens every year.  Plan ahead for it, like I didn't. :-/
Title: Re: 2014 World Championship Format
Post by Fritzlein on Jan 13th, 2013, 5:30pm
Also, if you use the same format, change the ranking of people who don't make the finals to be in reverse order of losses, not in order of wins.  In elimination we give the bye to the highest-ranked, which means we should count wins for ranking, but in the Swiss section we give the bye to the lowest-ranked, which means we should count losses for ranking instead.  Thanks, aaaa, for noticing this.  Fortunately, there is no prize money involved, so it is a question of glory only in 2013.
Title: Re: 2014 World Championship Format
Post by aurelian on Jan 27th, 2013, 5:10pm
Randomizing byes seems like a good ideea.

I have a proposition for the Swiss system bracket:
The Swiss system bracket shouldn't have byes but rather an extra game.

There should be an extra-game option in the scheduler, I accept/don't accept the possibility of an extra game. Allowing weekly switching of this option it's debatable. On one hand a player could better manage his own time, one the other hand there is the possibility of trying to mess with the system in order to obtain a weaker schedule.

The highest ranked wishing player within the players with the least number of games should get the extra game.

The main pro would be an extra winning incent for Swiss bracket players as it opens the possibility for increased world cup experience.

An variation would be to allow for 2/3 (instead of 0/1 for even/odd number) extragames.
Title: Re: 2014 World Championship Format
Post by chessandgo on Jan 28th, 2013, 4:40pm
Now that I understand (well, not really, but at least have an idea of how the pairing system works) the pairing thing, it has some kind of "randomness" feel to it, so I take back my random byes proposal.
Title: Re: 2014 World Championship Format
Post by Fritzlein on Jan 28th, 2013, 6:58pm

on 01/28/13 at 16:40:31, chessandgo wrote:
Now that I understand (well, not really, but at least have an idea of how the pairing system works) the pairing thing, it has some kind of "randomness" feel to it, so I take back my random byes proposal.

Yes, I don't see the need to make pairing more random that it is in 2013.  So far my only concern would be that Round 3 (including Omar getting the bye) would be considered too random.  However, although many players have been confused by the pairings, I haven't heard of anyone being upset by them.  It seems things are working out just fine.  And we have definitely solved a prime complaint from 2011, i.e. the same player having to "play up" a score group in consecutive rounds.
Title: Re: 2014 World Championship Format
Post by Boo on Feb 5th, 2013, 10:37am
As I mentioned in the chat, I think the current pairing system has problems.
As I understand it now, the pairings algorithm tries to:
1. sort the players by the difficulty of the path taken so far;
2. match the players with first-last, second-one from the bottom, etc.
The idea behind such match up is that pairings algorithm should compensate the path each player takes: if a player had taken a difficult path (played strong opponents), he should be matched with weak opponent and vice versa.
The problem comes when strong players happen to sit in a bottom. Somebody who had played strong opponents and is on the top of sorted list gets another strong opponent. For example, I sit on top of 3-0 group and sit on top of no-loss group. And as a 'compensation' for a really tough opponents so far I get chessandgo. :( And if I lose, I would sit on top of 3-1 group and would receive browni3141, geting big chances of a second loss in row. Nombril-Adanac-chessandgo-browni3141 in the first 5 rounds... Could random matchups do any worse?
The system is constantly trying to even out things for the player at the bottom part of the list, but might screw them up for the other.
I think 2nd step is not logical and should be changed to the following:
2. sort the players according to their WHR (or whatever was used for initial seeding).
3. match the top player of the 1st list with the last player of the 2nd list, remove those players from both list and repeat.
In other words, the player, who had the most difficult path so far, should be matched with the player, who is expected to perform the weakest in the future (has the lowest seeding).
If such a procedure was followed, the following pairings would have been obtained for round 5:

omar-ocmiente (both players are in the upper part of the 1st list- they both received opponents with low seeding)
chessandgo-clyring (chessandgo is second in the first list, he receives the lowest WHR available as compensation)
Nombril-Brendan_M
Adanac-Hippo
Alfons-browni3141 (both players sit at he bottom of the first list and have high seedings, the algorithm makes the path harder for both.)

I think this is more logical matchups, because this system tries to compensate for the tough/easy previous rounds immediately.
Title: Re: 2014 World Championship Format
Post by Hippo on Feb 5th, 2013, 1:07pm

on 02/05/13 at 10:37:57, Boo wrote:
As I mentioned in the chat, I think the current pairing system has problems.
As I understand it now, the pairings algorithm tries to:
1. sort the players by the difficulty of the path taken so far;
2. match the players with first-last, second-one from the bottom, etc.
The idea behind such match up is that pairings algorithm should compensate the path each player takes: if a player had taken a difficult path (played strong opponents), he should be matched with weak opponent and vice versa.
The problem comes when strong players happen to sit in a bottom. Somebody who had played strong opponents and is on the top of sorted list gets another strong opponent. For example, I sit on top of 3-0 group and sit on top of no-loss group. And as a 'compensation' for a really tough opponents so far I get chessandgo. :( And if I lose, I would sit on top of 3-1 group and would receive browni3141, geting big chances of a second loss in row. Nombril-Adanac-chessandgo-browni3141 in the first 5 rounds... Could random matchups do any worse?
The system is constantly trying to even out things for the player at the bottom part of the list, but might screw them up for the other.
I think 2nd step is not logical and should be changed to the following:
2. sort the players according to their WHR (or whatever was used for initial seeding).
3. match the top player of the 1st list with the last player of the 2nd list, remove those players from both list and repeat.
In other words, the player, who had the most difficult path so far, should be matched with the player, who is expected to perform the weakest in the future (has the lowest seeding).
If such a procedure was followed, the following pairings would have been obtained for round 5:

omar-ocmiente (both players are in the upper part of the 1st list- they both received opponents with low seeding)
chessandgo-clyring (chessandgo is second in the first list, he receives the lowest WHR available as compensation)
Nombril-Brendan_M
Adanac-Hippo
Alfons-browni3141 (both players sit at he bottom of the first list and have high seedings, the algorithm makes the path harder for both.)

I think this is more logical matchups, because this system tries to compensate for the tough/easy previous rounds immediately.


I don't think this is that bad ... no pairing system would work perfect. The most important thing is that it is deterministic. The system was affected by the early chessandgo's bye and the Browni's timeout.

Now you have received bye, what seems to be good compensation. If you want to win the turnament, you have to beat anybody paired with you (with high enough probability). And the probability to be paired again early with the same players is low ...
Title: Re: 2014 World Championship Format
Post by ocmiente on Feb 5th, 2013, 5:09pm
It does seem like the pairing algorithm this year has resulted in several close pairings of the best players early in the tournament.  

The idea of using WHR seems correct, provided that nobody purposely keeps their WHR lower than it should be prior to the tournament (which has happened in the past).  I like the idea that the tournament director can use his own discretion to modify any player's initial rating if rating manipulation is suspected.

One small, but important detail that I think should be preserved is that repeat pairings should be avoided as long as possible during the tournament.  So, omar vs. ocmiente in round 5 would not be a good pairing since it would be a repeat match up.


I really like the fact that there are so many closely matched games even this early in the tournament.  Seems like in past tournaments it took more rounds to get to the interesting games.
Title: Re: 2014 World Championship Format
Post by Boo on Feb 6th, 2013, 11:41am

Quote:
If you want to win the turnament, you have to beat anybody paired with you (with high enough probability).


What about the players who are in a 2 loss group and get a miscompensated matchup? Don't they have the right to enjoy the tournament a little bit longer? The system repairing itself in the long run doesn't help in this case.


Quote:
One small, but important detail that I think should be preserved is that repeat pairings should be avoided as long as possible during the tournament.


Yes, of course, the pairing algorithm should only change the order the opponents are chosen for the players sorted by the difficulty of the path taken so far.
Title: Re: 2014 World Championship Format
Post by chessandgo on Feb 6th, 2013, 1:19pm
You're talking a priori player strength here Boo, while the system (if I understand correctly)  is designed to reflect self-contained tournament "strength". If the objective was to be fair with respect to a priori strength, then the rules would not go "maximum likelihood set of ratings in the Bradley-Terry model based on blablabla" but only take into account seeding.

Obviously seeding is a better mesure than the tournament-only based rating, since it uses a much larger sample, but the latter is essentially manipulation-free.
Title: Re: 2014 World Championship Format
Post by Fritzlein on Feb 6th, 2013, 7:09pm
A simple compromise would be to discard the unbiased tournament performance rating and rely entirely on the seeded tournament performance rating.  I'm not sure how that would work out, but we could keep an eye on it to see whether it is an "even-up" mechanism that serves the intended purpose without over-reacting.
Title: Re: 2014 World Championship Format
Post by aaaa on Feb 6th, 2013, 8:49pm

on 02/06/13 at 19:09:22, Fritzlein wrote:
A simple compromise would be to discard the unbiased tournament performance rating and rely entirely on the seeded tournament performance rating.  I'm not sure how that would work out, but we could keep an eye on it to see whether it is an "even-up" mechanism that serves the intended purpose without over-reacting.

I would oppose that, because that would make the seeds directly affect the final standings. My position was and remains that no appeal should be made to any evidence external to the tournament except where it would resolve any indifference by the internal evidence.

I may be biased, but right now I haven't gotten any clear indication of the scheduling going astray with respect to what would be desirable. Three lives for each player should be sufficient for the highly dynamic strengths of schedule to become close to respectable by the time survival is on the line for a real contender.

Remember that, as enhanced as it may be, this is still an elimination tournament, so don't expect the final result to represent the "truth" with a confidence akin to that of a round-robin league.
Title: Re: 2014 World Championship Format
Post by Janzert on Feb 7th, 2013, 1:17pm

on 02/06/13 at 19:09:22, Fritzlein wrote:
A simple compromise would be to discard the unbiased tournament performance rating and rely entirely on the seeded tournament performance rating.


For me this isn't a compromise but closer to the ideal situation, so long as seeding was by EWHR. At the same time though while I'm mildly worried about the current system pairing it hasn't so far seemed problematic.

I'm disappointed though that we haven't tried out these different tournament systems in a simulator to see how they compare.

Janzert
Title: Re: 2014 World Championship Format
Post by Fritzlein on Feb 7th, 2013, 9:50pm

on 02/07/13 at 13:17:20, Janzert wrote:
I'm disappointed though that we haven't tried out these different tournament systems in a simulator to see how they compare.

If you simulated tournaments to compare different systems, what criteria would you use to say one system is better than another?  One beloved criterion is that the tournament should have a high percentage chance of crowning the truly strongest player as champion.  However, Omar showed that, under reasonable assumptions, the tournament format with the best chance of crowning the truly strongest player is the format that awards the title to the highest-rated player without playing any games.

Of course it is absurd to give the pre-tournament rating that much weight.  On the other hand, unlike aaaa, I don't hold it as a first principle that the effect of seeding should be minimized as much as possible.  I think it would be right and fitting for seeding to have more influence than it has under the current format, particularly if that made the "even-up" mechanism work less randomly than it has worked so far.


Quote:
Three lives for each player should be sufficient for the highly dynamic strengths of schedule to become close to respectable by the time survival is on the line for a real contender.

Yes, three lives gives a lot of time for seemingly unfair pairings to even out, but even so, why not minimize the unfairness in the first place?  If I took your position to the extreme, I could argue that we shouldn't use pre-tournament ratings at all, that the early rounds should be randomly paired when UTPR makes no distinction, and that there will be plenty of time for strength of schedule to even things out before any serious contender gets his third loss.  That's just as logical as your argument, only a bit more extreme in the a priori value placed on not using seeds.

To me it seems we could use the pre-tournament ratings to make the early-round pairings better, not only reducing the need for "even up" pairings later on, but also making the "even up" mechanism effective at an earlier date rather than needing to wait several rounds for an accurate strength of schedule to be established.
Title: Re: 2014 World Championship Format
Post by Hippo on Feb 8th, 2013, 2:27am
Once again, I don't think there is anything wierd in our current system. Chessandgo received bye ... the easiest pairing. This should be compensated by him receiving stronger opponent next round. So this is generaly fair for others, but it could look unfair for his next opponent...

The problem is we want to scale him high for his opponents, but we want to give him difficult matchups at the same time.

It seems to me current solution is good enough.
Title: Re: 2014 World Championship Format
Post by Boo on Feb 8th, 2013, 3:06am

Quote:
If you simulated tournaments to compare different systems, what criteria would you use to say one system is better than another?


Theoretically most of the time each player should take the final place = to the initial strehgth (WHR) seeding. So I would compare the normal distribution of the final places of each player, trying to answer questions:
how fast does a mean of the normal distribution approach its expected value (initial seeding) for any player?
which system has a less standard deviation for any player?


Quote:
why not minimize the unfairness in the first place?

I was going to ask the same question, but you were the first to do that :)


Quote:
Once again, I don't think there is anything wierd in our current system. Chessandgo received bye ... the easiest pairing. This should be compensated by him receiving stronger opponent next round. So this is generaly fair for others, but it could look unfair for his next opponent...
The problem is we want to scale him high for his opponents, but we want to give him difficult matchups at the same time.


Chessandgo should have played a strong player who had an easy path as well, not the one who had a difficult path.
Title: Re: 2014 World Championship Format
Post by Janzert on Feb 8th, 2013, 9:54am

on 02/07/13 at 21:50:40, Fritzlein wrote:
If you simulated tournaments to compare different systems, what criteria would you use to say one system is better than another?  One beloved criterion is that the tournament should have a high percentage chance of crowning the truly strongest player as champion.


In measuring the effectiveness of any tournament system there are multiple conflicting criteria, each one a valid and justifiable concern. I do not believe one criteria should be placed as "the one" concern that overrides all others. Every tournament system will be a compromise balancing the various criteria against themselves. Also at least some, maybe many of the criteria cannot reasonably be empirically or objectively measured.

So my disappointment is merely that the few criteria that are relatively easy to measure, some of which were measured and compared in the past, have not been looked at.

Janzert
Title: Re: 2014 World Championship Format
Post by browni3141 on Feb 8th, 2013, 6:20pm

on 02/07/13 at 21:50:40, Fritzlein wrote:
If you simulated tournaments to compare different systems, what criteria would you use to say one system is better than another?  One beloved criterion is that the tournament should have a high percentage chance of crowning the truly strongest player as champion.  However, Omar showed that, under reasonable assumptions, the tournament format with the best chance of crowning the truly strongest player is the format that awards the title to the highest-rated player without playing any games.

I agree that that is the most important criterion, but I disagree that the tournament format with the best chance of crowning the truly strongest player is simply giving the award to the highest rated player.
What about a match elimination format where each round each pair of players play best of 10,000 games, with the last standing player being crowned? Surely this would be more likely (assume the players both have the stamina) to award the strongest player the title than using ratings only. Ratings are subject to manipulation and are not always accurate.

Edit: I meant to ask for more details on what omar showed.
Title: Re: 2014 World Championship Format
Post by Fritzlein on Feb 8th, 2013, 8:06pm

on 02/08/13 at 18:20:07, browni3141 wrote:
Edit: I meant to ask for more details on what omar showed.

Right, best of the ones under consideration, i.e. better than triple elimination.
Title: Re: 2014 World Championship Format
Post by chessandgo on Feb 9th, 2013, 5:08am

on 02/08/13 at 03:06:45, Boo wrote:
Chessandgo should have played a strong player who had an easy path as well, not the one who had a difficult path.


strong = having had a difficult path, so I'm (was) weak for the pairing algorithm.
Title: Re: 2014 World Championship Format
Post by Fritzlein on Feb 10th, 2013, 10:22am
Note to 2014 TD: it would be wise for the rules to explicitly set a time frame for players to appeal to change the result of a game.  Currently the rules don't say anything on this score, so I have to make a ruling on my own judgment.
Title: Re: 2014 World Championship Format
Post by Boo on Feb 11th, 2013, 2:41am

Quote:
strong = having had a difficult path, so I'm (was) weak for the pairing algorithm.


This is currently how pairing algorithm works. I was talking about how it should be changed. Sorry if I was not clear enough.

After the second round the 0=loss player group according to the path difficulty was sorted this way (I don't remember for sure, this is for illustrative purposes only):
1. omar
2. Hippo
3. supersamu
4. rabbits
5. Thiagor
6. Boo
7. Adanac
8. 99of9
9. novacat
10. Alfons
11. chessandgo
12. bye
According to WHR the same group can be sorted this way:
1. chessandgo (2623.9)
2. Alfons (2430.7)
3. Adanac (2371.0)
4. Thiagor (2336.4)
5. Boo (2283.1)
6. Hippo (2210.0)
7. 99of9 (2178.2)
8. rabbits (2167.3)
9. novacat (2048.7)
10. supersamu (1978.3)
11. omar (1931.0)
12. bye (0000)
And by following the rule "the player, who had the most difficult path so far, should be matched with the player, who is expected to perform the weakest in the future (has the lowest seeding). " the pairing would be:
1) 1. omar - 12. bye (0000)
2) 2. Hippo - 10. supersamu (1978.3)
3) 4. rabbits - 9. novacat (2048.7)
4) 5. Thiagor - 7. 99of9 (2178.2)
5) 6. Boo - 3. Adanac (2371.0)
6) 10. Alfons - 1. chessandgo (2623.9)

This is what I meant by saying "Chessandgo should have played a strong player who had an easy path as well, not the one who had a difficult path. "
Title: Re: 2014 World Championship Format
Post by chessandgo on Feb 12th, 2013, 7:20am
gotcha, sorry.
Title: Re: 2014 World Championship Format
Post by Hippo on Feb 12th, 2013, 10:52am

on 02/11/13 at 02:41:33, Boo wrote:
This is currently how pairing algorithm works. I was talking about how it should be changed. Sorry if I was not clear enough.

After the second round the 0=loss player group according to the path difficulty was sorted this way (I don't remember for sure, this is for illustrative purposes only):
1. omar
2. Hippo
3. supersamu
4. rabbits
5. Thiagor
6. Boo
7. Adanac
8. 99of9
9. novacat
10. Alfons
11. chessandgo
12. bye
According to WHR the same group can be sorted this way:
1. chessandgo (2623.9)
2. Alfons (2430.7)
3. Adanac (2371.0)
4. Thiagor (2336.4)
5. Boo (2283.1)
6. Hippo (2210.0)
7. 99of9 (2178.2)
8. rabbits (2167.3)
9. novacat (2048.7)
10. supersamu (1978.3)
11. omar (1931.0)
12. bye (0000)
And by following the rule "the player, who had the most difficult path so far, should be matched with the player, who is expected to perform the weakest in the future (has the lowest seeding). " the pairing would be:
1) 1. omar - 12. bye (0000)
2) 2. Hippo - 10. supersamu (1978.3)
3) 4. rabbits - 9. novacat (2048.7)
4) 5. Thiagor - 7. 99of9 (2178.2)
5) 6. Boo - 3. Adanac (2371.0)
6) 10. Alfons - 1. chessandgo (2623.9)

This is what I meant by saying "Chessandgo should have played a strong player who had an easy path as well, not the one who had a difficult path. "


This is interesting proposal. I am not sure how it would work if a player has intentionaly lowered his rating.
Title: Re: 2014 World Championship Format
Post by Fritzlein on Feb 13th, 2013, 7:52am
It would be great if the pairing software could allow for players to drop out and rejoin later, as in the case of UMDRevan who lost Internet connection for days due to snowstorm.  In technical terms, this would mean allowing for zero-point byes instead of only one-point byes.
Title: Re: 2014 World Championship Format
Post by mistre on Feb 13th, 2013, 9:58am
I'd like to propose a change for next year:

"If a game ends due to a disconnection the tournament performance ratings of the two players are not effected."

The Browni3141 - Novacat result had an impact on tournament seeding that likely could have been minimized if performance ratings were not effected.

And this was a mild example.  What if Chessandgo had lost due to a disconnection to the bottom-ranked player in the first round?

I don't think this change would cause people to time out on purpose as you are still getting a L result.  

Thoughts?
Title: Re: 2014 World Championship Format
Post by Janzert on Feb 13th, 2013, 11:31am

on 02/13/13 at 09:58:33, mistre wrote:
I'd like to propose a change for next year:

"If a game ends due to a disconnection the tournament performance ratings of the two players are not effected."


It creates a perverse incentive to timeout whenever you are losing a game.

Janzert
Title: Re: 2014 World Championship Format
Post by Hippo on Feb 13th, 2013, 3:15pm

on 02/13/13 at 07:52:25, Fritzlein wrote:
It would be great if the pairing software could allow for players to drop out and rejoin later, as in the case of UMDRevan who lost Internet connection for days due to snowstorm.  In technical terms, this would mean allowing for zero-point byes instead of only one-point byes.


So these 0 point byes would count as losing for the elimination purposes ... so UMDRevan would get 2 loses by the storm ... in double elimination it would be fatal anyways, in triple elimination it is closed to it...
Title: Re: 2014 World Championship Format
Post by mistre on Feb 13th, 2013, 9:41pm

on 02/13/13 at 11:31:02, Janzert wrote:
It creates a perverse incentive to timeout whenever you are losing a game.

Janzert


Is there a way to know whether it is a true disconnect or not?  If not, just use the same scoring method that is used when you can determine if you can unrate a game or not.  

If a player is truly losing and times out, then nothing changes.  But if the arimaa score determines that you are winning at the timeout then performance ratings are not affected.
Title: Re: 2014 World Championship Format
Post by Fritzlein on Feb 13th, 2013, 9:49pm

on 02/13/13 at 15:15:02, Hippo wrote:
So these 0 point byes would count as losing for the elimination purposes ... so UMDRevan would get 2 loses by the storm ... in double elimination it would be fatal anyways, in triple elimination it is closed to it...

For an elimination tournament it hardly matters, but we have six rounds guaranteed before elimination.  Someone could miss the first two rounds and still be guaranteed four games, for example Zkid who forfeited in Round 1.  I would like such a person to be allowed to rejoin for the sake of participation.
Title: Re: 2014 World Championship Format
Post by Fritzlein on Feb 13th, 2013, 9:56pm

on 02/13/13 at 21:41:55, mistre wrote:
Is there a way to know whether it is a true disconnect or not?

There is no way to know.  If I wanted to fake a disconnect, I could rip my Ethernet cable out of the back of my computer, and it would look just like a "true disconnect".


Quote:
If not, just use the same scoring method that is used when you can determine if you can unrate a game or not.  

If a player is truly losing and times out, then nothing changes.  But if the arimaa score determines that you are winning at the timeout then performance ratings are not affected.

But you would still support each player being paired within his score group?  In particular, you didn't mind browni being a one-loss player, just that he was the bottom one-loss player?  If that's the issue, then using seeded tournament performance ratings addresses the problem better than making a hack to the unseeded tournament performance ratings.
Title: Re: 2014 World Championship Format
Post by mistre on Feb 14th, 2013, 2:35pm

on 02/13/13 at 21:56:23, Fritzlein wrote:
But you would still support each player being paired within his score group?  In particular, you didn't mind browni being a one-loss player, just that he was the bottom one-loss player?  If that's the issue, then using seeded tournament performance ratings addresses the problem better than making a hack to the unseeded tournament performance ratings.


Well, since there is no way to know a true disconnect, then the result of a timeout must stand or else you would have players timing out left and right on purpose.  My proposed solution was an attempt to mitigate the effect on the seedings if a top ranked player times out vs a lower ranked player.

If using seeded performance ratings is a better solution, then I am all for it.
Title: Re: 2014 World Championship Format
Post by Boo on Mar 8th, 2013, 10:21am
I am posting this here so that it wont sink in the chat archive. Maybe future commentators will find it interesting:

2013-03-08 10:52:59 Boo i wonder did anybody consider using smth like http://www.twitch.tv/directory for commentating?
2013-03-08 10:54:36 chessandgo I didn't know twitch
2013-03-08 10:57:57 Boo this might end up catching some random viewer, that is just browsing the games
2013-03-08 10:59:22 Boo like me who just joined to listen to commentaries of some random game "Tera"
<...>
2013-03-08 12:56:57 Mahrgell btw, since it was suggested to use twitch for arimaa-broadcasts...
2013-03-08 12:58:08 Mahrgell in that case i could also try to advertise it slightly on teamliquid.net (originally a starcraft/dota site), since that community is very open for such things, they also had a lot of simlar events, where other games advertised their broadcasts and gained quite some players from it
2013-03-08 12:58:29 Mahrgell so i think, in general, using twitch would be a great addition
2013-03-08 12:58:47 Mahrgell (and it isn't that hard to use)
2013-03-08 13:06:00 Mahrgell and they are pretty much the targetgroup anyway... hardcore dedicated game, that always try to improve themselves in whatever they do and love to philosoph about how to get better and better
2013-03-08 13:10:05 chessandgo yeah it's an interesting idea, you should mention it in the forum somewhere for everyone (especially Omar) to see
Title: Re: 2014 World Championship Format
Post by aaaa on Mar 10th, 2013, 9:37pm
I'm starting to agree that the effect an early bye has on the subsequent course of a tournament is problematic. The fact that the special case of having one life doesn't correspond to how seeding normally works in single-elimination tournaments (which is to have the highest-seeded surplus of players start a round later) should have been compelling.
  • As stated already, there is currently a possible incentive for players to aim for a lower seed.
  • The algorithm "wastes" an encounter or two between two high-level players by letting these take place in earlier, less eminent rounds, devaluing them in quality here by a faster time control.
  • Most importantly, the outcome of such a game will have a relatively large and pretty much misleading influence on the relative assessment of the players by the pairing algorithm, which at that stage would have little other internal evidence to go by.

Although I'm loath to add to an already quite byzantine system, I'm therefore proposing to add a sole dummy player with a seed rating equal to the downwards arithmetic progression of the two lowest-seeded players. Every bye given in the FTE section would, for the purpose of calculating the performance ratings, be counted as a win of the respective player over this dummy. These virtual games would also count for the purpose of scheduling inside the Swiss section, giving the dummy player four different performance ratings in total, just like the others. Byes in the Swiss section, however, would not get this treatment. As the tournament progresses, the performance ratings of the dummy player would steadily decrease by the byes, thus dissipating the rating boosts from getting them in a natural fashion.

I hope this would be enough to keep the maxim that externalities should never overrule information completely internal to the tournament. If not for this principle, a whole can of worms with respect to rating systems, the parameters they use, manipulation, etc. would be opened outside the "sandbox of indifference" to which it's currently confined.
Title: Re: 2014 World Championship Format
Post by Fritzlein on Apr 14th, 2013, 11:03am
The Round 13 timeout makes me think we should add the possibility of restarting a game from the point at which the arimaa.com server first meets the criteria for server issues.  Or is this opening a can of worms?  Should it remain, for simplicity, all or nothing, i.e. start over or restore from the end?
Title: Re: 2014 World Championship Format
Post by Fritzlein on Apr 15th, 2013, 4:17pm

on 04/14/13 at 15:22:19, omar wrote:
For situations where the game is to be resumed, it might be more fair if the player on move submits a sealed move (by email or PM) to the TD soon after it is decided that the game will be resumed. The TD can set when the sealed move must be submitted. When the game is resumed the player must play the sealed move. That way the player on move cannot benefit from a long thinking time and others discussing the position.

Historically, when one player seals a move, the other player gets no additional time on his clock.  That seems fair to me.  However, that is in cases where the game was not interrupted, so all thinking about the sealed move has been done "on the clock".

In the case of a game which is interrupted by server problems, sealing a move fifteen minutes later (or hours later as would have been the case this year) does not make things fair.  The timed-out player still has gotten extra thinking time that must be compensated.  Should the erstwhile winner get extra reserve thinking time on a sliding scale based on the length of interruption before the move is sealed?

Don't forget that both players get to analyze during any downtime.  Yes, the analysis will generally be of more benefit to the player who is on move, but that player doesn't get all of the benefit.  Indeed, it is even possible that the player not on move benefits more, for example if he is ahead on material and will win if only he can survive desperate attack, unsound but sharp, by the player who timed out.  In this case an adjournment may convert a probable win by the player not on move into an almost certain win, because both players get more time to think and avoid blundering.

I don't like the idea of adding extra complexity.  There needs to be a clearer gain in fairness in order to add a layer of procedure.  For 2013, I have decided that there will be no sealed moves.  Therefore, this is a matter of discussion for the 2014 rules.
Title: Re: 2014 World Championship Format
Post by omar on Apr 23rd, 2013, 9:47am
Currently it's not possible to select which move to resume the game from, but it could be added. However, I think it might complicate things since we will have arbitrary limits like load exceeding 10 as the point at which to reverse the game to before resuming. But then what if we find that during some game the load got to 12 for a few moves but the server managed to bring the load down again and the game finished normally. Can the losing player file a complain? I think I would prefer to keep it to resuming from the time out position.

In the past when we resumed a game from the timed out position it was soon after the time out occured. I think this is the first time we rescheduled a game. This is really a tough situation to decide how to handle fairly. I can understand your preference for not wanting to add additional steps to the process. Requiring the player on move to submit a sealed move and giving some additional time to the other player as compensation might not be that much more effort though.

I'm surprised that others have not yet given their views on these points. Please chip in.
Title: Re: 2014 World Championship Format
Post by Fritzlein on Apr 23rd, 2013, 3:13pm

on 04/23/13 at 09:47:40, omar wrote:
I'm surprised that others have not yet given their views on these points. Please chip in.

The discussion so far has been driven by problems in the 2013 World Championship, but now that the tournament is one game from finished, the discussion probably won't restart until the TD for 2014 is chosen, he drafts some rules, and then asks for comments on his draft.  Then people who don't like the proposal will perhaps be more motivated to respond than they are motivated to respond in the abstract.

But first I should ask, will 2014 be essentially the same large-participation format of 2013 or more like the elite tournament of 2012?  I unequivocally will not be TD in 2014 in either case.  I expect that which kind of tournament we have depends on who is motivated to run it according to their own vision.  I can report that the job does not have to be overwhelming thanks to broad community support.  I was able to be a full-time student at the same time as the TD; my classes didn't have to suffer in order for the tournament to keep on running.

We set a record for number of participants and number of games played and surely also number of games commentated.  We have a model that seems to scale well.  On the other hand, the peak viewership for the final games has been lower this year in part because, unlike Omar who committed to be at every single game in 2012 and give commentary if no-one else did, I have only given commentary as my schedule permitted in 2013.

Omar, what is your vision for the 2014 Arimaa World Championship?
Title: Re: 2014 World Championship Format
Post by mattj256 on Apr 24th, 2013, 2:03am
I apologize if this is a stupid question, but if I miss the live commentary is it archived somewhere?  
I'm only seeing two videos here (http://arimaa.com/arimaa/videos/2013wc/?C=N;O=D).
Title: Re: 2014 World Championship Format
Post by Fritzlein on Apr 24th, 2013, 1:06pm

on 04/24/13 at 02:03:13, mattj256 wrote:
I apologize if this is a stupid question, but if I miss the live commentary is it archived somewhere?  
I'm only seeing two videos here (http://arimaa.com/arimaa/videos/2013wc/?C=N;O=D).

Recorded commentary will hopefully someday be available indexed by format, recording quality, event, date, players, and commentators.  In the mean time, you can find stuff here: http://arimaa.com/arimaa/ftp/filemanager/filemanager.php
Title: Re: 2014 World Championship Format
Post by mattj256 on Apr 24th, 2013, 11:57pm
Thanks Fritzlein!


on 04/23/13 at 15:13:33, Fritzlein wrote:
Omar, what is your vision for the 2014 Arimaa World Championship?
Title: Re: 2014 World Championship Format
Post by omar on Apr 27th, 2013, 6:06am

on 04/23/13 at 15:13:33, Fritzlein wrote:
Omar, what is your vision for the 2014 Arimaa World Championship?


I would love to see another big tournament. But it's not a must. More importantly I would love to see another community member stepping up to organize the event. If the next organizer wants to change the format a bit, I'm open to considering it; especially if that's what it takes to get one motivated. If I am the organizer, I would like to give the big format a try, but only if I'm able to find enough help for running the event. I don't want to be the TD or TC if possible.

One way to ensure a big tournament next year is to start something like a kickstarter now to pool some funds for the event. For example if we raise $4000 for the event we could set aside $2000 for the prize fund and $2000 to pay the event helpers like commentators, TD, etc. With some money set aside for running the event, I'm sure we could find enough help. Four thousand might sound like a lot but if we could get 400 people to pitch in $10 to help the next WC be a big event that would do it. Also we could try to find sponsors for the event.
Title: Re: 2014 World Championship Format
Post by Fritzlein on Apr 27th, 2013, 1:01pm

on 04/27/13 at 06:06:47, omar wrote:
If I am the organizer, I would like to give the big format a try, but only if I'm able to find enough help for running the event. I don't want to be the TD or TC if possible.

OK, then I nominate Omar for organizer and Janzert for Tournament Director.  I am willing to be one of the group of Tournament Coordinators if that means I can still participate as a player.
Title: Re: 2014 World Championship Format
Post by Janzert on Apr 27th, 2013, 5:43pm

on 04/27/13 at 13:01:11, Fritzlein wrote:
OK, then I nominate Omar for organizer and Janzert for Tournament Director.


Thanks Fritz. Knowing that you had already declined for next year, I had been planning to throw my hat in the ring. But I have to make a disclaimer along with my offer to do it.

First my work schedule is such that calling it a schedule is a misnomer. ;) Fortunately during the time of the championship I generally do have a lot of free time. Unfortunately the two regularly scheduled things in my life are Sunday's (when many of the games are played) and Monday evening at 7pm eastern (when the scheduler should be run). Especially with the great team of TCs here, I don't think any of that is insurmountable but there may be someone else who won't have the time conflicts.

While I'm willing to TD unless something really weird were to happen where I couldn't agree with the rules used, I'm also willing to work on setting up the rules for next year. My goal would be to keep the same format and base as this year. Making a few tweaks and adding for various things not explicitly covered currently. Most changes we need I expect have probably been at least mentioned in this thread already.

Janzert
Title: Re: 2014 World Championship Format
Post by Fritzlein on Apr 28th, 2013, 10:24am
Woohoo!  I'm thrilled you are willing to be the TD, Janzert, and if I had known you were considering it, I would have been less worried.  I've been fishing for a 2014 organizer for over two months and haven't even gotten a nibble.  I don't hear a chorus of disappointed wails from people who wanted to be in charge; rather a huge collective sigh of relief from people who like a big tournament and want it to happen again.  Let's just consider it settled that you are TD, so that you can get to work as soon as you have time and energy.

I don't quite understand Omar's distinction between organizer and TD, but I do understand that if you are willing to take charge of something, he will let you.  In particular, the way I interpret his last post, I think you can go right ahead and start drafting the 2014 rules for discussion.  I believe that is the best way to carry forward the conversation, because it is easier to react to something concrete than to imagine abstractly.

I encourage you to mold the format into whatever type of tournament you think would be fun.  It was a major support to my motivation for the 2013 tournament that essentially everything was the way I wanted it to be.  That kept me cheerful and grateful.  When I feel obliged to run things in a way I don't particular care for, I always start to think, "I'm not getting paid enough for this.  
Wait, I'm not getting paid anything for this!"  ;)

Putting your stamp on the tournament is more than just a way to keep things fun and pay you for your time; it is also a way to contribute ideas and show that they can work.  If I were to run the tournament again, it would be nearly a carbon copy, because everything was already running my way.  Having a new person in charge is an opportunity to show that more than one format can work without having a stick-in-the-mud veto new ideas that haven't had a chance to prove themselves.

Grab this tournament with both hands and run with.  I'm sure you will do a fantastic job.
Title: Re: 2014 World Championship Format
Post by chessandgo on Apr 28th, 2013, 12:51pm
Hey great news, thanks for stepping up Janzert! And congrats to Fritz and all the organizers for a very smooth run in 2013.
Title: Re: 2014 World Championship Format
Post by omar on Apr 28th, 2013, 8:40pm
Janzert, I'm delighted to hear that you can be the TD for the 2014 WC. You've been a member of the Arimaa community for a very long time and have made many contributions in the past (bot_OpFor, Arimaa Engine Interface software, branching factor analysis, material evaluation calculators, etc). Although you have not participated in any tournaments, I know you have a good understanding of how the tournaments are run and I trust your judgement to serve as an excellent TD. Thanks for stepping forward.
Title: Re: 2014 World Championship Format
Post by omar on Apr 28th, 2013, 10:10pm

on 04/28/13 at 10:24:58, Fritzlein wrote:
I don't quite understand Omar's distinction between organizer and TD, but I do understand that if you are willing to take charge of something, he will let you.  In particular, the way I interpret his last post, I think you can go right ahead and start drafting the 2014 rules for discussion.  I believe that is the best way to carry forward the conversation, because it is easier to react to something concrete than to imagine abstractly.


In the 2013 WC you were wearing several hats. I have defined the roles in this Wiki page:

http://arimaa.com/arimaa/mwiki/index.php/Event_Roles

So based on that you were the organizer (you chose who would fill what roles), the legislator (you wrote the rules), the coordinator (you ran the tournament management tool) and the director (you made the final ruling on how to proceed when unexpected situations came up). The referee role mentioned in the wiki is actually what we referred to as TCs in the 2013 WC.

For the 2014 WC I will serve as the organizer and Janzert can serve as legislator, coordinator and director. As the organizer I will try to make this event even bigger than the previous one.
Title: Re: 2014 World Championship Format
Post by Janzert on Apr 30th, 2013, 4:04am
Thanks Omar and everyone. I look forward to building on the great base of support that has been built in this community, site and rules that have developed over the years here.

Janzert


Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB © 2000-2003. All Rights Reserved.