Author |
Topic: 2012 World Championship format (Read 15751 times) |
|
omar
Forum Guru
Arimaa player #2
Gender: 
Posts: 1003
|
 |
2012 World Championship format
« on: Jan 19th, 2011, 3:43pm » |
 Quote  Modify
|
Regarding the WC tournament format. It's really a complex issue with various inter-related factors when you start thinking about it in detail. Here are my thoughts on the various factors and what format we should use next year.
The WC tournament:
1. Should try to select the best player. The purpose of the tournament should be to select the best player (the one with the greatest skill in the game). It turns out that tournaments are not nearly as good at identifying the best player as a rating system based on high quality games. Tournament simulations have shown that crowning the player with the highest rating beats any other tournament format. The rating system inaccuracies have to get to about +-100 before this method gets worse than other formats. So we have to somewhat give up the notion that our tournaments are really selecting the best player. Not to belittle the achievement of the WC winner, but that just how it is. So why do we have a WC event and what can we say about the player who wins it. Well, we continue to have the WC event for the fun and excitement it brings to the community and the glory it brings to the players. We can say that the event selects the player who had the best performance during the event; it is not necessarily the player with the best skill in the game, but it is still quite an achievement. Then does the format of the tournament matter. Well realizing that the tournament will not be selecting the best player doesn't mean that we should give up trying and resort to just any format. We should still try to use a format that is good at selecting the best player. For a given number of players and other parameters like rating range and rating inaccuracy it not easy to find formats that are impartial to seeding and have a significantly better performance than other formats. It does seem to me that there are theoretical limits on the performance of formats that don't rely on seedings and these could probably be worked out mathematically in terms of the number of players in the tournament and the distribution of their true ratings. Maybe this would be a better way to characterize the different formats rather than the simulations. So the bottom line is that requiring the WC format to select the best player isn't as important a factor as I once thought it should be, but we shouldn't ignore it either. On an importance scale of 1 to 10 I would give it a 6.
2. Should finish in a tolerable amount of time. There seems to be a trade-off between the length of the tournament format in terms of the number of rounds and performance of the format in terms of selecting the best player. Longer formats will tend to be better at picking the best player. But this ability to pick the best player is also related to the rating difference between the best player and the average player in the tournament; and decreases rapidly as this difference decreases. In a series match between two players where the better player has a 65% chance of winning a game, the probability of the better player winning a 3 game series is 71.8%; and 7 game series is 80.0%. If the better player has a 55% chance of winning the game than the probability of the better player winning a 3 game series is 57.5%; and 7 games series is 60.8%. It requires a 69 game series to reach close to 80%. So in future years it would require longer and longer formats as the rating difference between the top player and the average of the field decreases. But how long of a format are we willing to tolerate. If the tournament takes too long it would have fewer people who could commit the time to participate. It would really be bad if a top player was not able to play because the tournament takes too long to finish. With the WC being completely online with one game per week, I think ideally about 10 weeks would be tolerable for most people. With me as the TC I'm willing to let it go beyond that to about 12 weeks. On an importance scale I would give this factor a 9 because if the tournament is too long we could lose top players.
3. Should allow a very large number of players to participate. Some believe that the significance of winning the WC increases based on the number of players that were participating in it. However, having a large number of players has other implications as well. The length of tournament formats increase with increasing number of players. The probability of selecting the best player decreases with increasing number of players. The number of games increases thereby increasing the chance of something going wrong and the TD or TC needing to intervene. So for practical reasons I have always wanted to limit the number of players in the WC to about 8 or 16 of the highest rated players willing to play. But is it possible that this is not large enough to include the best player. Perhaps the best player is the 20th one on the rating scale. Assuming that the best player has not sandbagged his rating and is willing to play is it possible that we might miss the player because we don't have enough slots. With close to 1000 players on the Arimaa WHR rating list the difference in rating between #1 and #16 is 430. So currently I think the chances of us missing the best player in the tournament is higher due to the player choosing not to play rather than not having enough slots. Out of curiosity I checked the FIDE top 100 list to see how things would be if we had as many rated Arimaa players as Chess. The rating difference between #1 and #32 is 100 points. 100 rating points translates to about 70% chances of winning for the better player. So it's highly likely that the best Chess player still willing to play is in the top 32 players. The significance of winning the WC I think should be based more on average rating of the players participating and not on how large a number of players participated. The main reason for allowing a large number of players is to make it an event that anyone can play in. This is quite possible when the pool of players is small, but it becomes harder to allow anyone to be able to play in the WC when the pool of players gets larger. In 2008 my kids were playing in the WC because we didn't have nearly as many strong players as we do now. Of course if we allow an unlimited number of players to play in the WC than anyone can play in it. But having a large number of players in the tournament comes with it's own set of problems. It is important that everyone should have an equal opportunity to be able to play in the WC, but literally allowing everyone to play in the actual tournament is not possible from a practical standpoint. Another reason for allowing a large number of player in the WC is because it provides players an opportunity to play serious event games (game where they were not able to chose the opponent) and build their WHR and WHRE ratings. This is primarily the reason why the current WC format is the way it is. The WC preliminary rounds are really another tournament which I used to call the Open Classic. The purpose of the Open Classic was to allow large numbers of players to take part in a serious tournament. The top 8 players from this tournament are the ones playing in the WC tournament. Thus, the WC tournament is limited entry, but rather than limit entry based on ratings we decided to limited entry based on performance in the Open Classic. So why not continue doing this. Because of two key developments since 2008 when we started this two tournament WC format. First we now have the tools to allow more tournament coordinators to run events; and we are starting to see more events happening during the year. Second we now have the WHR and WHRE rating systems that provides more reliable ratings. Thus, allowing the ability to limit the WC participants using these ratings. So the Open Classic tournament can still continue, but there is less need for it to be tied with the WC tournament. So on an importance scale I would say having a large number of participants in the WC tournament is a 1.
4. Should be fair. Everyone should be able to agree that the winner deserved the top spot based on performance in only this tournament. No one should be able to say they did not get a fair chance. No one should be able to say that other players cooperated to give one of the players an advantage. Ideally a tournament format should not allow collusion, should give all players a fair schedule and should not use any seeding. But in our desire to have the tournament select the best player we tend to bias the tournament so that the top seeded player has a higher chance of winning. When I think about why a tournament has seeding it is really only to improve the performance of the tournament format in selecting the best player. It is a mild form of crowning the highest rated player as the winner. If we give up the notion that the WC tournament selects the best player and accept that it just selects the player who had the best performance during the event then we should not be using seedings at all in an effort to make it a fair tournament for everyone. However, even if we don't use seedings, elimination type tournament formats will still end up causing some players to have a more difficult schedule than others; just by chance. The only way to avoid it is to have each player play against every other player. But this opens up the possibility of players cooperating to help each other. In fact whenever a player can lose a game and still stay in the tournament there is possibility of collusion. It is even worse if a player has no chance of winning the title, but is allowed to play players who do. Only a single elimination tournament minimizes collusion, but even there some forms of collusion are not preventable. But a single elimination also has the problem of every game being very critical and not giving any room for a bad day. So there is no free lunch; all formats will have some aspect that is not quite fair. Trying to find a perfectly fair tournament is perhaps impossible, but that doesn't mean fairness can be ignored because it is easy to construct formats that are blatantly unfair. So similar to trying to select the best player, trying to have the tournament be absolutely fair isn't as important as I once thought it should be, but it is not something that we should ignore either. So I also give this factor an importance of 6.
5. Should be fun and interesting for spectators to follow. If the format is simple and easy to understand it makes it much easier to follow the tournament. As the tournament progresses it should become more climatic with each round. Using time controls that keep the game moving at a steady pace is also an important consideration, but completely unrelated to the tournament format. But why even worry about whether or not the tournament is interesting to spectators. Some would say that the WC tournament should only address the needs of the players and we should not worry about the spectators. Since we don't have many spectators to begin with they would be right. But ignoring the needs of the spectators is also a good way to make sure we don't gain spectators. In the end having more spectators is good for the players. Knowing that people are watching makes them want to perform better and it encourages more top level players to want to participate. I like to set a precedence that regardless of how many spectators we actually have, we don't ignore the needs of the spectators. So for me this factor is quite important; I would weigh it in at 8.
This year has given me a chance to see what a tournament with 32 players would be like. It is really quite a lot of games to deal with especially when they are mostly on the weekend. My weekends are quite occupied now with the WC games. If I ask the question do we really need that many players to be able to make sure we don't miss out on including players with potential to win the championship the answer is clearly no. As I mentioned earlier even with top 16 players we have a rating range of more than 400 points. So I think 16 players would be sufficient. Even if Arimaa has as many rated players as Chess I don't think we need to go to more than 32 players. When the rating range gets down to about 200 we can consider increasing the number of players. But for now I will stick with 16 players.
So how will these 16 players be selected. It is possible to come up with all kinds of schemes, but I like the most simplest one. The 16 highest rated players on the WHR system who are willing to play will be able to enter the WC tournament. The purpose of the Swiss preliminary was to avoid using the gameroom ratings to select the 8 players who would be in the WC tournament. Since the gameroom ratings could easily be inflated by playing bots it was not a fair way to limit the entry in the WC. It would be good to continue having the Open Classic Swiss tournament each year where an unlimited number of players can participate, but I would rather that it not serve to determine who plays in the WC. Otherwise I have to make sure that the integrity of that tournament is as good as the WC tournament; that gets harder to do as more and more people are in the tournament. The WHR system is not perfect, but it much better than the gameroom ratings. One good thing about it is that it doesn't even require ongoing events for one to build their WHR ratings. One can simply challenge and play other human opponents to improve the ratings; even postal games against humans are included. Eventually when we do have more off season events going the WHRE ratings could be used. There is the possibility that after attaining a high rating a player does not play any games throughout the year but still gets to enter the WC. Until I see some adverse effects of this, I think I am OK with it. Toby Hudson in fact does this and it doesn't seem to be a problem.
The floating elimination format has worked well in the past to provide a very climatic and fun to watch tournament. Simulations have shown that it is about as good as the round-robin in selecting the best player. With 16 players even a floating triple elimination can fit into 12 weeks. In terms of fairness it is good at minimizing the possibility of collusion while still giving some chances to players to recover from a bad game. The main sticking point in terms of fairness is the seeding. Since the players will not all have the same schedule some are bound to have a tougher schedule than others. So in the past we have decided that since the schedule is not going to be fair to everyone anyways, we might as well bias to favor the top seeded player thereby slightly increasing the performance of the format in selecting the best player. This is clearly not fair if you happen to be the lowest seeded player. But when ratings from high quality games are used for the seeding then it is fair. If everyone knows that a higher rating will get you better seeding in the WC tournament, they have all year long to try and improve their ratings. Even though it is easy to drop ones rating, it is much harder to increase it especially against human opponents and especially if the players did not have a choice in selecting their opponents. Working on improving your rating during the year becomes the preliminary to the WC. On the other hand it might not be all that bad to have the players draw a random number in each round to determine their seeding. The chances of the best player winning the tournament would decrease. But with higher chances of winning for everyone else it might interest more top players to play. It might also make the tournament more exciting to watch for the spectators. But with the pairings not being deterministic, it could lead to other issues of unfairness. To prevent this we might have to fix a seed for the random number generator used by the pairing program before the start of the tournament. Minor technicalities, but important considerations none the less. By a very slight margin I think I favor random seeding before each round over seeding by ratings.
We have recently also discussed the possibility of allowing players who have been eliminated to continue playing against other players who have also been eliminated. This helps to provide more WC event experience to the eliminated players. I think this was a good idea suggested by Thomas Foy and if there is interest we should implement it. However, in the games database these games would be marked as WC side event games and not actual WC games. Also I don't think the players of these games can expect to have coverage. Actually even the players of WC games can't expect coverage of their games and so the expectation would be even less for the games in the eliminated bracket.
|
|
 IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2012 World Championship format
« Reply #1 on: Jan 19th, 2011, 5:40pm » |
Quote Modify
|
Thank you for going into great detail on your thoughts on the format of the 2012 World Championship. I infer that you haven't made any final decision, and that everything is still open to further discussion.
After reading your presentation, I think the most important change to make to the format, starting immediately, is to reduce your work load. In particular, you should not be doing things that can be done by other people. You should not give any live commentary, and you should not be responsible for capturing/reformatting/uploading the audio (even though I was pressuring you to do the latter). I will henceforth capture and re-encode my own commentary if I can't find another volunteer, because the worst thing that can happen to the Arimaa community is for you to burn out. You need to conserve your energy for technical difficulties like the JS client not working (see site discussion thread), the pairing software not behaving as expected, rescheduling games upon consent of both players, etc., because presently nobody else can do those things.
I know how exhausting it is to give commentary, so I was stunned at how many games you commentated during round two on top of everything else you were handling. Why did you prioritize this? Perhaps it is because the live commentary is such a great community builder. People hang around the site more if they have the radio to listen to, and they chat more. It helps build up Arimaa as a spectator sport, which is a goal that I know both you and I set as a high priority.
It turns out, however, that I sharply disagree with you in at least one respect of building the spectator base for Arimaa. I believe that the only people who will ever watch Arimaa games are Arimaa players. It is simply not a sport such as, say, gymnastics, which one can enjoy as a spectator without ever having been a player. I can barely do a somersault myself, but I can tell whether the gymnast did a single flip or a double with a half twist. I can be amazed by the physical feats of gymnasts despite my near ignorance of the sport.
Arimaa, by contrast, can't be enjoyed by people who don't know what is going on. Yes, one's understanding (and hence enjoyment) of Arimaa can be enhanced by live commentary, but the enjoyment can't be created out of thin air. Fantastic moves aren't going to be breathtaking to people who didn't grasp the tension in the position before the move was made. For example, I found your move 17g against Adanac thrilling, but is was a thrill that a non-Arimaa player simply cannot experience even if a commentator is saying, "Be thrilled now." After all, it was just a horse sideways four steps with no goal or goal threat, no capture or capture threat, and not even a push or pull. I know you dream of non-players watching Arimaa with enjoyment, but I think you are harboring an illusion on that score. Directing a lot of energy toward an illusory goal saps energy from achieving what can in fact be achieved.
The achievable goal that will increase the number of Arimaa spectators is to increase the number of Arimaa players. If there is someday money and sponsorship for Arimaa the way there is for chess today, then it will come from the players themselves. This is the case for chess right now; it is the huge base of chess players that ultimately supports large prize funds. You imagine that future Arimaa support will come from spectators, and so do I, but the difference is that I am sure the spectators will be almost exclusively players.
This difference between us is why I fundamentally disagree with your wanting to make the World Championship an invitational event rather than an open event. The richest source of spectators who are paying attention this year, but wouldn't be paying attention if you limited the field to 16 players, are the 17-32 seeds. There are even a few of the lower seeds who are relatively new to Arimaa. By having the World Championship be an open tournament every year, we are directly building the player base and spectator base. Indirectly, too, the more people who are playing in the tournament, the more friends and family of those people will hear about Arimaa, and perhaps give it a try themselves.
You say that limiting the tournament to the top 16 by rating is sufficient for determining the best player in the world, and I agree with that. But if determining the best player in the world is only a 6 on your scale, and having a great spectator event is an 8, then limiting the participation is counter-productive to your own goals. Why would we tell people who want to participate that they can't participate, when allowing open participation is the easiest way to engage and activate the fan base?
You give two arguments against an open field: that is makes the tournament too long and that it is too much work. I'm not very sympathetic to the first argument. I estimated before that the length of an elimination tournament is C + lg(N) + lg(lg(N)), where C is a constant and N is the number of players, so doubling the tournament size from 32 to 64 adds only one and a quarter rounds. If we could actually get that many players (and double the prize donations -- how's that for spectator support?) it would be well worth the additional length. Would I accept and eighteen-round triple-elimination if we could get a thousand players signed up for it? You bet I would! For a piece of a ten-thousand-dollar prize pool, I would gladly play one serious game of Arimaa every week for four months with no breaks.
I am much more sympathetic to your argument that a larger tournament is more work for you. However, if you offload the responsibility of commentary (where your efforts don't scale at all) and incrementally improve the tournament automation as the player base increases, then your work load shouldn't grow at all. Critically, we members of the community must help you run the tournament in whatever capacity can be arranged. I think the volunteers stand ready and able to take over various functions and keep your workload manageable.
In short, I rate the open format of the World Championships a 10 on the importance scale, and I think you would be making a huge mistake to change it.
|
|
IP Logged |
|
|
|
Isaac Grosof
Forum Guru
Longtime Arimaa Fan
Gender:
Posts: 175
|
|
Re: 2012 World Championship format
« Reply #2 on: Jan 19th, 2011, 8:16pm » |
Quote Modify
|
As one of those "17-32 seeds", (28  ) I would like to make a few comments.
1. It seems intuitively possible that the tournament could both have an arbitrary number of people and have limited length for most of the players by giving the higher rated players initial byes. Then, the lower seeds' average playing time could be limited by having an elimination process. The only people who would have to play the whole tournament would be low seeds who did well, and those people would probably e willing to go on. (I know I would!)
Alternatively, players who did not want to play the whole tournament could be given byes on specific weeks at their request, with byes counting as draws.
The point is, there is not a clear trade-off between number of players and length of the tournament.
|
|
IP Logged |
Sorry about that one thing.
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: 2012 World Championship format
« Reply #3 on: Jan 19th, 2011, 8:19pm » |
Quote Modify
|
OK, here's a draft of my proposed tournament structure for the human championship:
Tournament is open to all.
Everyone plays until they have lost three times. At the end of each round, any player who has lost for the third time is eliminated.
Before each round, each remaining player gets (re-)ranked by highest performance rating (per the Bradley-Terry model) calculated over all non-forfeited games played in the tournament so far, adding for each player, a half-weighted dummy draw (i.e. a prior of .25 wins out of .5 games) against a virtual player having the corresponding WHR rating fixed before the tournament. Ties are broken randomly.
Pairing rules (each strictly overruling the next, with forfeits treated as normal losses and each double forfeit as a loss for both players):
- If an odd number of players remain, the bye must go to some player among the players with the fewest byes so far.
- Minimize the number of pairings occurring for the Nth time, then N-1th time, etc.
- Give the bye to the player with the fewest losses.
- Minimize the number of pairings between players whose number of losses differ by N, then N-1, etc.
- Give the bye to the player with the best dynamic seed (notice the swap with the previous rule).
- Based on a ranking of the non-eliminated players primarily by least number of losses and secondarily by dynamic seed, maximize the sum of the squares of the differences in rank among paired players with equal number of losses minus the sum of the squares of the differences in rank among paired players with different number of losses.
Color assignment as usual.
Consolation playoff games as usual (will the increase of number of lives cause any problems here?).
If there are more than 8 players left in the tournament, then the time control of a game is 60s/5m/100/0/4h/4m, otherwise it's 90s/5m/100/0/6h/5m (notice the full banking of time; perhaps to compensate, the starting reserves could be lowered).
|
|
IP Logged |
|
|
|
Nombril
Forum Guru
Arimaa player #4509
Gender:
Posts: 292
|
|
Re: 2012 World Championship format
« Reply #4 on: Jan 19th, 2011, 9:27pm » |
Quote Modify
|
Thanks for starting a separate thread on this. I'd like to point out a couple of things for consideration:
Use of Ratings:
I agree the WHR is much more accurate than the gameroom rating, but I don't think it should be used as the sole selector for the WC participation, or even to determine the 'best' Arimaa player. Some reasons:
A. My rating at Postal speeds 400 points higher than my overall WHR rating. Should I quit playing live games to help my WHR?
B. I'm terrible at Blitz speed. Should I quit playing games at faster times? (Going to WHRE doesn't change this - we had a number of Blitz speed events during the year.)
C. The results of playing casual games becomes more "important" and less "fun". Playing during the year to improve my seed in the Open Classic is one thing, but if I'm playing just to make the cut for the WC it becomes more like work.
I guess I've felt that ratings are a good way to find a roughly equal opponent to have a good game. Making them more important would make it more difficult to test new openings, explore different strategies, play hanzack, etc. etc.
Linking the Open Classic to the WC:
It feels like the 'prize' of qualifying for the WC is part of the draw for participation in the Open Classic. I agree with Fritz that more participation leads to more interest/involvement/spectators/excitement/etc.
Instead of a unified tournament (triple elimination), here is a suggestion to modify the current WC selection: Reserve some portion of the spots as invitation only to the top rated players, and also have some open spots for the top finishers in the Open Classic. This would address your 2nd point and allow any top players that can't commit to 3 months of weekly games a chance to opt out of the Open Classic and just play the WC.
|
|
IP Logged |
|
|
|
Sconibulus
Forum Guru
Arimaa player #4633
Gender:
Posts: 116
|
|
Re: 2012 World Championship format
« Reply #5 on: Jan 19th, 2011, 9:34pm » |
Quote Modify
|
I like that idea Nombril, It doesn't force people to pay attention to their rating to get a slot, but it also doesn't force people to play a qualifier if they're clearly good enough to make the tournament.
|
|
IP Logged |
|
|
|
ginrunner
Forum Guru
Arimaa player #5449
Gender:
Posts: 163
|
|
Re: 2012 World Championship format
« Reply #6 on: Jan 19th, 2011, 11:58pm » |
Quote Modify
|
I really like Nombril's idea about having reserved spots for the highest rated players and having open spots to those that do well in tournaments. Also, I think Fritz is spot on with his statement that cutting the tournament down would kill some of the attention, specifically from the people who would just be outside of the cut. Personally I am on the site now every day looking at who my opponent is, reviewing some of their old games for traits, and reviewing my games as well. I know that I am not one of the top players in the tournament this year but since the start this year I can already see where I am lacking in certain areas and am improving much faster than I would be if it were something I was not directly involved with. Maybe next year people will swear under their breath when they see they are paired vs me but I doubt this year that is the case. Allowing an open high level tournament like the WC boosts the community enjoyment and involvement immensely. I suggest keeping an open tournament with a high intrinsic reward (my goal for this year was making the top 8 as well as getting some high level games in). If you are looking for some way of cutting down people may I suggest making it a prerequisite to climb the entire bot ladder (not the full one of course). I honestly thought it already was a prerequisite until I saw some of the entries and realized it wasn't. This would allow only people who would be able to defend the human's title in the Arimaa Challenge to compete in a tournament designed to find those people to who should defend the title in the first place.
My 2 cents on the matter
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2012 World Championship format
« Reply #7 on: Jan 20th, 2011, 12:15pm » |
Quote Modify
|
Thanks for the feedback everyone.
Karl, I think sensational players and great commentators is what attracts non-player spectators. I do agree that having more players would also help us gain more spectators. But, I think we differ on what we should do to gain more players. I think having more tournaments especially ones limited to intermediate and beginner level players throughout the year would help us gain more players. Making a once a year event intended for the advanced players even bigger by allowing anyone to play just doesn't seem like the right approach to me anymore. It's overloading one tournament with the needs of too many players. Having more tournaments intended for different skill levels would be the right way to go. It would give many more people a taste of what it's like to win a tournament. I really appreciate the help you have offered. You already do so much for the Arimaa community I can't see myself asking you to do more. I know you are volunteering, but when the tournament started you were doubtful about how much you would be able to participate this year with your courses going on. I would hate to see you get overloaded and withdraw from Arimaa. What I would really like to see is more people stepping forward to organize more events, as well as volunteering to be tournament coordinators and tournament directors. I really think that because Joel organized the AWL and AOF this past summer it helped to increase the number or active players and thus the participation in the WC this year. Part of the difficulty of having a tournament with many games is making sure every game gets scheduled as the players desired, making sure each game gets played without any technical problems and being available to resolve problems if something should go wrong. The more games there are the higher the chances of things going wrong. With 16 games I am already hitting my practical limits. Since the ability to run the WC is now all web based, I am open to allowing someone else be the tournament coordinator. Perhaps they would be willing to handle tournaments with larger number of players and longer schedules.
Also I think it is important to mention that this format is not an invitational or closed to anyone. In fact it is more open than what we currently have. The preliminary is open to everyone and there is no registration fee to participate in the preliminary; just play whenever you want and work on improving your WHR rating; you have all year long to be able to do this; and you can play as many or few games as necessary. Currently there is a fee to participate in the preliminary. Way more people could participate in this format than what any preliminary tournament would be able to handle. This preliminary doesn't put a burden on anyone unlike actually conducting a preliminary tournament. Only the top 16 players from the preliminary move on to the finals. This may sound limiting or invitational, but compare it to our current situation of only the top 8 from a tournament moving on to the finals. Much less chances that we lose the best player going from preliminary to finals. Also even if you are #20 on the rating scale you might still have a good chance of going to the WC finals since some of the players who qualified may not be able to. Then the finals is a triple elimination instead of double elimination; further improving the chances that the best player will win the tournament. This format is certainly not excluding someone who wants to participate from having an equal chance at winning the WC. I guess what it doesn't provide is the physiological feeling of having participated in the WC if you tried your best to improve your rating, but did not make it to the finals.
Eric, linking the Open Classic to the WC is what we already have. In fact it is 100% linked right now. We are trying to get away from having any other tournament directly linked to the WC (even partially) because it will require that tournament to be conducted to the standard of the WC and even then there will not be an end to discussions of what that tournament format should be, how fair it is, how many players from that tournament should get slots in the WC, etc. I agree even the WHR ratings are not the best measure of ones ability to play well in the WC, but I would guess that no preliminary tournament will be able to do better than that anyways. We still have room to make the rating system an even better measure in the future by separating postal and live games as well as using event games. I think we should at least try out the simple and straightforward method of using ratings.
aaaa, I think allocating the bye before trying to find the best pairing was simplifying the algorithm. Wouldn't swapping that make it take longer to run? Also can you discuss the advantages of doing the seeding this way.
|
| « Last Edit: Jan 20th, 2011, 12:16pm by omar » |
IP Logged |
|
|
|
Sconibulus
Forum Guru
Arimaa player #4633
Gender:
Posts: 116
|
|
Re: 2012 World Championship format
« Reply #8 on: Jan 20th, 2011, 12:56pm » |
Quote Modify
|
Omar, the main problem I have with using ratings, and only ratings to decide who can play in the tournament is that it will strongly disincentivize "bubble" players from playing games they think they have a good chance of losing, especially against other "bubble" players. Do we really want to do that? Even if we used WHRE, it would give a bit of a bonus to sandbagging gameroom ratings by 1-200 points and signing up for a lot of Autopostal games to get a large number of players slightly worse than you as event game opponents.
I'm not saying that ratings can't be used at all, it does make sense to pull the top rated players out so they don't have to fight their way in every year, but a reasonable number of players should have the opportunity to play in as well, maybe not from one large tournament if that's too intensive, but perhaps podium finishers from some number of less-intensive tournaments throughout the year.
|
|
IP Logged |
|
|
|
Nombril
Forum Guru
Arimaa player #4509
Gender:
Posts: 292
|
|
Re: 2012 World Championship format
« Reply #9 on: Jan 20th, 2011, 2:24pm » |
Quote Modify
|
Yes, I understand we currently have a 100% link between the Open Classic and WC. I suggested the hybrid to get the best of both worlds. There would be the option to skip straight to the WC for the top players that aren't interested in the longer Open Classic. Then the Open Classic would help deterimine the final slots for those "bubble" players that probably shouldn't be sorted strictly by a rating system.
You are concerned about never ending discussions for how many slots to allocate between 'ratings' and 'tournament' players, but I could see similar arguments about ratings. If we keep Postal games out, should we also keep out 15s games? What about 30s games? Do we have enough event games to use WHRE? Etc.
By using two qualifying mechanisms, it seems it would be easier to ignore the inherent disadvantages that will always be present in each method, and it becomes far less important for either to be 'perfect'.
You also talk about improving your ratings through out the year as the new "preliminary". I'm sorry, but I just can't imagine that playing more casual games will generate the same interest and excitement as compared to the large Open Classic.
I also just noticed that aaaa's suggested format is open to everyone. Were you still planning on limiting the WC to 16 players?
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2012 World Championship format
« Reply #10 on: Jan 20th, 2011, 8:28pm » |
Quote Modify
|
on Jan 20th, 2011, 12:15pm, omar wrote:| Also I think it is important to mention that this format is not an invitational or closed to anyone. In fact it is more open than what we currently have. The preliminary is open to everyone and there is no registration fee to participate in the preliminary; just play whenever you want and work on improving your WHR rating; you have all year long to be able to do this; and you can play as many or few games as necessary. |
|
You are portraying the rating system as an ongoing tournament, but this is a very weak analogy. In a tournament the players do not get to select their opponents and their time controls. In a tournament no one can protect his standing by refusing to play. Heck, if we could do that, I would take a pass this round instead of playing hanzack.
The WHR ratings, although they don't include bots, are extremely open to abuse and manipulation. It would be trivial to cheat outright. For example, just before the World Championship begins, the #18 player could realize he needs a few more points, and persuade his friends to lose to him on purpose. Or he could create sockpuppet accounts, get those up to a high level, and then get his friends to use those sockpuppet accounts to lose to his main account. If you think that the current tournament format is too much work, think about how much work it would be to try to police everyone's rating to prevent ratings abuse.
But even if constant policing of the ratings managed to prevent outright cheating, a general focus on keeping one's rating as high as possible would be pernicious to the Arimaa community. You imply that people who are trying to get a high rating will play lots of games, but this is simply not true; the key to an inflated rating is avoiding bad pairings. Nombril has already pointed out the time control issue; I too have a far better record postally than in live games, thus to keep my rating high I should play only postally. Furthermore, I have a relatively poor record against PMertens, so I should avoid playing him. Adanac has a poor record against me, so he should avoid playing me, etc.
There will probably be some people who don't try to inflate their rating via opponent selection and time-control selection. They will play anyone at any speed, for fun, and not care that it lowers their rating. But if I care about my own rating, I must avoid playing those happy-go-lucky people at all costs. On average, playing against someone who is underrated will drag my own rating down. I will only gain on average by playing against overrated opponents.
People who are drastically underrated could become pariahs. For example, why would anyone in his right mind play against hanzack? If the top sixteen players all needed to preserve their high ratings in order to qualify for the World Championship, then logically all sixteen should refuse to play against him. Indeed, everyone in the top sixteen would have a disincentive to play against anyone outside the top sixteen. In what sense is a tournament "open" if the top players can implicitly band together to keep another player out by avoiding him?
Setting aside the issue of cheating, it would reduce the number of rated HvH games if everyone played to maximize his WHR rating. There is no pairing that will, on average, help both player increase their ratings. There is no bonus for playing many games, but there is a bonus for avoiding disadvantageous pairings. Always one player has more to lose than to gain, on average. It takes two to tango, but one of the two will always (correctly) see that it would be better for his rating if he refused the game.
We have discussed ratings at length before, Omar, and I'll bet that I can find a quote from you agreeing to this fact: The only way to ensure accurate ratings is to prevent people from choosing their opponents. In a tournament, you don't get to choose your opponents, but in every other circumstance you do get to choose. This is what makes ratings a very poor substitute for a preliminary tournament.
Quote:| I think having more tournaments especially ones limited to intermediate and beginner level players throughout the year would help us gain more players. Making a once a year event intended for the advanced players even bigger by allowing anyone to play just doesn't seem like the right approach to me anymore. It's overloading one tournament with the needs of too many players. Having more tournaments intended for different skill levels would be the right way to go. It would give many more people a taste of what it's like to win a tournament. |
|
I understand the danger in overloading a tournament with too many goals, because one goal can suffer when trying to meet another. But the needs of the top players do not suffer from allowing lots of lower-rated players to register. Well, OK, I admit that in 2008 the large field slightly undermined the tournament atmosphere, but that was because you allowed kids to register without paying an entry fee to show commitment. A non-refundable entry fee that contributes to the prize money solves that problem. This year's tournament atmosphere has been the best ever despite the large field.
The fact that so many people have signed up in this year has swelled the prize fund that I expect to win part of. Katie is my witness: at the beginning of this year's tournament I said that I would have worked harder to prepare if I had known the prize fund would be so large. One extra round of tournament time is a small disincentive compared to the extra incentive of double prize money.
Like you, I think it would be great to have more tournaments offered during the whole year. But who is going to come to those events? The Arimaa World Championship format is obviously motivating people to participate. We had thirteen players under 1800 who decided to register this year. Just as an experiment, I want you to announce an under-1800 Arimaa tournament with a $10 entry fee some time before the next World Championship and see whether you get more or fewer than thirteen people to sign up. By your theory, a beginner tournament will better meet the needs of beginners, and all the prospective players will have a chance of winning, so more people should flock to that tournament.
My theory is that people want to play in the World Championship because it is the World Championship. Do you know how many players spend $10,000 to enter the World Series of Poker, even though they have no realistic chance of winning? They drop that $10,000 precisely because it is the World Series of Poker and not some lesser event. I'm trying to imagine the WSOP organizers saying: "This tournament has just gotten too big. We're not going to take your money and let you play unless you are in the top hundred in the world." No, the WSOP is way too savvy for that. Instead they have increased the length of the event year by year, and split it out between multiple venues, and tackled every other logistical challenge because they realize it is in their best interest to have their event get bigger every year.
The thirty-three entrants to this year's Arimaa World Championship was a wonderful occurrence. Run with it. Figure out what we need to do to keep the momentum and make it feasible to hold an even bigger event next year, should we be so fortunate. Yes, let's offer other events all year to try to cater to the needs of all players, but if players by their own volition think that the World Championship serves their needs, certainly don't bar the door to them.
|
| « Last Edit: Jan 20th, 2011, 8:41pm by Fritzlein » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2012 World Championship format
« Reply #11 on: Jan 20th, 2011, 10:58pm » |
Quote Modify
|
on Jan 20th, 2011, 12:15pm, omar wrote:| With 16 games I am already hitting my practical limits. Since the ability to run the WC is now all web based, I am open to allowing someone else be the tournament coordinator. Perhaps they would be willing to handle tournaments with larger number of players and longer schedules. |
|
It think it is a great idea for you to offload as much of the work load as you can. If the World Championship can now be scheduled by someone else via your tournament tool, then someone else should do it. If the scheduling e-mails can be sent out by someone else, then they should be. I would be willing to take on that role myself, and I don't think I am the only one who would be willing. (Maybe it would best be someone who is not in contention for the title, to avoid conflict of interest, but even so better me than you.)
Of course your work load during the World Championship would remain high. There are always an infinite number of things that only you can do, such as restart a game that was interrupted by server problems. But once you have developed a tool to automate a task, that task should pass on to the community.
|
|
IP Logged |
|
|
|
ginrunner
Forum Guru
Arimaa player #5449
Gender:
Posts: 163
|
|
Re: 2012 World Championship format
« Reply #12 on: Jan 20th, 2011, 11:40pm » |
Quote Modify
|
I can almost guarantee that a beginner only tournament with an entry fee would flop. having many free and fun tournaments during the year would be a great idea and I am also positive that players would be willing to run these small tournaments. For example, while my tag team seemed to be a flop if it had been met with more approval I would have been willing to run the entire thing. Other members have written in the wiki and done things like that, those players would have no problems running a small event on their own. The allure to the small tournaments is "I won the so and so tournament." That would boost the enjoyment of the community more than having multiple entry fee events I think.
|
|
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: 2012 World Championship format
« Reply #13 on: Jan 21st, 2011, 11:56am » |
Quote Modify
|
on Jan 20th, 2011, 12:15pm, omar wrote:| aaaa, I think allocating the bye before trying to find the best pairing was simplifying the algorithm. Wouldn't swapping that make it take longer to run? |
|
The matching algorithm simply expects numbers signifying the desirability of each pairing, while the assigning of byes can be determined by the use of a virtual player. The hierarchy of priorities with respect to the pairing rules can be respected by the use of appropriate weights. The fact that it's only the pairing rule with the lowest priority that allows for a considerable variety of scores, should mean that the weights that are calculated by this approach should remain manageable and that there should be little risk of overflow.
Janzert has already succeeded in creating an implementation and I should follow shortly with one following above draft.
Quote:| Also can you discuss the advantages of doing the seeding this way. |
|
It was Fritzlein who pointed out that having a good seed that remains fixed for the entire tournament would be an enduring advantage, especially as the number of lives increase, since that means more opportunities for the lowest pairing rule to make a difference. Such a thing may be currently justifiable with the several rounds of the Swiss preliminary determining the seed, but not when having a triple elimination tournament from the start.
I would also like to point out that, given your desire of not having too many participants, you should actually welcome such a cutthroat tournament format, as it should discourage too weak players from signing up and that without throwing up any artificial barriers.
|
| « Last Edit: Jan 21st, 2011, 11:56am by aaaa » |
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2012 World Championship format
« Reply #14 on: Jan 21st, 2011, 5:37pm » |
Quote Modify
|
Karl, I agree that even WHR can be abused and yes I've been saying we eventually want to move to WHRE and possibly even split the ratings into postal, regular and fast. But, I just don't think that somebody who doesn't already have a high rating will abuse the system so much just to be able to get into the WC. They are just going to end up paying a registration fee with have much chance to win anything. Even if there is abuse, it won't effect the top players and I'm sure the best player would not be lost by this filter.
In Poker I would think that people who are putting up such money really do think they have a chance to win and are not doing it just to say they played in the World Series. With Poker having a bit of luck everyone does have more equal chances than in a pure skill game like Arimaa. Also they keep expanding because they are doing it for profit. We can't compare that to our situation.
Quote:| Figure out what we need to do to keep the momentum and make it feasible to hold an even bigger event next year, should we be so fortunate. |
|
We still have a lot of time before the 2012 WC, so maybe we can figure something out before that.
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print