Author |
Topic: 2007 World Championship Format (Read 5290 times) |
|
PMertens
Forum Guru
Arimaa player #692
Gender: 
Posts: 437
|
 |
IRe: 2007 World Championship Format
« Reply #30 on: Sep 22nd, 2006, 10:42am » |
 Quote  Modify
|
hmm ... in my humble eyes the primary goal should be fun ... and nothing else ... 
There is just very little point in participating as worst rated of the crowd if you will always be paired against the best rated available ...
Limiting the tourney to the top 4-8 does not make it much fun ... even if the rest of the crowd does not really stand a chance ... no matter how you pair them.
Betting is not really fun if a top-dog is fighting a mouse ... neither is watching.
Sorry, but watching 2 beginners to battle it out while blundering all the time does sound more interesting to me than watching Fritzlein punish every mistake on the spot and finishing up his opponent ... slowly but unavoidable ...
Promoting the best-rated does not make it more accurate anyway ... as can be seen in our recent fun tournament.
|
|
 IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2007 World Championship Format
« Reply #31 on: Sep 22nd, 2006, 4:08pm » |
Quote Modify
|
on Sep 21st, 2006, 8:56pm, Janzert wrote:| Also as a secondary goal I think the rules for any tournament should be as transparent as possible. |
|
Thanks for bringing up the issue of transparency, Janzert. I agree with you in thinking the pairing system should be transparent. It is a disadvantage if the optimal pairing can't be determined by a casual observer who doesn't write a program to exhaustively check all possibilities.
However, floating elimination is transparent in one very important way: if you think you have a better pairing than the official one, it is simple to check whether or not it is actually better. Anyone who thinks they got an unfair deal is free to propose an alternative pairing, and we can easily show how the alternative is worse according to clearly-defined priorities. We may realize the priorities are sub-optimally ordered, as last year's tournament showed, but at least (even then) they are clearly defined by whatever Omar writes into the rules.
Quote:| I personally do not think the primary goal in determining the world championship tournament format should be to make it "fair". Rather the primary goal should be to make it accurate, i.e. having the greatest likelyhood that the truly best player in the world wins. |
|
I've been trying to figure out why "fairness" and "accuracy" should be pitted against each other. Why can't we be fair and accurate both? Indeed, on general principles, it would seem that the most accurate system is also the fairest.
The bone of contention is the extent to which ratings (a.k.a. pre-tournament performance) should be taken into consideration. If you believe in accuracy above all, there is definitely some information contained in ratings. You can infer reasonably well from a player's rating the likelihood that he is in fact the best player in the world. Last summer Omar demonstrated that (depending on our assumptions about ratings) triple-elimination is not as accurate in crowning a World Champion as giving the title to the highest-rated player with no tournament at all.
I think Omar was right. If accuracy is our only objective in crowning a World Champion, a tournament might be the wrong way to go. I could claim that I have already proven I am the best player in the world at present, and playing a tournament for the World Championship only introduces the possibility of error.
Seeding a tournament by ratings falls somewhere between the extremes of ignoring ratings totally on the one hand, and having ratings decide everything on the other hand. There is a lot of gray area in the middle. Floating double-elimination favors high seeds less than fixed-bracket double-elimination. Swiss pairing (winners play winners, losers play losers) favors high seeds less than straight folding pairing every round. Giving byes based on W/L record favors high seeds less than giving byes based on seed. Indeed, I think I've supported just about every way to de-emphasize ratings apart from random pairing, which goes a bit too far for me.
Being World Champion, in my opinion, should recognize a specific over-the-board achievement against everyone else who chooses to compete for the same prize. Fairness in this context is allowing everyone to compete in that event on reasonable terms. I think it is reasonable to have the "regular season" be part of the total achievement, in the form of seeding, but to me the focus should remain as much as possible on performance in the actual tournament.
That's my wishy-washy middle-ground position, and I'm sticking to it!
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: IRe: 2007 World Championship Format
« Reply #32 on: Sep 22nd, 2006, 5:27pm » |
Quote Modify
|
on Sep 22nd, 2006, 10:42am, PMertens wrote:| hmm ... in my humble eyes the primary goal should be fun ... and nothing else ... |
|
Fun is good, but a bit hard to define. If we're only out for fun, for example, should we really keep track of winners and losers? Let's just play lots of Arimaa! Oh, wait, we do that all year already just for fun...
Quote:| There is just very little point in participating as worst rated of the crowd if you will always be paired against the best rated available ... |
|
What do you mean there is little point for the worst-rated to participate? Do you think that nobody would want to sign up unless they had, say, at least a 1% chance to win all the marbles? I respectfully disagree. The worst-rated player plays for fun, just like you and I do. Having fun is precisely the point. (And for the record, the worst-rated player won a game last year before being eliminated.)
If a 1500-rated player doesn't want to get clobbered, that's fine; nobody is forcing him to sign up for the World Championship. He can play all year against opponents of his own choosing, if that is more fun. He can play in the Postal Tournament and get paired only against players on his end of the bracket. I like that pairing scheme. But it is also fun (for a somewhat different reason) to take a one-in-a-million shot at winning the biggest prize in your sport. Fun comes in all different colors, and trying to be best in the world is a valuable shade of fun too.
Quote:| Betting is not really fun if a top-dog is fighting a mouse ... neither is watching. |
|
OK, you make it clear what is fun in your book, namely games between roughly equal players. I agree that level games are the most fun to watch, and the most fun to bet on. I think it would fit this definition of fun best if the top four entrants played double round robin for the "A-level World Championship", the next four played for the "B-level World Championship", the next four for the "C-level World Championship", etc.
Quote:| Limiting the tourney to the top 4-8 does not make it much fun |
|
Oh, wait, you also don't want to limit the field, because that's not fun. I entirely agree. I think having an open tournament is one of the funnest things about the Arimaa World Championship. I love the thought that everyone starts six wins from becoming World Champion, and if they win those six games, nothing else matters. Each player controls his own destiny, regardless of everything that has happened up to that point.
But there's an inherent contradiction between wanting an open tournament, and not wanting any mismatched games. You can't avoid mismatches by random first-round pairing; sooner or later they have to happen. To take the idea to the extreme, if you intentionally paired #1 vs. #2, #3 vs. #4, #5 vs. #6, etc., you would get matches as even as possible for the first round, but the mismatches would still have to occur in later rounds. For round two, the winner of #23 vs. #24 has to play someone... David has to take on Goliath at some point if they are competing for the same prize. In order to preserve the fun of an open tournament, you simply must embrace the David vs. Goliath matches.
Playing friendly Arimaa games for fun is a great idea. I devote a considerable percentage of my life to it. However, I believe that it is also good and fun to have one event per year be an everybody-plays fight to be King of the Hill. It is (and should be) an integral part of The World Championship tournament that everyone is trying to play the best Arimaa, not just better than their opponent du jour, but better than everyone else in the world.
Given a tournament where everyone is fighting to be best, and given that mismatches must occur later if they don't occur sooner, I am definitely in favor of having them sooner. If the first round randomly pairs #1 vs. #2 and also #3 vs. #20, the loser of the top game has a legitimate complaint. Player #3 will likely score a win, not primarily because he's a great player, but because he got lucky. Player #2 will likely score a loss and be half-dead in the tournament, not primarily because he's a poor player, but because he got unlucky. With two lives left rather than one, Player #3 will have a much better shot to win it all than Player #2, due to pure chance.
You could turn my argument around on me to say that #2 will probably have to play #1 eventually to win, which is true, but it is still a pure-luck benefit to #3 that one of the players above him must lose while #3 advances easily, and perhaps backs into the championship game.
A perfectly "fair" way to decide the tournament would be to have a lottery among all players, in which each has an equal chance to become World Champion. This is silly, though, because then being World Champion has nothing to do with playing well, and everything to do with luck. On this point I'm not in the middle ground, rather I'm towards the opposite extreme. Being World Champion should have everything to do with playing well, and nothing to do with luck, to the extent that we can make it so.
|
| « Last Edit: Sep 22nd, 2006, 5:34pm by Fritzlein » |
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2007 World Championship Format
« Reply #33 on: Sep 22nd, 2006, 7:24pm » |
Quote Modify
|
I think next summer we should start the Arimaa Open Classic tournament. The puropose of the tournament will just be to have fun and see who comes out on top and it will not be designed to select the best player. The tournament will be single elimination and pair players who are close in rating. The time control will about 30 sec per move.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2007 World Championship Format
« Reply #34 on: Sep 22nd, 2006, 7:32pm » |
Quote Modify
|
I got Paul's program integrated into the tournament simulator. The progam runs a bit slow with 16 players and sometimes even crashes so I have not been able to run it through a lot of trials yet.
Here's a result of one trial.
##### Player List #####
player true measured
p1 1988 1977
p2 1968 2000
p3 1944 1918
p4 1904 1892
p5 1902 1931
p6 1859 1823
p7 1817 1805
p8 1795 1845
p9 1735 1762
p10 1671 1649
p11 1665 1683
p12 1654 1663
p13 1654 1632
p14 1649 1659
p15 1619 1574
p16 1503 1536
#######################
----- From Tournament Format -----
* Round 1
pick p1 p15
pick p10 p3
pick p11 p6
pick p12 p8
pick p13 p5
pick p14 p4
pick p16 p2
pick p7 p9
----------------------------------
===== From Game Simulator =====
* Round 1
pick p1 p15 winner p1
pick p10 p3 winner p3
pick p11 p6 winner p6
pick p12 p8 winner p8
pick p13 p5 winner p5
pick p14 p4 winner p14
pick p16 p2 winner p2
pick p7 p9 winner p9
==================%3
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2007 World Championship Format
« Reply #35 on: Sep 22nd, 2006, 10:14pm » |
Quote Modify
|
on Sep 22nd, 2006, 7:32pm, omar wrote:I got Paul's program integrated into the tournament simulator. The program runs a bit slow with 16 players and sometimes even crashes so I have not been able to run it through a lot of trials yet.
Here's a result of one trial. |
|
Awesome. I'm glad you could get it hooked up. It's good to see that it at least gets folding pairing right in the first round.
It's not too surprising it runs slowly on 16 players, since there are 2,027,025 possible pairings the first round. Indeed, if we have 24 players as I expect, it probably won't ever finish running, given the 316,234,143,225 possible pairings in the first round. Fortunately the first two rounds of pairings can be easily done by hand, after which there are eliminations which reduce the number of players.
Would you be willing to post a complete tournament or two on, say, 11 players or 13 players, so we can see the pairing algorithm in action when lots of byes are flying around?
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2007 World Championship Format
« Reply #36 on: Sep 22nd, 2006, 10:19pm » |
Quote Modify
|
on Sep 22nd, 2006, 7:24pm, omar wrote:| I think next summer we should start the Arimaa Open Classic tournament. |
|
I think there is room and interest in the summer for at least one extra tournament, and probably more. There could be a fast or blitz tourney. There could be a partial round-robin, paired like the postal tourney, but with live games. I'd even like to see a revival of the Ryder Cup concept, maybe the EU vs. North America vs. the Rest of the World this time.
I hope your new job and new house work out fantastically, so that by next summer you'll have tons of free time on your hands to devote to Arimaa. 
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender:
Posts: 1413
|
|
Re: 2007 World Championship Format
« Reply #37 on: Sep 23rd, 2006, 1:55am » |
Quote Modify
|
on Sep 22nd, 2006, 10:19pm, Fritzlein wrote:| EU vs. North America vs. the Rest of the World |
|
Ok thorin and blue22... let's start warming up now!
|
|
IP Logged |
|
|
|
PMertens
Forum Guru
Arimaa player #692
Gender:
Posts: 437
|
|
Re: 2007 World Championship Format
« Reply #39 on: Sep 23rd, 2006, 9:09am » |
Quote Modify
|
Forgive me for not answering in the same length as you posted Fritzl
A lottery would be neither fair nor fun ...
I do not want to intentionally make games as level as possible ... I just do not want to intentionally make them as unlevel as possible
Anyway ... since few people else seem to be much interested in this and I personally will have no real advantage one way or the other I will stop continuing to explain my way of thought
I will play you in the WC and I will beat you ... 
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2007 World Championship Format
« Reply #40 on: Sep 23rd, 2006, 5:13pm » |
Quote Modify
|
on Sep 23rd, 2006, 9:09am, PMertens wrote:
I will play you in the WC and I will beat you ... |
|
Now, that's the Arimaa spirit I would love watching this game.
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: 2007 World Championship Format
« Reply #41 on: Sep 23rd, 2006, 7:54pm » |
Quote Modify
|
Pmertens wrote,
Quote:| hmm ... in my humble eyes the primary goal should be fun ... and nothing else ... |
|
And I think you are precisely correct, for the rest of the tournaments during the year. I was actually going to say something like that in my original post but obviously forgot. I think the WC is, and should be, a little different from the rest of the tournaments. I think if we show that we take the WC a little more seriously it will help the rest of the world to take Arimaa a little more seriously.
But remember in all that I say, I'm neither a player nor have helped to sponser these tournaments. So my words are worth basically what I'm charging you for them, i.e. hovering somewhere around zero, hopefully at least slightly to the positive side thereof.
Fritzlein wrote,
Quote:| Indeed, on general principles, it would seem that the most accurate system is also the fairest. |
|
If no information is allowed to leak in from outside the tournament then I believe that is correct, the most accurate format would also be a fair format. By fair I mean gives a level playing field for every participant. If you are able to use information external to the tournament, then depending on how much and in what ways you trust that information to be accurate the format should bias itself to that information.
In this case of course the external information that has traditionally been used is the ratings of the players. If you were to believe that the ratings give a true ranking of all players at the time of the WC, then yes the most accurate format would be to simply hand the title to the highest ranked player. If you think the ratings are completely random then a format that ignores them make sense.
I personally think at any given moment the rating do a fairly good job at ordering, but are questionable in absolute ranking. Also the ratings carry an assumption of transitivity. I think this is a fairly safe simplifying assumption for HvH, but is much more questionable when it comes to Bots.
Janzert
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2007 World Championship Format
« Reply #42 on: Sep 24th, 2006, 4:35am » |
Quote Modify
|
on Sep 23rd, 2006, 8:08am, omar wrote:| Use the following links to view the trial runs. |
|
Omar, thank you so much for posting full trial runs. I played through them all, attempting to do the pairings by hand. My conclusion is that 16-player or 17-player FDE can be accurately paired by hand throughout. You have to be very attentive and double-check everything, but it isn't that hard to find the optimal pairing. The early rounds have few constraints, and the late rounds have few players.
Triple elimination, on the other hand, gets too complicated somewhere in the middle for me to do accurately by hand. Around round five to seven of the triple-elimination tournaments, there were enough possible repeat matchups to confuse me, but also too many possible pairings for me to be sure I had the optimal one.
I agree that the program produced the best pairings in every round of all eight tournaments except in two cases:
http://arimaa.com/arimaa/tourn/compare/fte/paul/tr1
In round seven of this tournament there is exactly one pairing of the five players that avoids repeat pairings, namely 5-bye, 1 vs 2, 3 vs 4. Unfortunately, the program instead accepted a repeat pairing in order to give the bye to the highest-ranked eligible player. I strongly feel that avoiding repeat pairings should be in all ways more important than pre-tournament ratings.
http://arimaa.com/arimaa/tourn/compare/fte/paul/tr4
In round nine of this tournament, the same issues arises. The three remaining players, 1, 4, and 10, have each had one bye. The pairings 1 vs 4 and 1 vs 10 have each occurred once so far, while the pairing 4 vs 10 has occurred twice. The one totally wrong pairing is to give 1 the bye and pair 4 vs 10 for the third time. I can see using rating to break a tie, i.e. to give 4 the bye rather than 10 because 4 has the higher rating, but player 1's pre-tournament rating can't justify forcing the other two players into a third match. That's just what happened with Bomb-Clueless-Aamira in last year's CC, and why we needed a new algorithm.
The formulas for weighting the various pairings are in the last couple of functions of Pogonyshev's code. After I get some sleep, I'll suggest some alternative to correct what I perceive as a bug.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2007 World Championship Format
« Reply #43 on: Sep 24th, 2006, 11:09am » |
Quote Modify
|
Thanks for checking this out Karl. I guess we will need to experiment with different formula's for scoring tournaments. I have not had a chance to look at that part of the code yet. Im still trying to figure out why it crashes sometimes; or produces the wrong number of pairings. I've found a couple of input files that cause this.
case1
-------------------
elimination 2
16
p1 1935
p2 2009
p3 1875
p4 1886
p5 1904
p6 1894
p7 1829
p8 1886
p9 1744
p10 1722
p11 1730
p12 1721
p13 1650
p14 1632
p15 1544
p16 1510
3
p1 p15 p1
p10 p8 p10
p11 p3 p3
p12 p4 p4
p13 p6 p6
p14 p5 p5
p16 p2 p2
p7 p9 p7
p1 p7 p1
p10 p2 p2
p11 p14 p11
p12 p13 p12
p15 p9 p9
p16 p8 p8
p3 p5 p3
p4 p6 p4
p1 p4 p1
p10 p6 p10
p11 p7 p7
p12 p5 p5
p2 p3 p2
p8 p9 p8
-------------------
case2
-------------------
elimination 2
16
p1 1960
p2 1941
p3 1888
p4 1960
p5 1882
p6 1873
p7 1784
p8 1787
p9 1761
p10 1694
p11 1694
p12 1665
p13 1743
p14 1687
p15 1607
p16 1563
1
p1 p16 p1
p10 p5 p5
p11 p6 p6
p12 p2 p2
p13 p8 p13
p14 p3 p3
p15 p4 p4
p7 p9 p7
p10 p1 p1
p2 p6 p2
p3 p5 p3
---------------------
To try it out, download Pauls pairing program:
http://download.gna.org/quarry/tournament.cpp
Complie like:
g++ tournament.cpp -o tournament
Save the above input cases to files: case1 and case2.
Run the program on the input files:
tournament < case1
or
tournament < case2
Sometimes it works, but sometimes it does not, so you have to run it a few times.
I've narrowed it down to this line:
return &best_nodes[rand () % best_nodes.size ()];
and found that changing it to:
return &best_nodes[ best_nodes.size () - 1];
fixes the problem. But I haven't figured out yet why bad pairings were getting into the list to begin with. Has anyone else look had a chance to look at Pauls program.
|
|
IP Logged |
|
|
|
seanick
Forum Guru
SeaNICK
Gender:
Posts: 97
|
|
Re: 2007 World Championship Format
« Reply #44 on: Sep 24th, 2006, 12:06pm » |
Quote Modify
|
I haven't looked at the program, but for the last line, don't you mean, instead of this:
return &best_nodes[ best_nodes.size () - 1];
you actually meant this, right?
return &best_nodes[rand() % ( best_nodes.size () - 1)];
(because leaving out the random factor would mean it was always the last player in the best_nodes arrray that was picked)
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print