Author |
Topic: 2008 Postal Tournament Format (Read 3054 times) |
|
Fritzlein
Forum Guru
Arimaa player #706
Gender: 
Posts: 5928
|
 |
2008 Postal Tournament Format
« on: Oct 5th, 2007, 2:13pm » |
 Quote  Modify
|
on Oct 5th, 2007, 12:21pm, mistre wrote:| On another note, I am also interested in participating in the next Postal Tournament. I looked at the results from last year and I was disappointed that there is no ranking or winner. Is there a way to make the Postal Tournament more like the World Championship? |
|
The key problem in making a championship out of postal games is that the length of the games prevents having enough rounds in the same year. At one day per player per move, a sixty-move game takes 120 days, and that's without reserve time and with some kind of nasty cutoff for longer games. Even a single-elimination tournament with five rounds would span multiple years.
The solution was to pair all the games at the beginning of the tournament, and have the games be simultaneous rather than consecutive. But that introduced another problem. The fair way to have simultaneous games pick a champion is to play a full round-robin, but a full round-robin was more games at once than most people wanted to play. So Omar introduced a ten-game cap if more than eleven players signed up, with the greatest ratings mismatches being discarded.
As people pointed out at the time, that made the postal tournament a bit un-championship-like in 2005 and 2006. Someone in the bottom of the draw with an 8-2 record had a great tournament, true, but that wasn't as impressive a result as going 7-3 at the top of the draw. It was just an accident that we had exactly one 10-0 player each year, namely me in 2005 and 99of9 in 2006. But Omar actually didn't mind that the postal tournament was poor at ordering players by skill, and in fact was happy that we didn't need a bunch of the mis-matched slaughter games that are necessary in the World Championship.
So in 2007, he made the postal tournament even less like a championship and more just-for-fun by allowing people to specify how many simultaneous postal games they thought they could handle. Game-hungry players got 14 or 15 games against the whole range of opponents, while time-constrained people chose to play 3 to 5 games against only opponents near to them in rating. I was somewhere in the middle with 8 games.
I personally don't mind the anti-championship feel of the tournament. I like that my games were interesting, and I gained some new insights into Arimaa theory, so I'm happy. And as it turned out, we once again had a clear champion anyway, with chessandgo going undefeated in twelve games. The fact that camelback also went undefeated in four games doesn't undermine anyone's clarity about who the best postal player was. Indeed, you can't even really say camelback didn't have a chance to be champion: he did have a chance by signing up for twelve games and beating all comers.
All in all, I was quite content with the format for 2007. But I really wouldn't mind going back to the 10-games-each style of 2005. And of course, we can always make changes for next year. Suggestions are welcome, and the sooner they are made, the greater chance they have of getting enough support to be adopted.
|
|
 IP Logged |
|
|
|
camelback
Forum Guru
Arimaa perl monger
Gender:
Posts: 144
|
|
Re: 2008 Postal Tournament Format
« Reply #1 on: Oct 6th, 2007, 5:43pm » |
Quote Modify
|
on Oct 5th, 2007, 2:13pm, Fritzlein wrote:
The fact that camelback also went undefeated in four games doesn't undermine anyone's clarity about who the best postal player was. Indeed, you can't even really say camelback didn't have a chance to be champion: he did have a chance by signing up for twelve games and beating all comers.
|
|
Aha.. The fact is chessandgo is World Champion and I'm a noob, joined on Jan 2007 and postal games started Feb 25. I didn't have any clue about the game by then and even now..
My ratings were low and I covered a bit of ground while the games were in progress. I was just happy-go-lucky to have a finish like this. This is attributed to TheMob discussions that taught me how to analyze the game.
I like the current format. It gives freedom pick your number of games.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2008 Postal Tournament Format
« Reply #2 on: Oct 7th, 2007, 7:11am » |
Quote Modify
|
I would not have been able to play 10 games this year. I think being able to select the number of games allowed for more participation.
|
|
IP Logged |
|
|
|
mistre
Forum Guru
Gender:
Posts: 553
|
|
Re: 2008 Postal Tournament Format
« Reply #3 on: Oct 9th, 2007, 9:01am » |
Quote Modify
|
OK - how about we keep the current format where you can choose as many games as you want to play, but also have a cap of say 10 or 12 games. Whoever wants to play fewer can. And then at the end - have the top 4 players (how you would determine top 4 is another question), face off in a single elimination semifinal and then a final.
This would keep the format relatively the same with adding an element at the end to have a champion.
I am sure Karl or others would have ideas on how to determine the top 4.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2008 Postal Tournament Format
« Reply #4 on: Oct 9th, 2007, 1:46pm » |
Quote Modify
|
It's going to take a bit to sell me on the value of a two-round, four-player playoff, versus the cost of at least an additional 240 days, spilling over into the next year when the players might rather be starting on the next year's postal tournament. Two more rounds doesn't necessarily increase clarity much, and it loses the nice feature that the tournament wraps up soon enough that it doesn't overlap other tournaments.
How would a two-round playoff have worked this year? Let's take the top four finishers by wins-losses, with a maximum of twelve games played: chessandgo, me, Ron Weasley, and camelback. Let's say I beat Ron Weasley while chessandgo beats camelback, and then I beat chessandgo in the finals. Are we happy because we have a clear champion? No! In my two head-to-head games with chessandgo, we each won one. Overall I would be 8-1 he would be 12-1. By what grounds am I champion, other than getting lucky late rather than early? We would need a playoff playoff.
Or what if Ron Weasley won the playoff? He would be Postal Champion, but only 1-1 against me, only 1-1 against chessandgo, and 11-3 overall compared to my 7-2 and chessandgo's 13-1. The point is that the playoff system, although it unambiguously selects a winner, doesn't even necessarily select the person who played best within the tournament.
On a related note, I'm sure you can guess how I feel about all the clamour for a playoff system to determine a national champion of college football. If the NCAA caves in and introduces an eight-team playoff, then it won't be long before some 8-3 team upsets an 11-0 team in the final, so the "National Champion" is clearly neither the best team in the country, nor the team that had the best year. I hope someone feels silly then. 
I believe I'd rather live with some ambiguity in the current format (although there wasn't any ambiguity this year) and sign up to play eight more games next year (or twelve or four or however many my schedule allows). Maybe there is some way for the tournament to be fun and friendly and flexible and not too long and still reliably determine a champion, but maybe not. Maybe focusing too much on a champion will force us to cut out other desirable features.
The International Chess Federation (FIDE) decided last decade that having a World Championship match every year was better than once every three years, so they introduced knockout tournaments to crown champions. But they ended up having to crown someone ranked below #30 in the world as "World Champion" which was ridiculous. It takes a long time and a lot of games to determine who is best, as FIDE discovered to their sorrow.
We are lucky that the Arimaa World Championship so far has done reasonably well at picking the best player within only eight rounds (only five at first!), but the more popular the game becomes, the longer the tournament will have to stretch out to do a decent job of what it is trying to do. I'm not eager to have the Postal Tournament suffer the same stretching. I expect the tournament length required to do a decent job would simply not be worth it compared to other nice features sacrificed along the way.
(of course, you don't have to convince me otherwise, just Omar  )
|
| « Last Edit: Oct 9th, 2007, 1:53pm by Fritzlein » |
IP Logged |
|
|
|
mistre
Forum Guru
Gender:
Posts: 553
|
|
Re: 2008 Postal Tournament Format
« Reply #5 on: Oct 9th, 2007, 3:10pm » |
Quote Modify
|
Points taken. There is obviously a reason that it is set up the way it is - I was just offering suggestions. Since no one else has stepped forward with ideas, last year's format is fine with me.
I noticed that there was some kind of point system as well as W-L last year. What was that about? Another way to try to rank players?
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2008 Postal Tournament Format
« Reply #6 on: Oct 9th, 2007, 5:26pm » |
Quote Modify
|
The scoring by points is obsolete. It used to be that the refund of entry fees was based on points, so you could get back only part of your entry fee, or even turn a profit. Points were +60 for winning, +N for losing in N moves (with a cap of 60) and -60 for timing out or resigning. To a small extent, winners earned at the expense of losers, but mostly it was people who didn't time out earned at the expense of people who abandoned games. I believe that in 2006, everyone who played all their games to the finish without resigning or timing out turned a profit regardless of W-L record.
I liked the old system, because it encouraged people to fight to the death in every game. Even psuedo-resigning by making suicidal moves was punished, because losing in fewer moves was worth fewer points.
Now the refund is all or nothing. One timeout is the same as ten; you forfeit the entry fee. Game length no longer matters. I understand it is much easier for Omar this way, but I do miss the old points system.
|
| « Last Edit: Oct 9th, 2007, 5:31pm by Fritzlein » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2008 Postal Tournament Format
« Reply #7 on: Oct 10th, 2007, 11:42am » |
Quote Modify
|
For the record the 2007 Postal Tournament started February 25 and finished September 27, lasting just over seven months. We can keep that in mind when consider whether to keep the time control of 1d/60d/100/0/300d/21d for 2008.
|
|
IP Logged |
|
|
|
mistre
Forum Guru
Gender:
Posts: 553
|
|
Re: 2008 Postal Tournament Format
« Reply #8 on: Oct 10th, 2007, 8:26pm » |
Quote Modify
|
Could we use 1d/14d/100/14d/0 instead? It appears to be the most popular time control and the one I use the most often.
I am not sure how much shorter the overall tournament would be - how many moves went over 14 days?
|
|
IP Logged |
|
|
|
jdb
Forum Guru
Arimaa player #214
Gender:
Posts: 682
|
|
Re: 2008 Postal Tournament Format
« Reply #9 on: Oct 11th, 2007, 3:01pm » |
Quote Modify
|
on Oct 10th, 2007, 8:26pm, mistre wrote:Could we use 1d/14d/100/14d/0 instead? It appears to be the most popular time control and the one I use the most often.
I am not sure how much shorter the overall tournament would be - how many moves went over 14 days? |
|
This format was used in a previous year.
It was discovered that if a person goes away for a week (say on vacation) it creates too much time pressure.
|
|
IP Logged |
|
|
|
chessandgo
Forum Guru
Arimaa player #1889
Gender:
Posts: 1244
|
|
Re: 2008 Postal Tournament Format
« Reply #10 on: Oct 11th, 2007, 4:27pm » |
Quote Modify
|
on Oct 10th, 2007, 8:26pm, mistre wrote:Could we use 1d/14d/100/14d/0 instead? It appears to be the most popular time control and the one I use the most often.
I am not sure how much shorter the overall tournament would be - how many moves went over 14 days? |
|
Just as I see a point of having WC games with longer thinking time than popular training games, it feels good to have postal tourney games with longer thinking time than usual postal games. What point is there to have a tournament if the games quality is not higher ?
I'm really for a long time control for the postal tourney ... at least 1d/60d/100/0/300d/21d
|
|
IP Logged |
|
|
|
Isaac Grosof
Forum Guru
Longtime Arimaa Fan
Gender:
Posts: 175
|
|
Re: 2008 Postal Tournament Format
« Reply #11 on: Oct 11th, 2007, 11:11pm » |
Quote Modify
|
on Oct 11th, 2007, 4:27pm, chessandgo wrote:
Just as I see a point of having WC games with longer thinking time than popular training games, it feels good to have postal tourney games with longer thinking time than usual postal games. What point is there to have a tournament if the games quality is not higher ?
I'm really for a long time control for the postal tourney ... at least 1d/60d/100/0/300d/21d |
|
possibly one formal and one informal?
|
|
IP Logged |
Sorry about that one thing.
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2008 Postal Tournament Format
« Reply #12 on: Oct 25th, 2007, 4:31pm » |
Quote Modify
|
The big rule change we should implement for 2008 is something that we discussed but didn't implement in 2007, namely making it easier and more attractive for developers to enter their bots. In the early years the tournament was more like a championship, with money available for winning games, so it made sense for the restrictions on computers to be more severe. Now, however, there is no prize money, only an all-or-nothing refund for participation, so the justification for bot strictures is gone.
To me the only restriction that still makes sense is asking that the developer not help the bot think. For example, if the developer sees that the bot should really take a rabbit on a particular move, he could change the evaluation function to make rabbits ten times as valuable for that move only. Allowing this is silly, because we want to gauge the performance of algorithms, not humans disguised as algorithms.
On the other hand, I heartily dislike the rule that no code changes be allowed during the tournament. We want developers working on their bots all year long if we can induce it by any means. We don't want code frozen for months. As soon as a developer makes improvements, he should be able to try them out in real games, including ongoing postal games.
Another rule we should do away with is restrictions on hardware. The $1000 price restriction makes sense for the Arimaa Challenge, but not for the Postal Tournament. If somebody wants to enter a supercomputer, I would be delighted to see how much difference the extra speed makes. This is an excellent way to gain insight into the brains/brawn trade-off, since the Challenge itself explicitly prohibits brawn.
Finally, I support allowing the thinking time to change throughout the tournament. If the hardware isn't available full time, let's give the developer flexibility to move quickly for a cycle so the machine can do something else for a day. Also, as some games finish up, let's let the developer think longer per move in the remaining games, just as the humans do.
We are in stage where humans are totally dominant and computers can't compete, particularly at postal speeds. In this context, we should give extra advantages to the machines to see whether anything makes them competitive. Maybe this will give a boost to low developer morale, and stimulate some developers who have given up because the Challenge is just too far out of reach.
|
|
IP Logged |
|
|
|
arimaa_master
Forum Guru
Arimaa player #2010
Gender:
Posts: 358
|
|
Re: 2008 Postal Tournament Format
« Reply #13 on: Oct 26th, 2007, 12:16pm » |
Quote Modify
|
on Oct 25th, 2007, 4:31pm, Fritzlein wrote:The big rule change we should implement for 2008 is something that we discussed but didn't implement in 2007, namely making it easier and more attractive for developers to enter their bots. In the early years the tournament was more like a championship, with money available for winning games, so it made sense for the restrictions on computers to be more severe. Now, however, there is no prize money, only an all-or-nothing refund for participation, so the justification for bot strictures is gone.
To me the only restriction that still makes sense is asking that the developer not help the bot think. For example, if the developer sees that the bot should really take a rabbit on a particular move, he could change the evaluation function to make rabbits ten times as valuable for that move only. Allowing this is silly, because we want to gauge the performance of algorithms, not humans disguised as algorithms.
On the other hand, I heartily dislike the rule that no code changes be allowed during the tournament. We want developers working on their bots all year long if we can induce it by any means. We don't want code frozen for months. As soon as a developer makes improvements, he should be able to try them out in real games, including ongoing postal games.
Another rule we should do away with is restrictions on hardware. The $1000 price restriction makes sense for the Arimaa Challenge, but not for the Postal Tournament. If somebody wants to enter a supercomputer, I would be delighted to see how much difference the extra speed makes. This is an excellent way to gain insight into the brains/brawn trade-off, since the Challenge itself explicitly prohibits brawn.
Finally, I support allowing the thinking time to change throughout the tournament. If the hardware isn't available full time, let's give the developer flexibility to move quickly for a cycle so the machine can do something else for a day. Also, as some games finish up, let's let the developer think longer per move in the remaining games, just as the humans do.
We are in stage where humans are totally dominant and computers can't compete, particularly at postal speeds. In this context, we should give extra advantages to the machines to see whether anything makes them competitive. Maybe this will give a boost to low developer morale, and stimulate some developers who have given up because the Challenge is just too far out of reach. |
|
I fully support all these ideas.
|
|
IP Logged |
|
|
|
chessandgo
Forum Guru
Arimaa player #1889
Gender:
Posts: 1244
|
|
Re: 2008 Postal Tournament Format
« Reply #14 on: Oct 27th, 2007, 3:14am » |
Quote Modify
|
me too
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print