Arimaa Forum - Print Page


    
      
        Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> Events >> 2008 Postal Tournament Format
        
(Message started by: Fritzlein on Oct 5^th, 2007, 2:13pm)

Title: 2008 Postal Tournament Format
Post by Fritzlein on Oct 5^th, 2007, 2:13pm

on 10/05/07 at 12:21:36, mistre wrote:

On another note, I am also interested in participating in the next Postal Tournament. I looked at the results from last year and I was disappointed that there is no ranking or winner. Is there a way to make the Postal Tournament more like the World Championship?

The key problem in making a championship out of postal games is that the length of the games prevents having enough rounds in the same year. At one day per player per move, a sixty-move game takes 120 days, and that's without reserve time and with some kind of nasty cutoff for longer games. Even a single-elimination tournament with five rounds would span multiple years.

The solution was to pair all the games at the beginning of the tournament, and have the games be simultaneous rather than consecutive. But that introduced another problem. The fair way to have simultaneous games pick a champion is to play a full round-robin, but a full round-robin was more games at once than most people wanted to play. So Omar introduced a ten-game cap if more than eleven players signed up, with the greatest ratings mismatches being discarded.

As people pointed out at the time, that made the postal tournament a bit un-championship-like in 2005 and 2006. Someone in the bottom of the draw with an 8-2 record had a great tournament, true, but that wasn't as impressive a result as going 7-3 at the top of the draw. It was just an accident that we had exactly one 10-0 player each year, namely me in 2005 and 99of9 in 2006. But Omar actually didn't mind that the postal tournament was poor at ordering players by skill, and in fact was happy that we didn't need a bunch of the mis-matched slaughter games that are necessary in the World Championship.

So in 2007, he made the postal tournament even less like a championship and more just-for-fun by allowing people to specify how many simultaneous postal games they thought they could handle. Game-hungry players got 14 or 15 games against the whole range of opponents, while time-constrained people chose to play 3 to 5 games against only opponents near to them in rating. I was somewhere in the middle with 8 games.

I personally don't mind the anti-championship feel of the tournament. I like that my games were interesting, and I gained some new insights into Arimaa theory, so I'm happy. And as it turned out, we once again had a clear champion anyway, with chessandgo going undefeated in twelve games. The fact that camelback also went undefeated in four games doesn't undermine anyone's clarity about who the best postal player was. Indeed, you can't even really say camelback didn't have a chance to be champion: he did have a chance by signing up for twelve games and beating all comers.

All in all, I was quite content with the format for 2007. But I really wouldn't mind going back to the 10-games-each style of 2005. And of course, we can always make changes for next year. Suggestions are welcome, and the sooner they are made, the greater chance they have of getting enough support to be adopted.

Title: Re: 2008 Postal Tournament Format
Post by camelback on Oct 6^th, 2007, 5:43pm

on 10/05/07 at 14:13:49, Fritzlein wrote:

The fact that camelback also went undefeated in four games doesn't undermine anyone's clarity about who the best postal player was. Indeed, you can't even really say camelback didn't have a chance to be champion: he did have a chance by signing up for twelve games and beating all comers.

Aha.. The fact is chessandgo is World Champion and I'm a noob, joined on Jan 2007 and postal games started Feb 25. I didn't have any clue about the game by then and even now..

My ratings were low and I covered a bit of ground while the games were in progress. I was just happy-go-lucky to have a finish like this. This is attributed to TheMob discussions that taught me how to analyze the game.

I like the current format. It gives freedom pick your number of games.

Title: Re: 2008 Postal Tournament Format
Post by omar on Oct 7^th, 2007, 7:11am

I would not have been able to play 10 games this year. I think being able to select the number of games allowed for more participation.

Title: Re: 2008 Postal Tournament Format
Post by mistre on Oct 9^th, 2007, 9:01am

OK - how about we keep the current format where you can choose as many games as you want to play, but also have a cap of say 10 or 12 games. Whoever wants to play fewer can. And then at the end - have the top 4 players (how you would determine top 4 is another question), face off in a single elimination semifinal and then a final.

This would keep the format relatively the same with adding an element at the end to have a champion.

I am sure Karl or others would have ideas on how to determine the top 4.

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Oct 9^th, 2007, 1:46pm

It's going to take a bit to sell me on the value of a two-round, four-player playoff, versus the cost of at least an additional 240 days, spilling over into the next year when the players might rather be starting on the next year's postal tournament. Two more rounds doesn't necessarily increase clarity much, and it loses the nice feature that the tournament wraps up soon enough that it doesn't overlap other tournaments.

How would a two-round playoff have worked this year? Let's take the top four finishers by wins-losses, with a maximum of twelve games played: chessandgo, me, Ron Weasley, and camelback. Let's say I beat Ron Weasley while chessandgo beats camelback, and then I beat chessandgo in the finals. Are we happy because we have a clear champion? No! In my two head-to-head games with chessandgo, we each won one. Overall I would be 8-1 he would be 12-1. By what grounds am I champion, other than getting lucky late rather than early? We would need a playoff playoff.

Or what if Ron Weasley won the playoff? He would be Postal Champion, but only 1-1 against me, only 1-1 against chessandgo, and 11-3 overall compared to my 7-2 and chessandgo's 13-1. The point is that the playoff system, although it unambiguously selects a winner, doesn't even necessarily select the person who played best within the tournament.

On a related note, I'm sure you can guess how I feel about all the clamour for a playoff system to determine a national champion of college football. If the NCAA caves in and introduces an eight-team playoff, then it won't be long before some 8-3 team upsets an 11-0 team in the final, so the "National Champion" is clearly neither the best team in the country, nor the team that had the best year. I hope someone feels silly then. :P

I believe I'd rather live with some ambiguity in the current format (although there wasn't any ambiguity this year) and sign up to play eight more games next year (or twelve or four or however many my schedule allows). Maybe there is some way for the tournament to be fun and friendly and flexible and not too long and still reliably determine a champion, but maybe not. Maybe focusing too much on a champion will force us to cut out other desirable features.

The International Chess Federation (FIDE) decided last decade that having a World Championship match every year was better than once every three years, so they introduced knockout tournaments to crown champions. But they ended up having to crown someone ranked below #30 in the world as "World Champion" which was ridiculous. It takes a long time and a lot of games to determine who is best, as FIDE discovered to their sorrow.

We are lucky that the Arimaa World Championship so far has done reasonably well at picking the best player within only eight rounds (only five at first!), but the more popular the game becomes, the longer the tournament will have to stretch out to do a decent job of what it is trying to do. I'm not eager to have the Postal Tournament suffer the same stretching. I expect the tournament length required to do a decent job would simply not be worth it compared to other nice features sacrificed along the way.

(of course, you don't have to convince me otherwise, just Omar ;-))

Title: Re: 2008 Postal Tournament Format
Post by mistre on Oct 9^th, 2007, 3:10pm

Points taken. There is obviously a reason that it is set up the way it is - I was just offering suggestions. Since no one else has stepped forward with ideas, last year's format is fine with me.

I noticed that there was some kind of point system as well as W-L last year. What was that about? Another way to try to rank players?

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Oct 9^th, 2007, 5:26pm

The scoring by points is obsolete. It used to be that the refund of entry fees was based on points, so you could get back only part of your entry fee, or even turn a profit. Points were +60 for winning, +N for losing in N moves (with a cap of 60) and -60 for timing out or resigning. To a small extent, winners earned at the expense of losers, but mostly it was people who didn't time out earned at the expense of people who abandoned games. I believe that in 2006, everyone who played all their games to the finish without resigning or timing out turned a profit regardless of W-L record.

I liked the old system, because it encouraged people to fight to the death in every game. Even psuedo-resigning by making suicidal moves was punished, because losing in fewer moves was worth fewer points.

Now the refund is all or nothing. One timeout is the same as ten; you forfeit the entry fee. Game length no longer matters. I understand it is much easier for Omar this way, but I do miss the old points system.

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Oct 10^th, 2007, 11:42am

For the record the 2007 Postal Tournament started February 25 and finished September 27, lasting just over seven months. We can keep that in mind when consider whether to keep the time control of 1d/60d/100/0/300d/21d for 2008.

Title: Re: 2008 Postal Tournament Format
Post by mistre on Oct 10^th, 2007, 8:26pm

Could we use 1d/14d/100/14d/0 instead? It appears to be the most popular time control and the one I use the most often.

I am not sure how much shorter the overall tournament would be - how many moves went over 14 days?

Title: Re: 2008 Postal Tournament Format
Post by jdb on Oct 11^th, 2007, 3:01pm

on 10/10/07 at 20:26:28, mistre wrote:

This format was used in a previous year.

It was discovered that if a person goes away for a week (say on vacation) it creates too much time pressure.

Title: Re: 2008 Postal Tournament Format
Post by chessandgo on Oct 11^th, 2007, 4:27pm

on 10/10/07 at 20:26:28, mistre wrote:

Just as I see a point of having WC games with longer thinking time than popular training games, it feels good to have postal tourney games with longer thinking time than usual postal games. What point is there to have a tournament if the games quality is not higher ?

I'm really for a long time control for the postal tourney ... at least 1d/60d/100/0/300d/21d ;)

Title: Re: 2008 Postal Tournament Format
Post by 722caasi on Oct 11^th, 2007, 11:11pm

on 10/11/07 at 16:27:27, chessandgo wrote:

possibly one formal and one informal?

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Oct 25^th, 2007, 4:31pm

The big rule change we should implement for 2008 is something that we discussed but didn't implement in 2007, namely making it easier and more attractive for developers to enter their bots. In the early years the tournament was more like a championship, with money available for winning games, so it made sense for the restrictions on computers to be more severe. Now, however, there is no prize money, only an all-or-nothing refund for participation, so the justification for bot strictures is gone.

To me the only restriction that still makes sense is asking that the developer not help the bot think. For example, if the developer sees that the bot should really take a rabbit on a particular move, he could change the evaluation function to make rabbits ten times as valuable for that move only. Allowing this is silly, because we want to gauge the performance of algorithms, not humans disguised as algorithms.

On the other hand, I heartily dislike the rule that no code changes be allowed during the tournament. We want developers working on their bots all year long if we can induce it by any means. We don't want code frozen for months. As soon as a developer makes improvements, he should be able to try them out in real games, including ongoing postal games.

Another rule we should do away with is restrictions on hardware. The $1000 price restriction makes sense for the Arimaa Challenge, but not for the Postal Tournament. If somebody wants to enter a supercomputer, I would be delighted to see how much difference the extra speed makes. This is an excellent way to gain insight into the brains/brawn trade-off, since the Challenge itself explicitly prohibits brawn.

Finally, I support allowing the thinking time to change throughout the tournament. If the hardware isn't available full time, let's give the developer flexibility to move quickly for a cycle so the machine can do something else for a day. Also, as some games finish up, let's let the developer think longer per move in the remaining games, just as the humans do.

We are in stage where humans are totally dominant and computers can't compete, particularly at postal speeds. In this context, we should give extra advantages to the machines to see whether anything makes them competitive. Maybe this will give a boost to low developer morale, and stimulate some developers who have given up because the Challenge is just too far out of reach.

Title: Re: 2008 Postal Tournament Format
Post by arimaa_master on Oct 26^th, 2007, 12:16pm

on 10/25/07 at 16:31:14, Fritzlein wrote:

I fully support all these ideas.

Title: Re: 2008 Postal Tournament Format
Post by chessandgo on Oct 27^th, 2007, 3:14am

me too

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Jan 26^th, 2008, 9:22am

I just noticed in the event schedule that registration for the Postal Mixer opens on February 3, which will be weird if the rules are unknown as of that time. Omar, would you mind posting a draft of the rules, possibly incorporating suggestions from this thread? Otherwise there will be no discussion of the proposed rules until after they are the official rules.

Title: Re: 2008 Postal Tournament Format
Post by omar on Jan 27^th, 2008, 7:07am

Thanks for the reminder on this Karl. I've posted the details for the upcoming postal here:

http://arimaa.com/arimaa/tourn/postal/2008/

I've included the longer time control and the very minimal restrictions for bots. I didn't include anything to give a monetary incentive for putting up a longer fight. For now I want to keep the amount returned simple.

Let me know if I missed anything or if there are other suggestions. I really appreciate the feedback from everyone in this thread; thanks.

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Feb 3^rd, 2008, 5:09pm

I think the rules governing bots are now sufficiently loose. Some might say they are too loose, because they allow giving the bot one brain for half of its games and a different brain for the other half of its games, etc. I doubt, however, that this latitude will be abused. With no prize money on the line, it doesn't bother me if Zombie plays smarter against me than against the other players; in fact I would welcome it.

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Feb 16^th, 2008, 10:50pm

Where do we register? According to the schedule registration is officially open, but I can't find a link. As soon as registration for the Spectator Contest closes we should replace the front page link for that registration with a front-page link to this registration.

Title: Re: 2008 Postal Tournament Format
Post by omar on Feb 17^th, 2008, 12:20am

Thanks for the reminder Karl. I've turned on the registration for the Postal.

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Feb 17^th, 2008, 11:26am

Holy cow. I registered as soon as I saw your post, and I'm the sixth one in! I wonder if we will break a participation record this year.

Title: Re: 2008 Postal Tournament Format
Post by nbarriga on Feb 20^th, 2008, 12:36am

Maybe I'm a bit offtopic here, but anyway: my rating as seen on the gamerrom is 1669, and my rating on the players list of the postal is 1534, which was my rating when the 2007 postal started. Bug?

Title: Re: 2008 Postal Tournament Format
Post by chessandgo on Feb 20^th, 2008, 4:38am

on 02/20/08 at 00:36:44, nbarriga wrote:

Maybe I'm a bit offtopic here, but anyway: my rating as seen on the gamerrom is 1669, and my rating on the players list of the postal is 1534, which was my rating when the 2007 postal started. Bug?

Yeah, there was the same phenomenon for the WC , ratings as announced before the start are those of previous year, but I'm sure Omar will take care of this before the tourney starts, no worry :)

Title: Re: 2008 Postal Tournament Format
Post by mistre on Feb 20^th, 2008, 9:15am

Are any bots going to be registered for the postal tournament? I am hoping to see a few in there. Zombie? Opfor?

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Feb 20^th, 2008, 3:13pm

Ten registrants already, and it's more than a month from the deadline! I'm starting to get excited.

If the deadline approaches and there are no bots registered, I will look into getting bot_Arimaanator to play from my desktop, but it would be much more exciting if a bot entered that was under active development rather than an outdated and buggy bot. Note that this year the bot can change throughout the tournament, so it is an opportunity for developers to try out a lot of stuff.

Title: Re: 2008 Postal Tournament Format
Post by Janzert on Feb 20^th, 2008, 4:03pm

OpFor will probably enter. But I am, and I'm sure most other developers are, doing the last minute deadline wrap everything up for the computer championship drill at the moment. ;D

Janzert

Title: Re: 2008 Postal Tournament Format
Post by nbarriga on Mar 15^th, 2008, 12:46am

Will there be an email to all Arimaa players reminding them to join the Postal Mixer? The deadline approaches, and I don't see any low ranking and only a few midranking players registered.

It would be no fun to be shamfully beaten in all the 8 games I registered for ;D

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Mar 15^th, 2008, 6:29pm

I'm also in favor of a dual-purpose mass mailing, both to invite everyone to play in the Postal Mixer and to invite everyone to play against sharp and Bomb in the qualifying round.

Given that we already have 12 registered players and three weeks left to register, I would be very surprised to see fewer than 20 players total. This event has been a ton of fun in the past, and people know it.

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Mar 26^th, 2008, 8:17pm

Only nine days left to register! :o :o :o I'm surprised that not all of the top players from the World Championship are in yet. 99of9, jdb, naveed, ArifSyed, camelback, froody... I assume you are just waiting for the last minute to register. ;) Personally, if I could only play in one event in the whole year, it would be the Postal Mixer and not the World Championships, because I like the non-championship (fewer mistmatch) pairing format of the postal better.

I'm going to start harassing the active newcomers in the gameroom to join. I think some folks are shy about playing humans at first, even if they love Arimaa. I know that for a while I was afraid that I wasn't good enough to play humans, so I wanted to play it safe and get good against bots first. But that was silly; it's so much more rewarding to play a variety of humans. Maybe also with all the "championships" going on, relative newcomers expect the Postal to be brutal instead of being paired as close to your own rating as possible.

Also, I'm hoping to see OpFor and sharp registered. If you guys aren't planning to play, let me know. I think there should be at least one bot in the tournament, and I'll set up bot_Arimanator to be that one if need be.

Title: Re: 2008 Postal Tournament Format
Post by mistre on Mar 26^th, 2008, 8:53pm

Also, it is a lot less intimidating for new players to play postal vs. humans as there is no time pressure.

Newer players should be encouraged to maybe sign up for 4-6 games to get their feet wet. I just missed out on the tournament last year. I either missed it because I joined after it started, or it wasn't publicized enough.

Let's spread the word!

Title: Re: 2008 Postal Tournament Format
Post by Janzert on Mar 27^th, 2008, 1:53am

Funny you should mention the bots Fritzlein. I just sent omar a message to register opfor this afternoon and looks like omar has the registration finished now. :)

Just so any other developers that wish to enter know; you need to contact omar directly. The automatic registration goes through the gameroom login and of course doesn't work for a bot.

Janzert

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Mar 27^th, 2008, 8:14am

on 03/27/08 at 01:53:23, Janzert wrote:

Funny you should mention the bots Fritzlein. I just sent omar a message to register opfor this afternoon and looks like omar has the registration finished now. :)

Yay! I'm glad OpFor is in for lots of games so that I will probably get one. With the current registration I would certainly get paired to OpFor, and I think that will stay true even if lots of people sign up.

I hope Opfor's participation is not only fun for the rest of us, but also useful to you in further development.

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Mar 31^st, 2008, 6:53pm

Bot_sharp is in! Woohoo! Only ten games, though, so I probably won't get one of them. :(

Title: Re: 2008 Postal Tournament Format
Post by Fritzlein on Apr 12^th, 2008, 3:48pm

Well, it looks like I'll get a game against sharp after all. With the lower number of players this year, the tournament format will be closer to a round-robin. Will the pairings be up today, Omar? I see we are scheduled to start tomorrow...