Welcome, Guest. Please Login or Register.
May 7th, 2024, 6:02am

Home Home Help Help Search Search Members Members Login Login Register Register
Arimaa Forum « 2007 Computer Championship and Challenge »


   Arimaa Forum
   Arimaa
   Events
(Moderator: supersamu)
   2007 Computer Championship and Challenge
« Previous topic | Next topic »
Pages: 1 2  Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print
   Author  Topic: 2007 Computer Championship and Challenge  (Read 2275 times)
omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
2007 Computer Championship and Challenge
« on: Oct 7th, 2006, 2:26am »
Quote Quote Modify Modify

I've finished editing the pages for the 2007 Computer Championship and Challenge match (I think).  
 
http://arimaa.com/arimaa/wcc/2007/
 
http://arimaa.com/arimaa/challenge/2007/
 
Please have a look and let me know if you have any feedback. Thanks.
« Last Edit: Oct 9th, 2006, 8:34pm by omar » IP Logged
unic
Forum Guru
*****



Arimaa player #1878

   


Gender: male
Posts: 63
Re: 2007 Computer Championship and Challenge
« Reply #1 on: Oct 7th, 2006, 3:25pm »
Quote Quote Modify Modify

Fairy won't be there (unless somebody else enters it...) - I'm doing programming for fun, not for research, and modifying Fairy to follow these conditions would not be fun for me:
Quote:
Participants in the tournament will be provided with accounts on a Linux computers to port and setup their programs before the tournament games begin.
[...]
    * provide a README file which explains the options that are available for the program and how they are to be entered.
    * be statically compiled and linked executables that can run on the provided Linux system.
 Be sure to use the -static option of the linker.
    * provide an option to specify a hardware independent play strength such as the number of nodes to evaluate before stopping, or the number of plys to search, etc. When this option is used the program should not look at the time control. This option will not be used in the championship or challenge match, but will be used after these events when the program is made available for others to play against in the Arimaa gameroom. The purpose of this is so that the program will play the same strength independent of the hardware or system load.
    * log all initial conditions at the start of the game and at the start of each move (such as the seed used for the random number generator) so that given these initial conditions the program can be made to play the same way again.
    * provide an option to give the program it's own initial conditions log file as input so that the values in that file can override the values it would use otherwise in order to repeat a game.

If somebody else wants to modify Fairy so that it fulfills the conditions above, and enter it, feel free to do so.  Source is at the download page (I have a newer experimental version... but that one is still weaker than the old one.)
IP Logged
omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: 2007 Computer Championship and Challenge
« Reply #2 on: Oct 7th, 2006, 4:14pm »
Quote Quote Modify Modify

The first and second conditions are fairly simple:
* Provide a README file for how to run the bot
* Statically compile using the -static option (if program is
   in C or C++); not needed for Java programs.
 
The third one is automatically there if you used the C sample bot code. Otherwise, just add an option to have the bot stop after a specified search depth or number of position evals or something else so that its play streangth can be fixed using such parameters.
 
The last two are not needed if your bot does not use any random number generator. If it does then it may produce a different move if invoked with the same position a second time. The reason for these conditions was so that it could be proved that the a particular move was produced by the bot by giving it the same game state and same seed for the random number generator (assuming the bot logged to a file what the seed value was) and having it produce the same move again.
 
However, thinking about this again, I just realized that the small variations in the server load could also cause a slight fluctuations in the number of positions a bot evalutes from one run to another even the clock time for the move is the same and thus cause it to produce different moves even if all the initial conditions were the same.
 
Thus, I will eliminate these last two conditions.  
 
Hopefully the other conditions aren't too much of a burden for the developers.
« Last Edit: Oct 7th, 2006, 4:20pm by omar » IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: 2007 Computer Championship and Challenge
« Reply #3 on: Oct 8th, 2006, 11:11am »
Quote Quote Modify Modify

It would be a shame if Fairy didn't compete in the computer championship, especially if (as expected) there are fewer than eight entrants.  If unic feels it is too great a burden to code in some mechanism to limit the search depth of Fairy, thus not qualifying, I hope that someone else will take the earlier, published version of Fairy, add the depth-limit feature, and enter that version of Fairy just for benchmarking.  Even if the developer getting Fairy ready for entry was entering his own bot (99of9, jdb, doublep, etc.) it seems totally in the spirit of the rule "one bot per developer" to have one version of Fairy playing, one way or another.
IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: 2007 Computer Championship and Challenge
« Reply #4 on: Oct 8th, 2006, 11:19pm »
Quote Quote Modify Modify

on Oct 7th, 2006, 2:26am, omar wrote:
I've finished editing the pages for the 2007 Computer Championship and Challenge match (I think).  
 
http://arimaa.com/arimaa/wc/2007/

I guess you mean http://arimaa.com/arimaa/wcc/2007/
 
The one issue I have with the format is what to do in case of a tie for second place based on the number of wins.  The rules now say a tie will be broken by pre-tournament rating, but why not have an elimination playoff between anyone involved in a tie for second?  If there are N players tied for second, it only takes N-1 more games to have a playoff.  (Usually it will be two players tied for second, if there is a tie at all, and thus require only one playoff game.)
 
Quote:
http://arimaa.com/arimaa/challenge/2007/
 
Please have a look and let me know if you have any feedback. Thanks.

In the first paragraph you say that the prize is currently $15,000, but that leaves out $1,500 pledged from PMertens and $600 from me, for a total of $17,100.  At least a couple of community members are taking your side of the bet!
 
I would also try to make this wording more clear: "Any human player that wants to play against the programs must play both programs equally in terms of number of times and color. A human player must not play more than two times against either program and no more than one time with the same color. Thus, if a human player plays program A as gold, the player must then play program B as gold."
 
Rather than saying players "must" do this or that, is it possible for you to technically enforce the rule?  Can you set the bots up to only accept invitations from humans they haven't yet played with that color?  If so, then I would say, "A human player who wishes to participate in the qualifying process will be permitted to play each of the two contenders once with each color, for a total of four games.  A game does not count for the standings until the same human plays the other contender with the same color."
 
Perhaps the ideal mechanism is to have a link that says "Participate in the Challenge qualifying process", such that clicking on that link will pair the human appropriately with a contender and color to play.
 
I like that the defenders of the Challenge are barred from these qualifying games.  I would also bar the developers of the two contending bots from playing in these qualifying games, because they have a strong incentive to try to skew the standings by losing to their own bot.  I think that will still leave plenty of possible opponents, given the current size of the Arimaa community.
 
For good measure, I would include a rule that any games which appear to have been intentionally lost in order to distort the standings may be excluded at your discretion.
 
I see that you have listed the time control for all the qualifying and challenge games as 2/2/100/10/8 .  I can understand having a long time control in order to give humans the greatest possible advantage, but I don't think humans need the slow time control this year.  Why not have the games be at 90 seconds per move?  This would make it easier for the human defenders (less time commitment) without significantly affecting winning chances, and would make it more interesting for spectators (less boring wait for the computer to move).  Also the faster time control would probably get more people to play the two bots during qualifying, and thus give us a better idea of which bot is really stronger against humans.
 
You could always change the time control back to two minutes per move in future years if it seems the humans might need it, and if Arimaa community has grown to the point that there is a surplus of people wanting to play in the qualifying games.  I understand that a slow time control is appropriate to let humans give their best defense of their own intelligence before a program ultimately wins the Challenge.  In the mean time, however, speeding up the games to 90 seconds per move is good for spectators, and good for participants, and it makes the dominance of humans all the more clear.
« Last Edit: Oct 9th, 2006, 12:46pm by Fritzlein » IP Logged

omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: 2007 Computer Championship and Challenge
« Reply #5 on: Oct 9th, 2006, 8:36pm »
Quote Quote Modify Modify

on Oct 8th, 2006, 11:19pm, Fritzlein wrote:

I guess you mean http://arimaa.com/arimaa/wcc/2007/

Yes, that's what I ment. Thanks. I've fixed it.
IP Logged
omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: 2007 Computer Championship and Challenge
« Reply #6 on: Oct 10th, 2006, 8:45am »
Quote Quote Modify Modify

on Oct 8th, 2006, 11:19pm, Fritzlein wrote:

The one issue I have with the format is what to do in case of a tie for second place based on the number of wins.  The rules now say a tie will be broken by pre-tournament rating, but why not have an elimination playoff between anyone involved in a tie for second?  If there are N players tied for second, it only takes N-1 more games to have a playoff.  (Usually it will be two players tied for second, if there is a tie at all, and thus require only one playoff game.)

But in the tournament rules I would need to specify it for all cases (like 3 or 4 players tied for second) and that gets kind of messy so I thought lets just use the pre-tournament rating. But thinking about it again, I guess we could just say: in case of ties for second place a single elemination fold tournament will be used to determine second place. Byes will be given to the player with the highest pre-tournament rating and the lower rated player within a pairing will chose the color assignment.  
 
I've changed it to this now.
 
Quote:

In the first paragraph you say that the prize is currently $15,000, but that leaves out $1,500 pledged from PMertens and $600 from me, for a total of $17,100.  At least a couple of community members are taking your side of the bet!

I was under the impression that those pledges were only good for one year. If that's not the case please let me know and I will add them to the total.
 
Quote:

I would also try to make this wording more clear: "Any human player that wants to play against the programs must play both programs equally in terms of number of times and color. A human player must not play more than two times against either program and no more than one time with the same color. Thus, if a human player plays program A as gold, the player must then play program B as gold."
 
Rather than saying players "must" do this or that, is it possible for you to technically enforce the rule?  Can you set the bots up to only accept invitations from humans they haven't yet played with that color?  If so, then I would say, "A human player who wishes to participate in the qualifying process will be permitted to play each of the two contenders once with each color, for a total of four games.  A game does not count for the standings until the same human plays the other contender with the same color."
 
Perhaps the ideal mechanism is to have a link that says "Participate in the Challenge qualifying process", such that clicking on that link will pair the human appropriately with a contender and color to play.

Yes, that's exactly what I was planning to do. I'll consider changing the wording.
 
Quote:

I like that the defenders of the Challenge are barred from these qualifying games.  I would also bar the developers of the two contending bots from playing in these qualifying games, because they have a strong incentive to try to skew the standings by losing to their own bot.  I think that will still leave plenty of possible opponents, given the current size of the Arimaa community.
 
For good measure, I would include a rule that any games which appear to have been intentionally lost in order to distort the standings may be excluded at your discretion.

Good ideas. I've added them to the challenge page. But instead of me we should let the match director decide if the game appears to be thrown.
 
Quote:

I see that you have listed the time control for all the qualifying and challenge games as 2/2/100/10/8 .  I can understand having a long time control in order to give humans the greatest possible advantage, but I don't think humans need the slow time control this year.  Why not have the games be at 90 seconds per move?  This would make it easier for the human defenders (less time commitment) without significantly affecting winning chances, and would make it more interesting for spectators (less boring wait for the computer to move).  Also the faster time control would probably get more people to play the two bots during qualifying, and thus give us a better idea of which bot is really stronger against humans.
 
You could always change the time control back to two minutes per move in future years if it seems the humans might need it, and if Arimaa community has grown to the point that there is a surplus of people wanting to play in the qualifying games.  I understand that a slow time control is appropriate to let humans give their best defense of their own intelligence before a program ultimately wins the Challenge.  In the mean time, however, speeding up the games to 90 seconds per move is good for spectators, and good for participants, and it makes the dominance of humans all the more clear.

We can't change the time controls later. I am working towards fixing them based on experience now, but once they are fixed we can't change them or anything else about the match.
« Last Edit: Oct 10th, 2006, 8:51am by omar » IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: 2007 Computer Championship and Challenge
« Reply #7 on: Oct 10th, 2006, 4:56pm »
Quote Quote Modify Modify

on Oct 10th, 2006, 8:45am, omar wrote:
We can't change the time controls later. I am working towards fixing them based on experience now, but once they are fixed we can't change them or anything else about the match.

Oh, well, if you are going to fix the time control for all the remaining years of the Challenge, then I would go with two minutes per move.  Even slower than that would help the humans more, of course, but two minutes is extremely slow already.  It's what I think of as the slowest reasonable time control.
 
However, before you set the rules in stone, do you want to at least experiment with 2/12/75/0/8/5 ?  So far the main argument against allowing reserve to accumulate beyond the max time per move is that it is experimental, but if we never even experiment with it, how will we know?  
 
By the way, one thing that is even more experimental is the method of having the top two Computer Championship bots contend for the right to be Challenger.  I'd be extremely leery of setting that in stone unless we try it for at least two years.  It seems much harder to test the selection method than to test a time control, because the time control would be tested nine times in a single match, whereas the selection process is only tested once per match.
 
Quote:
But in the tournament rules I would need to specify it for all cases (like 3 or 4 players tied for second) and that gets kind of messy so I thought lets just use the pre-tournament rating. But thinking about it again, I guess we could just say: in case of ties for second place a single elemination fold tournament will be used to determine second place. Byes will be given to the player with the highest pre-tournament rating and the lower rated player within a pairing will chose the color assignment.

Ah, I had just imagined the second-place playoff being a continuation of the main tournament.  Everyone tied for second place gets an extra life, and they play on as if they were the only ones left in the tournament.  This is the same as the single-elimination you are proposing, except that it retains the memory of repeat pairings and color assignments from games already played.  The difference is small, but since the two formats are conceptually equally easy to state, why not go with the fairer one?
 
Quote:
I was under the impression that those pledges were only good for one year. If that's not the case please let me know and I will add them to the total.

For last year I made a pledge of $500, good for only one year.  Then after nobody won that, I made a pledge of $600, good for only one year (i.e. this year).  After this challenge match I expect to make another pledge, perhaps for a lesser dollar amount, but good until 2020, to simplify the bookkeeping.
 
Here is PMertens' pledge:
on Oct 13th, 2005, 1:31am, PMertens wrote:
Herewith I change my pledge a little bit:
 
2006: 2000 (this year)
2007: 1500
2008: 1000
2009: 500
2010: 250
 
This also reflects the timeframe I expect to be finished with my bot and win the challenge Smiley

I don't think anyone else has pledged, so we get $15000 + $1500 + $600 = $17100.
IP Logged

omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: 2007 Computer Championship and Challenge
« Reply #8 on: Oct 11th, 2006, 8:27am »
Quote Quote Modify Modify

on Oct 10th, 2006, 4:56pm, Fritzlein wrote:

However, before you set the rules in stone, do you want to at least experiment with 2/12/75/0/8/5 ?  So far the main argument against allowing reserve to accumulate beyond the max time per move is that it is experimental, but if we never even experiment with it, how will we know?  

How about the possibility of allowing the human player to select the time control, but it must have an average time-per-move between 1 to 3 minutes. This way I don't have to fix an exact time control. But there are a lot of pros and cons associated with this. We might need to start a new topic to discuss it. However, for this year we will stick with the 2/2/100/10/8 time control, but discuss this for possible use in future years.
 
Quote:

By the way, one thing that is even more experimental is the method of having the top two Computer Championship bots contend for the right to be Challenger.  I'd be extremely leery of setting that in stone unless we try it for at least two years.  It seems much harder to test the selection method than to test a time control, because the time control would be tested nine times in a single match, whereas the selection process is only tested once per match.

Yes, if we find problems with this, we will have to amend it or use something different.
 
Quote:

Ah, I had just imagined the second-place playoff being a continuation of the main tournament.  Everyone tied for second place gets an extra life, and they play on as if they were the only ones left in the tournament.  This is the same as the single-elimination you are proposing, except that it retains the memory of repeat pairings and color assignments from games already played.  The difference is small, but since the two formats are conceptually equally easy to state, why not go with the fairer one?

Yes, this would probably be more fair, but the problem is I don't have a program yet that can do the pairing in this situation. So for now we'll just use single elimination. Besides I think the chances of having more than 2 players tie for second place when starting with 16 players is fairly low that we don't really need to worry about this too much. Anyone want to take a crack at figuring out the probability.
 
Quote:

I don't think anyone else has pledged, so we get $15000 + $1500 + $600 = $17100.

Thanks; I've updated the challenge page.
IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: 2007 Computer Championship and Challenge
« Reply #9 on: Oct 11th, 2006, 1:15pm »
Quote Quote Modify Modify

on Oct 11th, 2006, 8:27am, omar wrote:
Besides I think the chances of having more than 2 players tie for second place when starting with 16 players is fairly low that we don't really need to worry about this too much. Anyone want to take a crack at figuring out the probability.

On further inspection this is simple enough you don't need to worry about having a computer program to pair it.
 
The number of wins a player has at the end of a tournament is equal to the round he was eliminated minus the number of losses minus the number of byes received.  For all eliminated players the number of losses is the same, so we can simply compare round eliminated minus number of byes to determine second place.
 
The player who is eliminated last by definition has survived one round longer than all other eliminated players.  Therefore this player will win second outright unless he has recieved one more bye than a player eliminated the previous round.  At most two players can have been eliminated the previous round, so there can at most be a three-way tie for second place.
 
A three-way tie for second seems rather unlikely, since it requires
* The tournament must have exactly four players left at some point.  Sometimes this won't happen because eliminations will move the tournament directly from five or six players down to three.
* In the round of four players, exactly two players must be eliminated.  Sometimes this won't happen because one of the two losers in that round has an extra life left to play on
* The two players eliminated must both have one bye fewer than a remaining player on the verge of elimination
* That remaining player with an extra bye on the verge of elimination must lose the next round.
 
I'm guessing less than 5% chance of a three-way tie for second place.  A two-way tie for second place however, seems much more likely.  In both the World Championship and the Computer Championship last year, there was a clear second place winner based on number of wins, but there could have been a tie for second place had one game gone differently, and another game gone as expected.  Maybe there's about a 20% chance of a two-way tie for second place.
 
A two-way tie for second place requires only a one-game playoff, and color assignment can be done by hand based on colors in the tournament so far.  A three-way tie for second place could be resolved by having the last-eliminated played take on the winner of the two others.  (Note that pairing the two players eliminated in the second-last round is unlikely to create worse repeat-pairing scenario than another pairing.  Those two didn't play each other on the round they both were eliminated, and one of them played the later-eliminated player.)  That's two games, and again the color assignment could be done by hand for each game.
 
In short, the tie-break algorithm could be sufficiently straightforward that computer assistance is not required.
« Last Edit: Oct 11th, 2006, 1:27pm by Fritzlein » IP Logged

unic
Forum Guru
*****



Arimaa player #1878

   


Gender: male
Posts: 63
Re: 2007 Computer Championship and Challenge
« Reply #10 on: Oct 12th, 2006, 1:32pm »
Quote Quote Modify Modify

Regarding time controls:  I think shortening it is a bad idea.  I also think limiting time per move is a bad idea - compare most other mind sports that are played seriously (chess, shogi, go); knowing when to spend half an hour (or more - in professional shogi games, I've seen times up towards a couple of hours on one single crucial move) on a move and when to move quickly - in other words, deciding how to use one's time - is an important skill.
 
If a player plays quickly for a while and thus builds up a significant reserve of time, I think the player should be able to use it - limiting reserve time to a maximum, or putting a maximum time per move, serves to remove a skill element from the game.
 
Also, anything less than 2-3 minutes per move average time strikes me as a bit on the fast side... especially as Arimaa doesn't have much of an opening phase where the players can move quickly, saving up time for later (compare with chess...)
 
I'm dubious about letting the top two programs contend for the right to challenge the human - while there are advantages (the best program against other programs might not be the best against humans), the method given for selecting the challenging program seems... inexact and unfair to me.  
 
(Oh, and about Fairy, if I manage to actually get a version that's significantly stronger than the current one, I might reconsider entering it... but for now, I can't see the point to spend effort on that while Fairy is still so much weaker than the top programs - but if somebody else wants to modify/enter it, feel free to do so.)
IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: 2007 Computer Championship and Challenge
« Reply #11 on: Oct 12th, 2006, 4:39pm »
Quote Quote Modify Modify

Thanks for joining the discussion, unic.  It's good to have someone take up the banner for long thinking time and no restrictions on its allocation.  MrBrain used to argue for this, but he hasn't been around for a while.  I agree that time management is indeed a skill, and any limitations on the time management of the players removes some of that skill.  I know almost all chess players like to have full control over their time.
 
What do you think about the desires of spectators?  Do you think the comfort of the players should be balanced somewhat against the entertainment of the onlookers?   If you look at what sponsors of chess tournaments want, they generally want the games to speed up for the sake of the spectators, because spectators generally want something to happen at least every couple of minutes.
 
on Oct 12th, 2006, 1:32pm, unic wrote:
I'm dubious about letting the top two programs contend for the right to challenge the human - while there are advantages (the best program against other programs might not be the best against humans), the method given for selecting the challenging program seems... inexact and unfair to me.

Inexactness is going to be present with any method of choosing a challenger, for example an inferior bot winning the Computer Championship by an upset, so unless the problem is worse than straight triple-elimination among bots, the inexactness doesn't bother me so much.  But unfairness bothers me much more.  Can you expand more on why you think it is unfair?
 
Quote:
(Oh, and about Fairy, if I manage to actually get a version that's significantly stronger than the current one, I might reconsider entering it... but for now, I can't see the point to spend effort on that while Fairy is still so much weaker than the top programs - but if somebody else wants to modify/enter it, feel free to do so.)

Thank you for your blessing on getting the public version of Fairy entered.  It will be a great addition to the tournament even if it doesn't win.
IP Logged

omar
Forum Guru
*****



Arimaa player #2

   


Gender: male
Posts: 1003
Re: 2007 Computer Championship and Challenge
« Reply #12 on: Oct 14th, 2006, 8:38am »
Quote Quote Modify Modify

on Oct 11th, 2006, 1:15pm, Fritzlein wrote:

On further inspection this is simple enough you don't need to worry about having a computer program to pair it.

 
Thanks for analyzing this Karl. I have updated the WCC and WC rules page to use this for determining second place.
IP Logged
unic
Forum Guru
*****



Arimaa player #1878

   


Gender: male
Posts: 63
Re: 2007 Computer Championship and Challenge
« Reply #13 on: Oct 14th, 2006, 9:57am »
Quote Quote Modify Modify

on Oct 12th, 2006, 4:39pm, Fritzlein wrote:
It's good to have someone take up the banner for long thinking time and no restrictions on its allocation.  MrBrain used to argue for this, but he hasn't been around for a while.  I agree that time management is indeed a skill, and any limitations on the time management of the players removes some of that skill.  I know almost all chess players like to have full control over their time.
 
What do you think about the desires of spectators?  Do you think the comfort of the players should be balanced somewhat against the entertainment of the onlookers?   If you look at what sponsors of chess tournaments want, they generally want the games to speed up for the sake of the spectators, because spectators generally want something to happen at least every couple of minutes.

... that's where I am an atypical spectator.  To me, the focus should be on the contest of skills - finding out which of the players is the better one - not on entertaining the spectators.  And I (as a spectator) get more enjoyment out of watching a game the higher its level is - which means that for chess, e.g., I find 25-minute games boring to watch, while I sat for hours following the classical games of the recent world championship match.
 
In my opinion, the spectators shouldn't be catered to.  The competition to find out who is best should be in center; not the spectators.  If they want to watch, then that's good... if they don't want to watch, well, I don't see it as a problem if there are no spectators.
 
(Incidentally, I follow Formula-1 racing, and am very annoyed with the same tendency there - changing the sport so that its focus is on providing "entertainment", not on finding out which team/driver is the most skillful.  In F1, the tendency is even stronger than in chess; presumably because of the larger amount of money involved.  On a personal level, the changes made in order to make it more entertaining has partially caused me to lose my interest - I watch much fewer of the races now than I used to.)
 
So... I guess the question is - what is the focus of the competition?  To find out the best player, or to provide entertainment?  For me, it is to find out the best player - if I want entertainment, there are many other ways I can get that.
 
(And, on a related note, I don't see why the computer championship can't use a better method for selecting the champion - multiple round-robin should be more exact, and the bots shouldn't mind having to play many games - they're doing it automatically, after all.  Again, triple-elimination seems to be more "spectator-friendly" - few games are played, so easy for humans to follow, and plenty of "critical" games, where a program must win or be eliminated.  Multiple round-robin, on the other hand, has lots of games, and not many critical... but precisely this means any single game has less impact, and thus the selection of the champion is more exact.)
Quote:
Inexactness is going to be present with any method of choosing a challenger, for example an inferior bot winning the Computer Championship by an upset, so unless the problem is worse than straight triple-elimination among bots, the inexactness doesn't bother me so much.  But unfairness bothers me much more.  Can you expand more on why you think it is unfair?

Inexactness is going to be present with any method; however, the amount of it depends on which method is used.  Again, what is the aim of the contest?  To find a good approximation with the fewest number of games (in which case, triple-elimination works well), or to find a better approximation even if it takes a huge number of games (which would be my preference).
 
The unfairness in my eyes is simply the lack of control around the structure.  Unfairness might be too strong a word... I see it as a large amount of inexactness beyond the influence of the competing bots.
IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: 2007 Computer Championship and Challenge
« Reply #14 on: Oct 16th, 2006, 11:36am »
Quote Quote Modify Modify

Thanks for explaining, unic.  I do see a point about how catering to spectators can make a game less interesting.  As a baseball fan, I hate the way parks have moved the fences in.  They think more home runs means more excitement, and that that will keep the fans happy.  I think the home run is the most boring play in basball, apart from a strikeout or a walk.
 
For a walk, a strikeout, or a home run, there are seven players standing out in the field (plus anyone on base) doing nothing.  Even a pop-up or a grounder is more exciting because fielders and baserunners have to make decisions, and have the possibility of errors.  What would put the excitement into baseball is to move the fences back 200 feet.  That would keep every ball in play, and make home runs (inside-the-park) truly exciting.
 
As an Arimaa spectator, I would hate for the match conditions to take strategy out of the game.  I would definitely not want Challenge games played at blitz speeds.  I would not want games to be primarily decided by who hung a piece, rather than by who had the better plan.  
 
Also, as a spectator, I like to have time to see what the plans are and understand them a bit between moves.  Maybe chess grandmasters could play a blunder-free game at ten seconds per move, containing deep strategies and profound moves, but I sure couldn't appreciate it at ten seconds per move.
 
In short, I do appreciate the argument for a slower time control.  This is, however, quite separate from the issue of giving players total control of the management of their time.  I don't like the way chess players blitz through their opening moves to save up time for the midgame, because that doesn't let me savor their options in the opening.  I don't like how often players have to blitz to get to time control because they thought too long in the midgame.  It is optimal for my enjoyment as a spectator if the moves are spaced evenly one to three minutes apart: enough time to analyze and enjoy, but not enough for boredom.  That's why, even at a slow time control, I favor the other parameters to encourage play at a steady pace.
 
on Oct 14th, 2006, 9:57am, unic wrote:

If they want to watch, then that's good... if they don't want to watch, well, I don't see it as a problem if there are no spectators.

Here I totally disagree.  It does matter if people like to watch the game, because it indirectly has a huge impact on my opportunity to enjoy the game.  An absence of fans of the game means no money and fewer opponents to play.  I know you understand this problem perfectly well yourself: you said in another post that you know of "better" games where you can't find opponents, thus you are here and not there.  Maybe if those "superior" games had a greater focus on entertainment, they would be more popular than Arimaa.
 
Quote:
So... I guess the question is - what is the focus of the competition?  To find out the best player, or to provide entertainment?
This is not a good dichotomy.  Finding out the best player is entertaining.  I try to win because it is fun.  It is fun because I'm trying to win.
 
If you focus 100% on entertainment, then I agree with you, the game will lose its focus on skill.  It might be entertaining to have secret landmines planted it the board, or to roll dice to determine how many steps you get to take, or other such silliness.  But things that take skill out of the game ultimately take the fun out of the game too, and we would be left more bored and less entertained than before.
 
On the other hand, if you focus 100% on skill, then you can squeeze the fun out in other ways.  Maybe there would be more skill on a 12x12 board than on an 8x8 board, but the games would take days to play, and that wouldn't be fun.  Maybe there would be more skill if there were less capturing, but that might again slow the game down to where it takes 200 moves to win, and thus is supremely boring.  You just can't leave the fun out of the equation, or nobody will play your game that is an awesome test of skill.
 
Taking a principled stand that skill is the only thing that matters would be just bad for Arimaa as saying that entertainment is the only thing that matters.  Neither is ultimately fulfilling.  In my mind we're trying for a balance.  Yes, limiting each move to a 5-minute maximum squeezes a tiny bit of skill out of the game.  On the other hand (for the vast majority of us) allowing a single move to take half an hour squeezes a whole lot of fun out of the game.  A five-minute per-move limit is good for Arimaa on balance.
 
Quote:
Unfairness might be too strong a word... I see it as a large amount of inexactness beyond the influence of the competing bots.
That makes sense.  At some point if it is too inexact it becomes unfair on some higher level.  It's unfair in some way whenever the best player doesn't win, even if all players are competing in equal conditions.
« Last Edit: Oct 16th, 2006, 12:29pm by Fritzlein » IP Logged

Pages: 1 2  Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print

« Previous topic | Next topic »

Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB © 2000-2003. All Rights Reserved.