Author |
Topic: 2007 qualifying for computer championship (Read 4012 times) |
|
acheron
Forum Full Member
Arimaa player #1613
Gender: 
Posts: 11
|
 |
Re: 2007 qualifying for computer championship
« Reply #15 on: Nov 18th, 2005, 12:21pm » |
 Quote  Modify
|
If you want to boost potential prize money I wouldn't look to Omar to provide it.
The real trick is to acquire corporate sponsors. These aren't going to be interested right now because no one has ever heard of Arimaa. Which then leads us to step one - acquire publicity.
Someone with a good mind for exposure could probably come up with an appropriate targeted list for media contact. Generate some press in the right circles and paint Arimaa as a more mainstream activity and corporate sponsorship will follow.
|
|
 IP Logged |
|
|
|
PMertens
Forum Guru
Arimaa player #692
Gender:
Posts: 437
|
|
Re: 2007 qualifying for computer championship
« Reply #16 on: Nov 18th, 2005, 3:57pm » |
Quote Modify
|
Quote:| It seems that bot development has cooled off a bit from last year, but it is probably cyclical and will pick up again without any structural changes. |
|
let's see what can be done about that 
|
|
IP Logged |
|
|
|
nbarriga
Forum Guru
Almost retired Bot Developer
Gender:
Posts: 119
|
|
Re: 2007 qualifying for computer championship
« Reply #17 on: Nov 18th, 2005, 4:44pm » |
Quote Modify
|
on Nov 18th, 2005, 12:21pm, acheron wrote:
Someone with a good mind for exposure could probably come up with an appropriate targeted list for media contact. Generate some press in the right circles and paint Arimaa as a more mainstream activity and corporate sponsorship will follow. |
|
One word: Slashdot.
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender:
Posts: 1413
|
|
Re: 2007 qualifying for computer championship
« Reply #18 on: Nov 18th, 2005, 6:57pm » |
Quote Modify
|
on Nov 18th, 2005, 9:39am, Fritzlein wrote:| It seems that bot development has cooled off a bit from last year, but it is probably cyclical and will pick up again without any structural changes. |
|
Actually I think there has been more bot development this year than ever before. It's just that the development hasn't been in the top bots. Haizhi, Weiser, and Aamira!
I agree us old-timers haven't put in the hard yards this year, but even that may pick up in December, as we all seem to be last minute people.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2007 qualifying for computer championship
« Reply #19 on: Nov 18th, 2005, 11:55pm » |
Quote Modify
|
on Nov 18th, 2005, 6:57pm, 99of9 wrote:
Actually I think there has been more bot development this year than ever before. It's just that the development hasn't been in the top bots. Haizhi, Weiser, and Aamira! |
|
That's true, there has been a lot of development. I guess I was forgetting Haizhi and Aami-ra because it looks like they aren't spending their qualifying time on line. I'm happy Weiser is still around, though, and still under active development.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2007 qualifying for computer championship
« Reply #20 on: Nov 19th, 2005, 10:53am » |
Quote Modify
|
on Nov 18th, 2005, 4:44pm, nbarriga wrote:
The server would probably crash though
Although this new server seems to be running pretty smooth even when there are a lot of people in the gameroom. I checked the load on the server during my WC game against Paul and the load was only like 0.5; I think there was about 12 to 14 people logged in at that time.
The load actually goes up when people start playing the bots that are also running on the server.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2007 qualifying for computer championship
« Reply #21 on: Nov 19th, 2005, 10:55am » |
Quote Modify
|
on Nov 16th, 2005, 11:50pm, Janzert wrote:
While I really like this idea from the pure "let's best try and determine if bots or humans are better at arimaa" point of view, I think it has a number of shortcomings when looking at ancillary concerns.
Probably top most of these would be, for spectators there is no annual "Grand match" focal point. Probably resulting in greater difficulty getting more sponsors as well.
Janzert |
|
Precisely. I would like to preserve that annual Grand Match nature of the current structure.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2007 qualifying for computer championship
« Reply #22 on: Nov 19th, 2005, 11:05am » |
Quote Modify
|
on Nov 16th, 2005, 6:20pm, Fritzlein wrote:
Really? How would the bot know whether it was playing a challenge game or a pactice game? If the bot bases its decision of how to play on the date, you could alter the system clock or something to prevent that. I'm obviously not very savvy with this kind of thing, but I'm surprised that you don't think you could set it up somehow.
|
|
Exactly. The date could be used by the program to play differently. I would never alter the date on a computer because a lot of other programs depend on it being right; including the bot interface scripts.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2007 qualifying for computer championship
« Reply #23 on: Nov 19th, 2005, 11:38am » |
Quote Modify
|
on Nov 18th, 2005, 9:39am, Fritzlein wrote:
Good point. I wonder if bot developers would participate if it were set up as a side contest, independent of the Arimaa Challenge. We have the Player of the Month contest to encourage humans to play each other. In parallel there could be a regular Bot Contender prize to encourage developers to work on their bots year-round. The prize money could be split between the developer and the first player to garner a +20 score, depending on how long the bot survived. (There I go giving away Omar's money...) We could have a hall of fame so that defenders of human supremacy would gain eternal glory.
I'm just brainstorming here. The Arimaa server is already a way cool place to play. It seems that bot development has cooled off a bit from last year, but it is probably cyclical and will pick up again without any structural changes. |
|
It takes a lot of effort to develop a bot and even more to improve a bot that is already playing pretty good. So I don't think the amount of money I could offer would produce much encouragement. The bot developers are really doing it for other reasons. The two main motives I see right now are: as projects related to school work, and because developing game programs is their personal hobby. Even in Chess top programs like Shredder and Junior are developed by hobbiests. Although in chess they can probably make a living off the program.
What I've been meaning to do, but haven't done yet is to mail out a call for participation letter to the computer science dept of major universities asking them to develop a bot to represent their university in the annual Arimaa computer championship. This would help fuel Arimaa getting known about in the academic community. The professors might start mentioning Arimaa in their intro to AI classes and more students might choose to develop a bot for their project.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2007 qualifying for computer championship
« Reply #24 on: Nov 19th, 2005, 11:53am » |
Quote Modify
|
Returning back to the topic of this dicussion.
Lets state the goals of what the qualifier should acheive. Here is what I would like to shoot for:
1. Require the bots to produce some minimum game record with some minimum games being against humans.
2. Require the bots to show their best performance in these games.
3. Be easy enough that someone can qualify even if they started late in the year.
4. Be hard enough that even the well established and best bots need to make some effort.
|
| « Last Edit: Nov 19th, 2005, 11:59am by omar » |
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: 2007 qualifying for computer championship
« Reply #25 on: Nov 19th, 2005, 12:13pm » |
Quote Modify
|
on Nov 18th, 2005, 11:55pm, Fritzlein wrote:
That's true, there has been a lot of development. I guess I was forgetting Haizhi and Aami-ra because it looks like they aren't spending their qualifying time on line. I'm happy Weiser is still around, though, and still under active development. |
|
Also don't underestimate bot_bomb. Remember David can change it till the last minute. The publicly available database of the games and the match offline scripts now allow a developer to be able to improve their bot without it actualling needing to play games in the gameroom.
I think we might be suprised by how well bomb2006CC will actually play in the tournament and perhaps in challenge match 
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: 2007 qualifying for computer championship
« Reply #26 on: Nov 19th, 2005, 4:00pm » |
Quote Modify
|
on Nov 19th, 2005, 12:13pm, omar wrote:Also don't underestimate bot_bomb. Remember David can change it till the last minute. The publicly available database of the games and the match offline scripts now allow a developer to be able to improve their bot without it actualling needing to play games in the gameroom.
I think we might be suprised by how well bomb2006CC will actually play in the tournament and perhaps in challenge match |
|
We may well be surprised, but that is partly because to date a broken version of bomb has been in the gameroom.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2007 qualifying for computer championship
« Reply #27 on: Nov 20th, 2005, 9:10pm » |
Quote Modify
|
on Nov 19th, 2005, 11:53am, omar wrote:Returning back to the topic of this dicussion.
Lets state the goals of what the qualifier should acheive. Here is what I would like to shoot for:
1. Require the bots to produce some minimum game record with some minimum games being against humans.
2. Require the bots to show their best performance in these games.
3. Be easy enough that someone can qualify even if they started late in the year.
4. Be hard enough that even the well established and best bots need to make some effort.
|
|
Heheh, sorry I started to wander pretty far afield with suggestions for radically changing the structure of the Arimaa Challenge. I agree with the comments that a climactic match once a year is by far the best for publicity and eventual corporate sponsorship, so my above barinstorming notwithstanding, I wouldn't want to mess with that setup.
It is good, however, for us to be clear that the qualifying for the Computer Championship is, at the present, very much entangled with the goals for the Challenge. For example, the first condition about having some minimum game record against humans really has nothing to do with the Computer Championship, does it? What relevance do human games have to a bot versus bot contest?
If a bot competed in and won the Computer Championship without having any record whatsoever against humans, that would in no way undermine the integrity of the Computer Championship. Moreover, as long as that bot thereafter played enough games against humans before it played in the Challenge, it would in no way undermine the integrity of the Challenge that the game record was compiled after the Computer Championship rather than before.
Conditions 1 and 2 seem to be directed at the Challenge, while 3 and 4 seem directed more more towards the Computer Championship. Yet one might think, "Why not kill two birds with one stone, and have the conditions for a bot to qualify for one event be exactly the same as the conditions to qualify for the other event?" I think that's where the problem starts, and is what inspired my initial post in this thread.
For a bot to have a game record against humans is not only unnecessary for the Computer Championship, it is a competitive disadvantage. Another bot developer could create an opening book against the bot he wants to beat, basing it on the weaknesses exposed by human play. A bot might be hurt in the Computer Championship by its prior exposure to humans, even though getting bashed around by humans says nothing about the fitness of a bot to compete against other bots.
Thus it makes sense, in the qualifying for the Computer Championship, to remove any criterion of having a certain number of games against humans or of spending a certain amount of time on line accepting challenges. Let the developers choose how much to test their bots, and against which opponents. There is no need to begrudge developers this control as they compete with each other.
Why not wait until after the Computer Championship is over, or at least until the final code for the bots has been submitted, to mandate play against humans? At that point it can't make any difference in the Computer Championship how badly the bots get kicked around by humans. It will only make a difference in the Challenge, which is as it should be.
You say, Omar, that there is no way to prevent a developer from having his bot play differently in practice games than in the Challenge itself. Since that is true no matter when those practice games are held, this is no argument in favor of having the practice games later or sooner. Cheating can happen at any time. So let's put aside the question of cheating, assume the developers will play fair, and consider only the process of developing a bot. If it is important that a bot have a record of games against humans to qualify for the Challenge, isn't it equally important that those games be played by the bot in its final version? Otherwise last-minute improvements may make the entire game record useless to the humans. It's as if we would allow a developer to qualify for the Challenge by handing us a record of games compiled by some other bot.
Indeed, if you recall, Kasparov had played a previous match against Deep Blue, and had those games from which to study and prepare. However, Deep Blue had changed and improved so much that this experience was useless to Kasparov. The computer had the same name, but it wasn't the same computer, much as there is currently a bot online that is named Bomb without being Bomb.
Assuming the developers play fair, why not mandate the exposure of the final version of the bot, i.e. the actual bot that competes in the Computer Championship? Exactly that bot is going to play in the Challenge, so exactly that bot needs to have a game record against humans. As a side benefit, the bots that lose the Computer Championship wouldn't have to be forced to qualify for the Challenge. Only the bot that wins and will actually play in the Challenge needs to play humans. (Of course, all bots eventually become members of the CC fold of bots, but that is separate from the qualifying process.)
To recap, I would split the qualifying criteria of the Computer Championship from the qualifying criteria of the Challenge. Conditions 3 and 4, for the Computer Championship qualifying only, could be met by specifying five bots from previous championships and requiring two games against each of them at two miutes per move. Even a latecomer could qualify, but it shows some seriousness on the part of the developer to clear this hurdle.
Conditions 1 and 2, for the Challenge, could be met by exposing the final version of the bots (or only the winner) during the Computer Championship (or after) for whatever amount of time or number of games seems appropriate.
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: 2007 qualifying for computer championship
« Reply #28 on: Nov 20th, 2005, 10:55pm » |
Quote Modify
|
on Nov 20th, 2005, 9:10pm, Fritzlein wrote:| As a side benefit, the bots that lose the Computer Championship wouldn't have to be forced to qualify for the Challenge. Only the bot that wins and will actually play in the Challenge needs to play humans. |
|
I quite like your suggestion overall Fritz, and I don't have time now to prepare a proper response, but the quote above is one thing you should think about a bit more:
The issue is that the humans can focus all their attentions on finding ways to beat one particular bot, because they know which one they will play. Bots do not know their Challenge opponent(s) in advance, so either they have to be set up very generally, or they have to have separate tuning done for every different possible human.
Therefore I think you would be unintentionally skewing the Challenge a little further in the favour of humans.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2007 qualifying for computer championship
« Reply #29 on: Nov 21st, 2005, 10:22am » |
Quote Modify
|
on Nov 20th, 2005, 10:55pm, 99of9 wrote:The issue is that the humans can focus all their attentions on finding ways to beat one particular bot, because they know which one they will play. Bots do not know their Challenge opponent(s) in advance, so either they have to be set up very generally, or they have to have separate tuning done for every different possible human.
Therefore I think you would be unintentionally skewing the Challenge a little further in the favour of humans. |
|
I quite agree that this is an important issue of fairness, but it could be addresses by reducing the contending bot's exposure to human games. To tell you the truth, I think it is quite fair to limit the exposure to three practice games for each of the three defenders. I would still put up my money that the Challenge won't be won under those circumstances, but by the same token don't think bot developers would have anything to complain about in terms of humanity ganging up on their bot.
The reason I said "whatever amount of time or number of games seems appropriate" is that I wanted to write a focused post (for once) purely about separating the qualifying for the Computer Championship from the qualifying for the Challenge. When I got into questions of how to prevent cheating, or how to structure the challenge, or how much exposure to humans is appropriate, I was distracting from what I think is a quite useful suggestion, and I didn't want that central suggestion to be rejected due to my opinions on other issues.
|
| « Last Edit: Nov 21st, 2005, 10:23am by Fritzlein » |
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print