Author |
Topic: Arimaa Challenge format for 2006 (Read 4640 times) |
|
clauchau
Forum Guru
bot Quantum Leapfrog's father
Gender: 
Posts: 145
|
 |
Re: Arimaa Challenge format for 2006
« Reply #15 on: Aug 18th, 2005, 9:48am » |
 Quote  Modify
|
I like the change Omar is making. I see flaws in any format, but I like the new flaws slightly better 
Well, I have subjective reasons. When I think how inconsistent and so totally subject to pressure I can be, while we know bots still are stronger on that regard, I feel released for the human players.
I'm not satisfied with the match between Deep Blue and Kasparov. Chess AI in general almost fulfilled its dreams and mine, at least statically. But what with Kasparov big act, his high level as a strategist in real life, the million he got, the chess experts unsettled? I know it's bad to question a past event and Kasparov, whom I am fond of. But about that event, I can't help.
Asking for a panel of players instead of one single judge, although less of a drama, gets us closer to collective and future acceptance of the result. The Loebner Prize awarding the first AI passing the Turing Test has a panel of judges too.
on Aug 17th, 2005, 2:56am, omar wrote:| it takes human many years to master complex strategy games. |
|
Well, human aren't that brilliant then and Arimaa isn't that interesting a challenge for AI? 
|
|
 IP Logged |
|
|
|
clauchau
Forum Guru
bot Quantum Leapfrog's father
Gender:
Posts: 145
|
|
Re: Arimaa Challenge format for 2006
« Reply #16 on: Aug 18th, 2005, 11:35am » |
Quote Modify
|
on Aug 15th, 2005, 6:43pm, 99of9 wrote:| I thought we were claiming that arimaa was orders of magnitude more difficult for computers than chess? |
|
By the way Arimaa seems to me now easier a game than I thought in the beginning. It is complex, harder than Chess, but BotBomb is not that lost at the elementary level I thought early bots would. In any case I still think Arimaa is challenging to AI, but maybe only of "one" order of magnitude.
|
|
IP Logged |
|
|
|
clauchau
Forum Guru
bot Quantum Leapfrog's father
Gender:
Posts: 145
|
|
Re: Arimaa Challenge format for 2006
« Reply #17 on: Aug 18th, 2005, 12:33pm » |
Quote Modify
|
on Aug 17th, 2005, 11:17am, Fritzlein wrote:| But now, if the challenge is spread across three weeks, each human only has to play one game per week. (The bot presumably doesn't get tired, so it can play three times per week.) |
|
Indeed that's not fair in case the bot needs as much time as humans to think over the past games and evolve before the next game. I hope AI will get to that point soon. In that case the bot shall be allowed as many running clones as there are human opponents.
Hey, about being fair, the clones shall be kept informed of every recent Arimaa games.
And how about letting the bot know its opponents' identities on each game? Humans know whom they are playing against.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Arimaa Challenge format for 2006
« Reply #18 on: Aug 18th, 2005, 6:15pm » |
Quote Modify
|
on Aug 18th, 2005, 12:33pm, clauchau wrote:
Indeed that's not fair in case the bot needs as much time as humans to think over the past games and evolve before the next game. I hope AI will get to that point soon. In that case the bot shall be allowed as many running clones as there are human opponents.
Hey, about being fair, the clones shall be kept informed of every recent Arimaa games.
And how about letting the bot know its opponents' identities on each game? Humans know whom they are playing against. |
|
The bots are allowed to run between games to analyze the game.
Also the Arimaa games archive is available for download so bots can make use of the recent games.
The name of the opponent is available to the bot as part of the game state.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Arimaa Challenge format for 2006
« Reply #19 on: Aug 18th, 2005, 6:48pm » |
Quote Modify
|
on Aug 17th, 2005, 10:05pm, Fritzlein wrote:
Unfortunately, I think that making the challenge more difficult to achieve will (A) not prevent the public from losing interest in Arimaa if a bot wins a simple majority in a match, and (B) send the signal that computers might be better than humans already without any AI breakthroughs.
|
|
I've always thought that as long as there was even one human player who could defeat the computer, then humanity wins. I think that the general public looks at it that way too.
Suppose Kasparov, Anand and Kramnik played Deep Blue (or some strong chess program) and the result was:
Anand: 0 0 0
Kramnik: 0 0 0
Kasparov: 1 1 1
The headlines would go crazy about how Kasparov defended humanity. No one would care that Anand and Kramnik got demolished.
I think the public looks at each players record independent of others. They would think that Anand and Kramnik aren't as good, but Kasparov is certianly better. The public doesn't care about the average performance of the humans; they just want to know if there is at least one that is better.
Suppose the result turned out as:
Anand: 0 0 0
Kramnik: 1 1 0
Kasparov: 1 1 0
And the computer is declared the winner since it won more total games. Now have fun trying to convince the average person that humans aren't better than the computer
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: Arimaa Challenge format for 2006
« Reply #20 on: Aug 18th, 2005, 7:45pm » |
Quote Modify
|
If I could interject a few words as an interested spectator. Of course having no other qualifications, feel free to ignore it.
Personally what I would find most interesting is simply entering computers into the world championship along side, and under the same rules as, humans. Especially once, as I imagine will happen within the next few years, a minimum rating is required to enter. In this regard I was happy to see bomb included in the recent postal tournament. But in regards to the challenge I can understand the practicality of a stand alone event in terms of excitement, PR, etcetera.
One of the things that has always bothered me about human vs. machine chess events is the frequent imbalance of game history available to the two sides. Almost always the computer team will have thousands of past games to study from the human opponent. Whereas usually the human has very few, often less than a hundred from the current version or even total. I think this may be even more detrimental in Arimaa where exploration is still continueing in basic strategies. So both lowering the prerequisite number of games and having 3 game matches (even though there are 3 of them) makes me very nervous. I would much rather see an expansion in the prerequisite games rather than any reduction. I can understand time requirements not allowing longer matches, in return it would be nice to have some optional practice games to help make up for this though. Maybe ideally something like the two weeks before the challenge the bot is available for reservation by the challengers to play games against.
These changes would make it harder for the computer to win. But I'm afraid a not unlikely scenario with the current rules is for the humans to lose the challenge when presented with a novel strategy, resulting in a loss of interest. But then the same bot to be regularly and easily beaten shortly afterwards when it's flaws are discovered.
Janzert
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Arimaa Challenge format for 2006
« Reply #21 on: Aug 18th, 2005, 8:52pm » |
Quote Modify
|
on Aug 18th, 2005, 6:48pm, omar wrote:| I've always thought that as long as there was even one human player who could defeat the computer, then humanity wins. I think that the general public looks at it that way too. |
|
You may be right about public perception. I guess we won't know until word about the new challenge format gets around (probably not before the challenge is actually played again) and the reactions start to trickle in.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Arimaa Challenge format for 2006
« Reply #22 on: Aug 20th, 2005, 10:08pm » |
Quote Modify
|
on Aug 17th, 2005, 4:10pm, omar wrote:
With regard to soliciting sponsors to help increase the challenge prize. I do plan to still do that, but only after the format is definitely fixed.
|
|
If I were a sponsor I would want the rules to fixed in stone before putting up any money, so this seems like a good idea.
Quote:
One thing I was considering is allowing a sponsor to set their own time frame for the amount they are offering. For example a sponsor could say they are offering $5,000 until the year 2010. So the total challenge prize for the current year would then be $15,000.
|
|
What are you willing to give in exchange for sponsorship? I'll bet you could get $100,000 if you were willing to go all the way with the sponsor on a 15-year contract. Intel, for example, is a likely candidate to put up the money, if you would do things such as
1) Rename it the "Intel Arimaa Challenge" and agree to refer to it that way on all your Web pages and press releases
2) Agree to have the challenge bot play on an Intel chip every year, so that if the challenge is awarded, Intel also wins (i.e. it would a bot running on Intel which beat down the humans)
3) Have a banner advertisement to Intel from the arimaa.com homepage, the gameroom, and all the other pages in arimaa.com.
Perhaps some corporations want to be associated with something just because it is cool, but probably to get any serious interest you would have to splash their name everywhere on any property you control.
I personally wouldn't mind if Arimaa were "sold out" in this way, because the increase in prize money and the backing of a major corporation would bring a bunch of new attention to the game. I could handle looking at a few advertisements if it helps the game take off even more than it already has for such a young game. Whatever it takes to spread the word, I'm in favor.
(By the way, Intel briefly was a big sponsor of chess until Kasparov bucked them to play Deep Blue. Obviously Intel was not pleased that IBM got all the good press, and Intel quitely totally withdrew their sponsorship. Despite being burned in that way, Intel might still be interested in computer gaming.)
Another thing to propose to make it seem less risky to the sponsor is to ask them to chip in $10,000 per year for each year that the challenge isn't won. Then they don't have to worry about putting up a huge amount of money for only one or two years of exposure.
|
| « Last Edit: Aug 21st, 2005, 8:12am by Fritzlein » |
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender:
Posts: 1413
|
|
Re: Arimaa Challenge format for 2006
« Reply #23 on: Aug 21st, 2005, 4:19am » |
Quote Modify
|
on Aug 21st, 2005, 2:43am, Arimanator wrote:| I don't think it's good strategy to mock a potential sponsor. |
|
Of course you are right that we should be careful about what we say about potential sponsors, but if it helps, I don't think Fritz's comments are mocking at all.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Arimaa Challenge format for 2006
« Reply #24 on: Aug 21st, 2005, 8:56am » |
Quote Modify
|
on Aug 21st, 2005, 2:43am, Arimanator wrote:Dear Fritz, you know that I esteem you and so there's nothing personal in my remark but I don't think it's good strategy to mock a potential sponsor.
|
|
I'm not sure what part of my post seems to be mocking Intel; perhaps they would freely admit that their motivation for sponsoring Arimaa would be advertising exposure more than just backing something cool. And I wasn't trying to laugh at them for being burned by Kasparov. On the contrary my sympathies are rather with Intel than the Professional Chess Association, and it was in my mind when I included that post-script that it is simply good for a business relationship to respect what your sponsor wants, as Kasparov failed to do.
On the other hand, I admit that I was writing for a local audience and didn't consider how it might read to a potential sponsor. It's good to remember that the forum is public, and the folks at Intel (or anywhere) can read it too. Thanks for that reminder, and I'll try to imagine someone from Intel reading each of my posts before I submit them in the future.
Quote:| [...]I'd say that $100000 is the equivallent of a fraction of a second of commercial time on a prime channel during a major Basket ball game[...] |
|
Yes, it is precisely because advertising is so expensive that I think the money is out there for Arimaa. While I'm rambling on the subject, I should mention that if Intel were the sponsor (or Dell or Hewlett Packard or anyone who would want the challenge bot running on their brand of hardware) they might actually want the challenge money to be awarded, because it would mean that their hardware would win. So among the things we could give to Intel would be
4) Allow the bots to run on as fast a machine as Intel will provide us for free, and mention everywhere what an awesomely powerful computer that is (e.g. the bot is considering more nodes per second on an Intel system than Deep Blue was against Kasparov!). Imagine how good that would make a hardware sponsor feel.
I know that this runs contrary to Omar's intentions that the challenge be won by better software rather than faster hardware, but I'm just trying to look at it from the perspective of the sponsor. A computer hardware sponsor would want the hardware to play as large a role as possible, and they might be hoping for humanity to lose even if that meant forking over $100,000 they could otherwise keep.
So it gets back to the question of how much Omar is willing to cooperate with the sponsor in exchange for money and other support. To drum up Intel's enthusiasm, we'd probably have to make it Intel's game. But if we had their enthusiasm, they might do more than just sponsor the challenge, they might also give free hardware for the Arimaa server and as prizes for the Computer Championship matches, etc. It's a tradeoff that could have large benefits for the Arimaa community, although it would require some faith from Omar that a 100-fold hardware speedup wouldn't be enough to put the bots ahead of the humans.
|
| « Last Edit: Aug 21st, 2005, 9:08am by Fritzlein » |
IP Logged |
|
|
|
Arimanator
Forum Guru
Arimaa player #1064
Gender:
Posts: 158
|
|
Re: Arimaa Challenge format for 2006
« Reply #25 on: Aug 21st, 2005, 9:54am » |
Quote Modify
|
on Aug 21st, 2005, 8:56am, Fritzlein wrote:
....although it would require some faith from Omar that a 100-fold hardware speedup wouldn't be enough to put the bots ahead of the humans.
|
|
To be perfectly honest I wouldn't stake my life on it  , especially if the bots are ones that never played humans in the open before and harbor some hidden new improvements.
|
| « Last Edit: Aug 21st, 2005, 9:55am by Arimanator » |
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: Arimaa Challenge format for 2006
« Reply #26 on: Aug 21st, 2005, 9:59pm » |
Quote Modify
|
on Aug 21st, 2005, 8:56am, Fritzlein wrote:| 4) Allow the bots to run on as fast a machine as Intel will provide us for free, and mention everywhere what an awesomely powerful computer that is (e.g. the bot is considering more nodes per second on an Intel system than Deep Blue was against Kasparov!). Imagine how good that would make a hardware sponsor feel. |
|
Now imagine how good it would make a hardware sponsor feel if their off-the-shelf hardware were able to consider more nodes per second than Deep Blue! Even that should be possible by 2020, shouldn't it?
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: Arimaa Challenge format for 2006
« Reply #27 on: Aug 21st, 2005, 10:03pm » |
Quote Modify
|
on Aug 21st, 2005, 8:56am, Fritzlein wrote:| it would require some faith from Omar that a 100-fold hardware speedup wouldn't be enough to put the bots ahead of the humans. |
|
Supercomputers nowadays have many more than 100-fold better performance than a PC...
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Arimaa Challenge format for 2006
« Reply #28 on: Aug 22nd, 2005, 10:05pm » |
Quote Modify
|
on Aug 20th, 2005, 10:08pm, Fritzlein wrote:
1) Rename it the "Intel Arimaa Challenge" and agree to refer to it that way on all your Web pages and press releases
|
|
I actually did call Intel and ADM and talked to the heads of their marketing departments shortly after releasing Arimaa to see if they might be interested in sponoring the challenge.
I found out that big companies don't like to sponsor things that aren't already well known and established.
It's the chicken or the egg kind of situation.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: Arimaa Challenge format for 2006
« Reply #29 on: Aug 22nd, 2005, 10:45pm » |
Quote Modify
|
on Aug 21st, 2005, 10:03pm, 99of9 wrote:
Supercomputers nowadays have many more than 100-fold better performance than a PC... |
|
I would say the top supercomputer is maybe even 10,000 fold better performance than a PC. There is already a difference of 100 fold between the high end supercomputers and the low end supercomputers and the low end supercomputers have 100's of processors that used in todays PCs. See:
http://www.top500.org/lists/plists.php?Y=2005&M=06
Im pretty sure that a supercomputer running an Arimaa program could definitely beat todays players. I would not have been so sure about it when I was first releasing Arimaa, since at the time I was not considering the "newness" factor.
But the point of the Arimaa challenge is not to encourage building something that can only be accquired by national labortories and big companies or which can easily disappears after the demonstration. Rather I would like to use the challenge to encourage work on AI software.
I am a strong beleiver that even the ordinary PC's of today already have enough power to beat even the best human players of 2020 if given the right software. Just that it might take a lot of computing power to produce that software
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print