Author |
Topic: State of the Challenge 2012 (Read 6265 times) |
|
Fritzlein
Forum Guru
Arimaa player #706
Gender: 
Posts: 5928
|
 |
State of the Challenge 2012
« on: Jan 16th, 2012, 8:04pm » |
 Quote  Modify
|
It has been a while since we have had a general discussion of the state of the Arimaa Challenge, where everyone pitches in with perceptions of the current man/machine balance and prognostications for when computers will surpass us. Here are some predictions from 2009
My own optimism that humans would defend the Challenge through 2020 is gradually eroding. I now put the chance of Omar having to pay out his $10,000 prize at 30%. Sharp and marwin each had performance ratings a bit over 2100 in last year's qualifying, and I expect the top bot to reach 2200 this year. Sharp might reach that level purely from parallelization; it won the Computer Championship last year on a single core!
Meanwhile humans seem not to have been forging ahead as anticipated. Three years ago I expected new players to emerge who would push the envelope of our strategic understanding of Arimaa. Tuks did manage to beat me out for third place in the 2011 World Championship, but the WHR ratings still show chessandgo, Adanac, and me ahead of Hippo, Nombril, Tuks, and rabbits. If chessando is still only around 2600 strength, humanity has a scant 400-point advantage over the bots. (One could argue for chessandgo being over 2700 based on WHR, but his gameroom rating is below 2500, so the evidence balances out.)
What does everyone else think? Will we hold out past 2020? Am I still underestimating the computers? Please share.
|
| « Last Edit: Feb 1st, 2012, 10:43am by Fritzlein » |
 IP Logged |
|
|
|
Adanac
Forum Guru
Arimaa player #892
Gender:
Posts: 635
|
|
Re: State of the Challenge 2012
« Reply #1 on: Jan 17th, 2012, 9:37am » |
Quote Modify
|
on Jan 16th, 2012, 8:04pm, Fritzlein wrote:| It has been a while since we have had a general discussion of the state of the Arimaa Challenge, where everyone pitches in with perceptions of the current man/machine balance and prognostications for when computers will surpass us. Here are some predictions from 2009 |
|
Yes, it’s becoming apparent that bots will overtake humans much sooner than many people, myself included, originally anticipated. Somewhere between 2018-2024 is my current guess. Humans are improving steadily, and I’m impressed by how much deeper our level of understanding is now versus, for example, back in 2008 – especially considering how small the Arimaa community is. But bots are obviously improving far faster than humans are. Specifically, I’m impressed at how well Marwin & Sharp judge when to use/not to use a camel in an attack or counter-attack, the key moment to advance a rabbit to create goal pressure, when to swarm, and anticipating when/where to preemptively blockade a key defensive square. These are all areas of the game where I feel that the top bots are much stronger than they had been 2-3 years ago. I haven't played as many games against Briareus or Clueless recently but they're getting tougher every year too. I’m still amazed at how easy it is to swindle bots with obvious goal attacks, though. Hopefully the machines will still have a few glaring weakness like that at the end of this decade 
It’s great that we’ve seen the emergence of a new crop of high-level human players in the past few years, but we’ll need a much deeper pool of Arimaa players in the coming years. I expected we’d have more 2600+ rated players by now and if we don’t get any new prodigies soon there will eventually be more and more pressure on Fritz & Jean every year during the Arimaa Challenge (assuming we’re fortunate enough that they can still devote their time to defending the Challenge in the future).
|
|
IP Logged |
|
|
|
Hippo
Forum Guru
Arimaa player #4450
Gender:
Posts: 883
|
|
Re: State of the Challenge 2012
« Reply #2 on: Jan 17th, 2012, 10:00am » |
Quote Modify
|
Yes I expect the challenge to be beaten till 2020 and I am going to join machine side  . I hope the challenge will last at least to 2018 to have motivation to make progress of my bot as well. I still hope alpha-beta is not good enough for Arimaa and changes in search methods are required.
|
|
IP Logged |
|
|
|
mistre
Forum Guru
Gender:
Posts: 553
|
|
Re: State of the Challenge 2012
« Reply #3 on: Jan 17th, 2012, 10:11am » |
Quote Modify
|
on Jan 17th, 2012, 9:37am, Adanac wrote:
I expected we’d have more 2600+ rated players by now and if we don’t get any new prodigies soon there will eventually be more and more pressure on Fritz & Jean every year during the Arimaa Challenge (assuming we’re fortunate enough that they can still devote their time to defending the Challenge in the future).
|
|
It might be too early to rule Boo out as a prodigy. He has only been playing for 7 months and a little over 200 games and is already ranked 11th in WHR for active players...
|
|
IP Logged |
|
|
|
Tuks
Forum Guru
Arimaa player #2626
Gender:
Posts: 203
|
|
Re: State of the Challenge 2012
« Reply #4 on: Jan 17th, 2012, 2:23pm » |
Quote Modify
|
and annoyingly, beats me all the time, haha, i expect he will lead us to break the top three this year, watch out!
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: State of the Challenge 2012
« Reply #5 on: Jan 17th, 2012, 11:32pm » |
Quote Modify
|
If there isn't a breakthrough in the AI software like what's happened in Go with Monte Carlo methods, then I think there is a 50% chance of the challenge being won by 2020 just from incremental improvements in hardware and software.
But the challenge match games will get harder and harder each year to the point where I might have difficulty finding defenders willing to play. Also there is no grantee that the current top players will still be interested in Arimaa until 2020. I think a lot will depend on our ability to attract young players with good potential.
|
|
IP Logged |
|
|
|
Boo
Forum Guru
Arimaa player #6466
Gender:
Posts: 118
|
|
Re: State of the Challenge 2012
« Reply #6 on: Jan 18th, 2012, 2:30am » |
Quote Modify
|
Ahh those nice words Thanks!
I am still having fun exploring the game and trying out the new things.
|
|
IP Logged |
|
|
|
megajester
Forum Guru
Istanbul, Turkey
Gender:
Posts: 710
|
|
Re: State of the Challenge 2012
« Reply #7 on: Jan 18th, 2012, 6:06am » |
Quote Modify
|
on Jan 18th, 2012, 2:30am, Boo wrote:Ahh those nice words Thanks!
I am still having fun exploring the game and trying out the new things. |
|
Ahh, look at him, so innocent and cuddly. Just you wait till he gets bigger... GWWRR0O0O0AO0A0OAO0ARRR! RUN MEN, RUN FOR YOUR LIVES!!
|
|
IP Logged |
|
|
|
chessandgo
Forum Guru
Arimaa player #1889
Gender:
Posts: 1244
|
|
Re: State of the Challenge 2012
« Reply #8 on: Jan 18th, 2012, 8:10am » |
Quote Modify
|
PS: SPQA, nice!!
|
|
IP Logged |
|
|
|
megajester
Forum Guru
Istanbul, Turkey
Gender:
Posts: 710
|
|
Re: State of the Challenge 2012
« Reply #9 on: Jan 19th, 2012, 8:08am » |
Quote Modify
|
on Jan 18th, 2012, 8:10am, chessandgo wrote:
PS: SPQA, nice!! |
|
Ave Chessandgoix the Gaul, morituri te salutant! (Aut non...)
|
|
IP Logged |
|
|
|
chessandgo
Forum Guru
Arimaa player #1889
Gender:
Posts: 1244
|
|
Re: State of the Challenge 2012
« Reply #10 on: Jan 19th, 2012, 4:37pm » |
Quote Modify
|
Hehe!
I do hope you won't die, indeed (hoping I understood your sentence correctly, that is )
|
| « Last Edit: Jan 19th, 2012, 4:39pm by chessandgo » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: State of the Challenge 2012
« Reply #11 on: Feb 1st, 2012, 8:07am » |
Quote Modify
|
The new top bots whipped the old top bots in their qualifying games. Look at the current gameroom rating list!
2443 chessandgo
2410 Fritzlein
2392 Adanac
2389 bot_marwin
2380 bot_sharp
2303 99of9
2291 bot_briareus
I remember when I first learned about Arimaa in the summer of 2004. I read on Wikipedia and on the arimaa.com home page how this new game was supposedly difficult for computers to play. Then I checked the rating list of the game room, where only three players (99of9, omar, and Belbo) were rated higher than the top bot (bot_bomb). I immediately put a big discount on the claims of computer-resistance, and wondered whether a computer would win the Arimaa Challenge in just a couple of years.
Almost eight years later I am experiencing creepy deja vu. Back in 2004, I eventually learned that the situation was much more subtle than my first impression, because humans had resources that had not yet been tapped. We are much more accustomed to computer progress than human progress, so the latter sometimes takes us by surprise. Also four years of stagnation of the top bot proved that progress on that side was not inevitable.
Now that we have had a few years of amazing bot progress and relative (although certainly not complete) stagnation at the top of the human list, I am forced to wonder what untapped resources we humans have left. We simply must start learning additional strategies to defend the Arimaa Challenge, or be overrun while we stand still. Where will this inspiration come from? From new players? From fencing with the new, stronger bots?
I don't know what is going to happen next, but I would definitely forgive newcomers for having the same first impression that I had, namely that claims of Arimaa's computer resistance are overblown, and it will be just a couple of years before silicon rules.
|
|
IP Logged |
|
|
|
rbarreira
Forum Guru
Arimaa player #1621
Gender:
Posts: 605
|
|
Re: State of the Challenge 2012
« Reply #12 on: Feb 1st, 2012, 10:14am » |
Quote Modify
|
on Feb 1st, 2012, 8:07am, Fritzlein wrote:Look at the current gameroom rating list!
2443 chessandgo
2410 Fritzlein
2392 Adanac
2389 bot_marwin
2380 bot_sharp
2303 99of9
2291 bot_briareus |
|
I definitely see your points Fritzlein (nice thread btw), however it should be pointed out that these bots have been playing a lot in Blitz and Fast time controls, which probably over-rates them somewhere between 100-200 points.
This indicates that at the 2 minutes time control, the top bot this year may perform between 2200 and 2300 (my guess would be closer to 2300).
|
| « Last Edit: Feb 1st, 2012, 10:16am by rbarreira » |
IP Logged |
|
|
|
Boo
Forum Guru
Arimaa player #6466
Gender:
Posts: 118
|
|
Re: State of the Challenge 2012
« Reply #13 on: Feb 1st, 2012, 10:29am » |
Quote Modify
|
Quote:| This indicates that at the 2 minutes time control, the top bot this year may perform between 2200 and 2300 (my guess would be closer to 2300). |
|
Yeah, it would be interesting to compare human/bot ratings including only 2 min games.
2min games is such a different story compared to 30s.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: State of the Challenge 2012
« Reply #14 on: Feb 1st, 2012, 10:33am » |
Quote Modify
|
on Feb 1st, 2012, 10:14am, rbarreira wrote:| This indicates that at the 2 minutes time control, the top bot this year may perform between 2200 and 2300 (my guess would be closer to 2300). |
|
Yes, the bot ratings at two minutes per move will be lower, and I stand by my estimate that it will be close to 2200 for the top bot even though I was shocked by the latest rating list. The influence of the time control is one of the things I didn't immediately grasp when I saw the rating list in 2004 either. (The top bot for a while was actually not bot_bomb but bot_speedy, which was bomb playing at a faster time control.) In addition to the time control, I expect that the game room ratings of the top humans are a bit low at present compared to the general scale of the game room ratings. I expect that if the seven players at the top of the list played a twenty-fold round-robin even at blitz, the humans would gain net rating points.
That said, deciding whether the human advantage under Challenge conditions is presently 200 points or 400 points is less important in the long run than the relative rate of change. Even if I am right that the current gap is 400 points, I believe bots will make up that gap in eight years if humans stand still. If we are to defend the Arimaa Challenge, we simply must raise our game to a new level.
I personally believe that I am capable of playing at a substantially higher level than I do now, but achieving that goal stands in conflict with passing my classes. 
|
| « Last Edit: Feb 1st, 2012, 10:40am by Fritzlein » |
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print