Arimaa Forum - Print Page


    
      
        Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> General Discussion >> bots are hard to beat
        
(Message started by: ocmiente on Feb 3^rd, 2012, 1:22pm)

Title: bots are hard to beat
Post by ocmiente on Feb 3^rd, 2012, 1:22pm

I've been playing Arimaa for almost three years now. My game room rating is currently over 2000. So, I feel somewhat qualified to write what follows.

Shortly after I started playing, in March of 2009, I gave up playing for 4 or 5 months because I just wasn't making progress learning new Arimaa strategies, and I wasn't inclined to ask anyone for help. It wasn't until Fritzlein's book came out that I was able to enjoy playing the game.

Even with the new knowledge of camel hostages, horse frames, elephant horse attacks, and the like, I would still get thrashed by bots from time to time. In fact, I still get thrashed by bots on a regular basis.

I think that there are many strong bots on the Arimaa site, and if it seems difficult to beat a particular bot, that's not surprising. Most people on the planet would probably have a hard time beating most of the bots in the Arimaa game room.

The reason I am posting this is because this thought has been stewing in my mind for a while: I found it a little frustrating at times when reading some post that a particular bot was brain dead (I'm exaggerating a little - but not much) while I was struggling to beat that particular bot more than 50% of the time. Bomb, in particular, is the one that bugged me for the longest time. If you are a new player, you should expect to be beaten by bots... a lot! There are few prodigies who can zip through the ladder, but those are few and far between. It may take 2 or more years before you can regularly beat all of the bots in the game room - and it may take longer than that since the bots are getting better.

What has kept me coming back for more beatings is that I have been able to see continual progress in my ability to play over the past couple of years, and it looks like I have at least a couple more years worth of things I can get better at.

Title: Re: bots are hard to beat
Post by Fritzlein on Feb 3^rd, 2012, 3:51pm

As a math student, I perpetually have a similar problem. Many things in math have an almost vertical learning curve, which is to say they are completely opaque and mysterious up until the point that you see what is happening, and thereafter they are trivially obvious. It is disheartening when someone who gets X says how easy X is when you are on the outside looking in, understanding nothing.

There is a real problem with language, because we need a way to distinguish between things that are obvious once you see them, and things that are still tricky and difficult after you have gotten the gist of them. (There are some of the latter in math too!) Unfortunately, we re-use the same words for that distinction that we also use to distinguish things that are difficult to gain an understanding of in the first place.

When I say a bot is brain-dead, I mean that there is something it doesn't get that is obvious to me. The bot seems even dumber because it never catches on. It will lose every game the same way. How can it never, ever see something that is so clear and so simple? But that doesn't mean it was easy for me to come by the strategic knowledge that is now second nature to me.

Relating this to the Arimaa Challenge, I won't be able to defend it for too many more years unless I learn some more of these "obvious" signposts. If I am trying to outduel a bot at something "hard", I will make more mistakes than the bot. My only hope of winning is to have understood something that bot doesn't understand at all, i.e. for my future self to think that the bot is brain-dead in some way, even if right now I can't beat that bot without a mighty struggle.

I sympathize with your frustration about bots being hard to beat in reality when so much of our talk is about how easy they are to beat. On the other hand, there are many bots you can now beat without breaking a sweat. It would just feel wrong if you were to go on about how tough it is to beat ShallowBlue. That sort of language would be just as misplaced, and would prompt someone else to open a thread entitled, "Bots are easy to beat!"

The difficulty is that the hard/easy distinction doesn't get us any closer to the gist of the matter. For me the fun of a game is in the strategic learning, and strategic learning is by its nature both hard and easy. It is hard to acquire, and it is easy to use once you have it. There's a song by the Indigo Girls with a lyric that I like: "The hardest to learn was the least complicated."

Title: Re: bots are hard to beat
Post by SpeedRazor on Feb 4^th, 2012, 1:36am

Wow, I really like this thread.

It's like searching for a door on an opaque wall, with little to no clues. Once you find it, and cross over, it's now mundanely easy for you. You now are the holder of the Hallowed Arcane Knowledge.

How often in history has this happened? Maybe by perspiration/inspiration; some weird intuition that even you don't understand; or just blind serendipity?

Mr. Firestone accidentally dropping latex into warm sulfuric acid? (Volcanized rubber) Albert Hoffman getting a tiny drop of a liquid - he surmises - from a weird ergot fungus on his skin? (LSD-25) [...] ...

You're right Ocmiente/Fritzlein, there needs to be terms for this. Does anybody have any ideas what, though? It should be catchy enough to become a meme, or it won't be fit to survive.

Title: Re: bots are hard to beat
Post by clyring on Feb 4^th, 2012, 7:12am

I feel that I have found the door for the majority of bots, but that I trip and fall going through it due to bad tactics... =S

Title: Re: bots are hard to beat
Post by Fritzlein on Feb 4^th, 2012, 8:11am

on 02/04/12 at 07:12:44, clyring wrote:

I feel that I have found the door for the majority of bots, but that I trip and fall going through it due to bad tactics... =S

Good tactics (at least for humans) also rely on simplifying principles. It's not like we can practice up until we learn to examine a millions options per second. Top players examine the same number of positions per second, but they have a good intuition as to which are the important ones that must be checked.

There is still a distinction between tactics and strategy, but both of them are made human-tractable by guiding principles that aren't immediately obvious.

Title: Re: bots are hard to beat
Post by ocmiente on Feb 6^th, 2012, 11:27am

This past weekend, bot_marwin was rated as the top player by its game room rating (http://arimaa.com/arimaa/gameroom/topRatedPlayers.cgi). Does anyone know the last time a bot was at the top of that list?

Title: Re: bots are hard to beat
Post by Fritzlein on Feb 7^th, 2012, 8:57am

on 02/06/12 at 11:27:12, ocmiente wrote:

Judging from this graph (http://arimaa.idahoev.com/images/ArimaaRatings2007-09.png), I'd say that a bot last topped the game room rating list in January of 2004, i.e. eight years ago, as discussed in this forum thread (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi?board=talk;action=display;num=1145426743;start=3#3). After I joined arimaa.com in July 2004, I became one of the bot-bashers, i.e. people who inflate their ratings by beating bots over and over. I gave up that project some years ago when it became boring, but others carried the torch. Until recently the list was topped by a human bot-basher rated about 2700. I believe humans drop off the list if they are inactive for over a year; otherwise a human would still top the list.

Perhaps if I get the time, I will try to get a 2800 gameroom rating. It certainly wouldn't be boring at first, as the bots have caught up to me substantially in recent years, but (assuming that I was able to recapture some semblance of dominance) it would probably get boring before I managed to set a new human gameroom rating record. Presumably it would only stay interesting if I am no longer reliably able to defend the Arimaa Challenge. :o

Title: Re: bots are hard to beat
Post by SpeedRazor on Feb 7^th, 2012, 6:13pm

This is probably the wrong place to pose this question, but Ocmiente's last post got me thinking. What if the greatest Arimaa player is a human - massively better than anything else: higher rating (2800²) ... Bobby Fischer in '70, etc.

(S)He wins his/her part of the Arimaa Challenge - of course, but the next two wet-ware players, playing their hearts out, lose to the silicon beast. Software beats human 2:1. A clear victory. But everybody knows that the greatest Arimaa player in the world is a human.

Please tell me that I'm wrong all these years, but doesn't the programmer only need to win two out of three? Even if the undeniable greatest Arimaa player is a still a Human?

Title: Re: bots are hard to beat
Post by rbarreira on Feb 7^th, 2012, 6:17pm

on 02/07/12 at 18:13:24, SpeedRazor wrote:

The bot needs to win 2 out of 3 games against each human in order to win the challenge.

Title: Re: bots are hard to beat
Post by SpeedRazor on Feb 7^th, 2012, 6:18pm

Thnx :) Cool

Then, maybe, the humans have a few more years yet. No way it's going to 2020, though...

Title: Re: bots are hard to beat
Post by Fritzlein on Feb 9^th, 2012, 9:15am

on 02/07/12 at 08:57:06, Fritzlein wrote:

Until recently the list was topped by a human bot-basher rated about 2700. I believe humans drop off the list if they are inactive for over a year; otherwise a human would still top the list.

My memory was off. iteY only broke 2600; it was Arimanator who topped 2700. Both of them retired on top and were only removed later due to inactivity or else they would still be the top two, as no bots have ever reached that level even temporarily. Here are the top peak gameroom ratings:
2715 Arimanator
2646 chessandgo
2619 iteY
2540 syed
2529 Fritzlein

Unless you believe that the entire rating system has deflated by a couple hundred points (and I don't), the appearance of a bot at the top of the ratings list must be related to the sport of botbashing going out of style. Of course I realize the bots are getting stronger too, yadda yadda.

Title: Re: bots are hard to beat
Post by ocmiente on Feb 9^th, 2012, 1:14pm

I suggest that someone with a bot start bot bashing other bots, to show that bots are better at bot bashing than humans. Hopefully, this would stop people from running bait and tackle against bomb just to get a high number in the game room.

I'm pretty sure that bots are better at the particular repetitive task of bot bashing than humans are. I just don't think anyone's tried it to the degree necessary to get to the top for good.

On second thought, I hope nobody takes this seriously. It's silly enough that people do it. It would be even more silly if bits did it.

Title: Re: bots are hard to beat
Post by Fritzlein on Feb 9^th, 2012, 3:33pm

on 02/09/12 at 13:14:40, ocmiente wrote:

On second thought, I hope nobody takes this seriously. It's silly enough that people do it. It would be even more silly if bits did it.

Let me suggest that the reason bots are so high the gameroom rating list is essentially that they have already engaged in bot-bashing. They all beat inferior bots repetitively. Yes, they were required to do so as part of qualifying for the Computer Championship, but the official sanction for bots bashing bots doesn't mean their current ratings are any more accurate than mine would be if I engaged in a round of bot-bashing. What do you think would happen to my rating if I went through the qualifying that the top bots just did? Do you think my rating would remain about the same, because I am approximately accurately rated relative to the six qualifying bots? Or maybe I would go down?

If there is something silly going on, it may be the extent to which you are taking it as a significant indicator that a bot topped the gameroom rating list, even though you are wise enough to dismiss the gameroom ratings of human bot bashers.

Title: Re: bots are hard to beat
Post by hyperpape on Feb 9^th, 2012, 4:23pm

It seems like the natural thing to do is compute some bot ratings excluding the BvB games.

Title: Re: bots are hard to beat
Post by ocmiente on Feb 9^th, 2012, 5:08pm

I don't think that would solve the problem of the game room ratings being inaccurate. If you were to do that, you should probably eliminate all of the bait and tackle games against bomb that humans have played - or just declare bomb to be so broken that no rating is possible. But what's the point? The game room ratings are only a rough indicator anyway.

And sorry for using the term 'silly'. That was probably too harsh. If someone gets joy out of seeing a high game room rating, then I'm happy for them.

After looking at bot_marwin's recent games, it appears that it was involved in some kind of bot bashing, even if it was unintentional. It won several games against an earlier version of itself. If marwin were a person, I would think there was something odd about beating up a younger version of itself.

Title: Re: bots are hard to beat
Post by Fritzlein on Feb 9^th, 2012, 6:44pm

on 02/09/12 at 17:08:48, ocmiente wrote:

And sorry for using the term 'silly'. That was probably too harsh. If someone gets joy out of seeing a high game room rating, then I'm happy for them.

Thanks for your permission to try to inflate my rating like I did in the old days, whether it is pointless or no. In fairness, you have my permission to try to complete the "Advanced Ladder". If you get joy out of that, then I'm happy for you. :D

Quote:

After looking at bot_marwin's recent games, it appears that it was involved in some kind of bot bashing, even if it was unintentional. It won several games against an earlier version of itself. If marwin were a person, I would think there was something odd about beating up a younger version of itself.

I think the best indicator we get of the advances in the bots is from the qualifying games that start in mid-March. At that time the bots are new, so we can't beat them with tricks tailored toward simple, known weaknesses. The games are slow, and if we are too impatient or distractable to play at two minutes per move, that's our own fault. Also we are limited to two games against each bot, so no individual can greatly distort things. We get a good measurement of strength against a variety of opponents. Yes, there aren't enough games for a super-accurate reading, but it is still the best indicator we get. I will treat the result of the qualifying with much more respect than the gameroom ratings of the bots, even if that means I have to admit I'm in danger of not being able to defend the Challenge.

Quote:

[...]or just declare bomb to be so broken that no rating is possible.

But wait, if we declared that bomb is broken wouldn't that be sending the message that bots are easy to beat? ::)

Title: Re: bots are hard to beat
Post by tize on Feb 10^th, 2012, 1:15pm

on 02/09/12 at 17:08:48, ocmiente wrote:

If marwin were a person, I would think there was something odd about beating up a younger version of itself.

I think that would be awesome to be able to meet up with an older version of yourself and play. :)

Title: Re: bots are hard to beat
Post by Fritzlein on Feb 10^th, 2012, 1:21pm

on 02/10/12 at 13:15:11, tize wrote:

I think that would be awesome to be able to meet up with an older version of yourself and play. :)

Oh, no, marwin is going to win the Challenge by using time travel!

Title: Re: bots are hard to beat
Post by clyring on Feb 10^th, 2012, 3:26pm

Don't worry- marwin2030 will use features not available on the current challenge computers... ;)

Title: Re: bots are hard to beat
Post by tize on Feb 10^th, 2012, 4:47pm

on 02/10/12 at 15:26:07, clyring wrote:

Don't worry- marwin2030 will use features not available on the current challenge computers... ;)

But as marwin2030 will be built for a quantum super core he will expect the features to be both present and absent and still be able to both win and lose the challenge matches. ;)

Title: Re: bots are hard to beat
Post by hyperpape on Mar 16^th, 2012, 12:56pm

on 02/09/12 at 15:33:22, Fritzlein wrote:

Let me suggest that the reason bots are so high the gameroom rating list is essentially that they have already engaged in bot-bashing.

Worth noting in light of this that Lucy has been rising up over 2400 based on play against strong human players.

Those are blitz games, so that's a whole 'nother caveat.

Title: Re: bots are hard to beat
Post by Adanac on Mar 16^th, 2012, 1:32pm

on 03/16/12 at 12:56:21, hyperpape wrote:

Worth noting in light of this that Lucy has been rising up over 2400 based on play against strong human players.

Those are blitz games, so that's a whole 'nother caveat.

I don’t know if Swynddla now has a super-computer in his basement, but it seemed that Lucy’s strength increased by a few orders of magnitude yesterday. Or maybe it’s just because I misplayed the openings yesterday and getting started on the wrong foot against top bots is a very painful experience these days. Building a solid position in the first 15 moves usually leads to a win and I just couldn’t do it in those last 3 games.

Because my initial attacks are being repulsed, and since Lucy obsessively pulls my pieces up at every possible opportunity, I’m thinking that maybe the way to win the opening is by swarming, even if it means getting stuck with a camel hostage. I’ll review tharkun’s thread on camel hostages before trying it out :)

Title: Re: bots are hard to beat
Post by Swynndla on Mar 16^th, 2012, 4:59pm

I haven't changed anything (well about a week ago I did try some changes for a couple of days but it made Lucy weaker so I reverted Lucy back to the WCC version which she still is and has been for the last week or so). There are lots of things that need changing though, and I appreciate you humans pointing out her weaknesses (eg Lucy gets her elephant cut off too easily, and she will tend to pull pieces even if it helps the opponent in their already strong attack, and Lucy is fairly passive and wont attack very often when she should).

Lucy is running on a dual core only. When Lucy2012 goes on the game server, she'll be weaker as bots only use a single core on the game server (as less memory for hash too).

I'd guess Lucy's CC rating is only 2100. I'm very surprised that Lucy peaked over 2400, and that must just because of blitz (ie Lucy is "human bashing"). Imagine what sort of rating marwin/briareus/sharp could get running blitz continuously!

Title: Re: bots are hard to beat
Post by mistre on Mar 16^th, 2012, 11:34pm

on 03/16/12 at 16:59:56, Swynndla wrote:

Lucy is running on a dual core only. When Lucy2012 goes on the game server, she'll be weaker as bots only use a single core on the game server (as less memory for hash too).

Ahhh, That explains why Lucy is arguably stronger (currently has a higher rating) than even SharpBlitz on the server (because of Dual Core vs Single Core). I couldn't figure that out but now it makes sense.

on 03/16/12 at 16:59:56, Swynndla wrote:

I'd guess Lucy's CC rating is only 2100. I'm very surprised that Lucy peaked over 2400, and that must just because of blitz (ie Lucy is "human bashing"). Imagine what sort of rating marwin/briareus/sharp could get running blitz continuously!

Yes, humans will have a tough enough time with Sharp2012, Marwin2012 and Briareus2012Blitz on the server. If they were dual core - They might even hit 2500!