Author |
Topic: Asymmetrical static evaluation? (Read 4983 times) |
|
woh
Forum Guru
Arimaa player #2128
Gender: 
Posts: 254
|
 |
Re: Asymmetrical static evaluation?
« Reply #15 on: Mar 8th, 2007, 11:24am » |
 Quote  Modify
|
on Mar 7th, 2007, 6:16pm, IdahoEv wrote:| Which reminds me ... A possible gameroom feature I could work up on the dev server: an invite option with side set to "random". |
|
Great idea!
I would make use of such an option.
|
|
 IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender:
Posts: 1413
|
|
Re: Asymmetrical static evaluation?
« Reply #16 on: Mar 8th, 2007, 4:23pm » |
Quote Modify
|
So would I.
|
|
IP Logged |
|
|
|
NIC1138
Forum Guru
Arimaa player #65536
Gender:
Posts: 149
|
|
Re: Asymmetrical static evaluation?
« Reply #17 on: Mar 8th, 2007, 5:15pm » |
Quote Modify
|
on Mar 7th, 2007, 1:28pm, JacquesB wrote:
Nice site, thanks!! 
|
|
IP Logged |
|
|
|
NIC1138
Forum Guru
Arimaa player #65536
Gender:
Posts: 149
|
|
Re: Asymmetrical static evaluation?
« Reply #18 on: Mar 8th, 2007, 5:34pm » |
Quote Modify
|
on Mar 7th, 2007, 4:56pm, Fritzlein wrote:It is beside the point to consider the confidence interval of a statistic that is known to be biased in the first place.
|
|
Not quite so, if you allow me a moment of pedantism license!...  What happens is that we will measure with accuracy the probability of silver winning "in a usual gameroom match". The usual gameroom match has a stronger silver player...
As yourself hinted on another thread, what we would really like is to measure with accuracy the probability of silver (or gold) winning given the difference of the player ratings... As you said, if there is an imbalance, a (little) stronger gold player would have 50% chance to win a (little) weaker silver player.
Even worse, what we would appreciate is to predict the winner given the ratings of each one, not only the difference!... It could be, for example, that for new players there is no imbalance, and the imbalance shows up for better players...
But unfortunately, the dimensional curse pokes a needle in our butt for each new variable we want to consider!... The less naïveté, the more games we need to play! 
Bringing the concept from particle physics, I would say we need to enhance the ilumminance of our arimaa player collider! 
|
|
IP Logged |
|
|
|
JacquesB
Forum Senior Member
Arimaa player #2380
Gender:
Posts: 26
|
|
Re: Asymmetrical static evaluation?
« Reply #19 on: Mar 9th, 2007, 12:14pm » |
Quote Modify
|
I just computed the intervals from the data in Nicolau's post. Knowing that it is a common procedure that the stronger player takes silver (which I didn't), that was clearly an error. This is a good example of wrong sampling or selection bias.
Nic's answer sounds justified, its not pedantic.
Furthermore, reading the thread pointed by Fritzlein (your method of taking the samples in pairs sounds ok) I see the bias is not measurable. Also, what PMertens says that it depends on the user's style is true. In go weak players don't really understand how important one move is. Therefore, they are balanced with a smaller komi (= compensation for playing first) than strong players. If the advantage exists, it is very possible that weak players miss it completely.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Asymmetrical static evaluation?
« Reply #20 on: Mar 10th, 2007, 12:16pm » |
Quote Modify
|
I agree, JacquesB, that an advantage which is important at a high level of play may not be noticeable at a lower level.
In keeping with NIC's (non-pedantic) point, it really depends on what we are trying to measure. Maybe for 1500-rated players it doesn't matter who moves first, and for 1800-rated players Silver has an advantage, while for 2100-rated players Gold has an advantage. Even if we measure something about the best players who exist today, what does it prove about Arimaa per se? At a still higher level, the advantage may swing again.
This is much different from chess where tests seem to show an advantage of about 50 rating points for white at all levels of play from beginner to grandmaster. Maybe that advantage disappears for both random play and for perfect play, but it is remarkable how constant it is in between.
When we have a large enough game database, we might be able to prove that neither side has an appreciable advantage at any level of play. Until then, it's anybody's guess why Silver apparently has a statistically significant advantage in rated human vs. human games. I'm leaning towards it being a fluke, but maybe my methodology is flawed in some important way we haven't seen yet.
|
|
IP Logged |
|
|
|
NIC1138
Forum Guru
Arimaa player #65536
Gender:
Posts: 149
|
|
Re: Asymmetrical static evaluation?
« Reply #21 on: Mar 28th, 2007, 12:10am » |
Quote Modify
|
on Mar 10th, 2007, 12:16pm, Fritzlein wrote:| This is much different from chess where tests seem to show an advantage of about 50 rating points for white at all levels of play from beginner to grandmaster. Maybe that advantage disappears for both random play and for perfect play, but it is remarkable how constant it is in between. |
|
It's sure a remarkable fact... Do you have any biblio about this? I can't say what is more interesting: an advantage to change with rating, or to keep stactic!  As a researcher I actually have an answer: the stactic advantage, since it's easier to detect! 
I do believe we will stumble into an advantage in the future... No reason. Should we start a bet, to run in parallel to the Arimaa Challange?
|
|
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: Asymmetrical static evaluation?
« Reply #22 on: Apr 22nd, 2007, 7:27pm » |
Quote Modify
|
Based on my last game against bot_Clueless2006Fast I ponder another kind of asymmetrical evaluation I could imagine making sense. In that game the bot managed to immobilize my elephant and subsequently when to extreme lengths maintaining the blockade. Problem was that it needed its own elephant for this, making it in effect worse than useless. Trying to rotate it out was clearly infeasible in that position especially given its lack of strategic insight. Since, however, one would like to prevent a bot from becoming itself a victim of an elephant blockade, it would make sense to evaluate a position differently depending on which side the bot is.
|
|
IP Logged |
|
|
|
NIC1138
Forum Guru
Arimaa player #65536
Gender:
Posts: 149
|
|
Re: Asymmetrical static evaluation?
« Reply #23 on: Apr 22nd, 2007, 9:46pm » |
Quote Modify
|
on Apr 22nd, 2007, 7:27pm, aaaa wrote:| Since, however, one would like to prevent a bot from becoming itself a victim of an elephant blockade, it would make sense to evaluate a position differently depending on which side the bot is. |
|
Where does the assymetry come in?... There is no contradiction in making a blockade and escaping one. You just need to set wat is the point when you would rather one than the other...
You mean that the program must have a form of "strategy register" where he decides what to do in the long-term, and then acts accordingly? BECause that is something I've been thinking about lately, and I wonder who would have done something like this already, does anybody has any references?....
|
|
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: Asymmetrical static evaluation?
« Reply #24 on: Apr 22nd, 2007, 10:15pm » |
Quote Modify
|
What I'm saying here is that to a bot being blockaded is more a disadvantage than being the one doing the blockading is an advantage. That's the asymmetry here.
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: Asymmetrical static evaluation?
« Reply #25 on: Apr 22nd, 2007, 11:38pm » |
Quote Modify
|
You are right aaaa.
I have not put that into my bot, but I can see that it would sometimes be useful.
|
|
IP Logged |
|
|
|
camelback
Forum Guru
Arimaa perl monger
Gender:
Posts: 144
|
|
Re: Asymmetrical static evaluation?
« Reply #26 on: Apr 23rd, 2007, 11:52pm » |
Quote Modify
|
on Apr 22nd, 2007, 7:27pm, aaaa wrote:| Based on my last game against bot_Clueless2006Fast I ponder another kind of asymmetrical evaluation I could imagine making sense. In that game the bot managed to immobilize my elephant and subsequently when to extreme lengths maintaining the blockade. Problem was that it needed its own elephant for this, making it in effect worse than useless. Trying to rotate it out was clearly infeasible in that position especially given its lack of strategic insight. Since, however, one would like to prevent a bot from becoming itself a victim of an elephant blockade, it would make sense to evaluate a position differently depending on which side the bot is. |
|
aaaa, here is another example of bot trying to maintain a useless blockade.
Gnobby maintained the blockade using all the pieces to the end and to my surprise lost the game 
http://arimaa.com/arimaa/games/jsShowGame.cgi?gid=50422&s=w
|
|
IP Logged |
|
|
|
ddyer
Forum Guru
Gender:
Posts: 66
|
|
Re: Asymmetrical static evaluation?
« Reply #27 on: May 17th, 2007, 12:55pm » |
Quote Modify
|
This thread is mixing discussions of two very different
things. The original question is if the evaluation FUNCTION
should be the same for both sides. The other question.
which took over the later stages of the thread. is if the
VALUE of the function initially favors silver or gold.
|
|
IP Logged |
visit my game site: http://www.boardspace.net/
free online abstract strategy games
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Asymmetrical static evaluation?
« Reply #28 on: May 17th, 2007, 2:02pm » |
Quote Modify
|
on May 17th, 2007, 12:55pm, ddyer wrote:| This thread is mixing discussions of two very different things. |
|
Actually three different things: whether the side to move affects static evaluation, which side has the advantage from the starting position, and whether the evaluations for the two sides are equal and opposite.
On the third point (which aaaa raised), I recall that the chess software Crafty used an asymmetrical evaluation, at least at some point in its evolution. In order to give Crafty a better chance of beating humans, Hyatt made Crafty hate locked pawns. If the pawn structure was static, Crafty could think that it was losing both sides of the position.
For a computer which is able to beat humans in most positions, it is an interesting concept to avoid the few positions that give humans an edge. On the other hand, I'm not sure how useful that concept is for Arimaa at present. Right now there are many strategic features that bots don't grasp well, so I would like any of those features to be present no matter which side I am playing. For example, against a bot I would like to
* Give my horse hostage to its elephant or get its horse hostage with my elephant
* Frame its camel or get my camel framed
* Get its elephant blockaded (almost always) or allow my elephant to be blockaded (in some circumstances)
* Get in a race where we are each capturing as fast as possible (bots don't weigh material properly against remote goal threats)
* Pull out the bot's rabbits (almost always) or advance my own rabbits (in some circumstances)
In short, many strategic features of Arimaa are double-edged. My advantage over bots comes from recognizing the importance of each feature, and how it changes my long-term objectives. You can't get around that by programming a bot to avoid all strategic features. Instead it has to learn how to play differently (and well) in each circumstance.
|
| « Last Edit: May 17th, 2007, 2:03pm by Fritzlein » |
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: Asymmetrical static evaluation?
« Reply #29 on: May 17th, 2007, 7:24pm » |
Quote Modify
|
To go back to the originating issue brought up by IdahoEv: The idea of according "en-prise penalties" can be intriguing at first, but you have to realize that complications set in if you are in a situation where different pieces can be captured in one move. Then you need your engine to figure out how many of them can be captured in one move. If it's just one, then instead of the sum the maximum of the en-prise penalties should be added to the evaluation score and if it's more, then it gets even more complicated. All in all, it just doesn't seem to be worth it given the concept of quiescence search (although in that case you might be able to find a use for asymmetric evaluation for the purpose of distinguishing stable and unstable positions).
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print