Author |
Topic: (no) absolute score values for pieces? (Read 38212 times) |
|
Fritzlein
Forum Guru
Arimaa player #706
Gender: 
Posts: 5928
|
 |
Re: (no) absolute score values for pieces?
« Reply #45 on: Dec 17th, 2005, 2:11pm » |
 Quote  Modify
|
on Dec 14th, 2005, 3:01am, Ryan_Cable wrote:| HOTFLAME (Human On The FLy Arimaa Material Evaluation). |
|
 Nice acronym
Quote:FAME ignores the non-matchup interactions between pieces:
EHCR vs. EMDR = ECCR vs. EMDR = -142 |
|
Excellent point. The material eval should definitely differentiate between different situations. One significant motivation for FAME was that equivalent situations be evaluated equivalently, e.g. EMR vs. ECCR = EDR vs. ECCR, but I appear to have flattened it too much.
Actually, my very first idea about dynamic piece valuation was to value a piece based on what proportion of enemy pieces it was better than, equal to, or inferior to. I could never get it to work out in a reasonable fashion. I eventually forgot about it when FAME started to give some reasonable numbers, but I think I may have to revive the idea in future.
Quote:| FAME has problems when one side has no Rs: |
|
My thought was that having ridiculous evals in the case of zero rabbits was a minor glitch, since that case probably has to be handled separately anyway. Also, whether having no rabbits is a possible draw or an automatic loss depends on the tournament rules, so that needs special handling as well. I recognize this as a bug in FAME, but perhaps not of the highest priority.
Quote:| EHHDDCCR vs. EMHHDDCC = ERRRRRRR vs. EMHHDDCC = -223 |
|
Not quite correct, because in the former case the side with the rabbit ties on some of the matchups, but the point is taken that a side with no rabbits can never be considered to be winning.
Quote:| Strictly speaking, you have not defined the score for situations where one side has more pieces than the other has pieces plus Rs. |
|
Sorry I didn't didn't specify this in the post. In chat discussion with Jeff, I suggested a fine workaround would be to have the weaker side have negative leftover rabbits. Rather than simply getting a zero rabbit bonus, they get a negative rabbit bonus. This isn't a supremely accurate way of measuring how much the stronger side is winning by, but is does retain an appropriate incentive towards capture when one side is far ahead in material. For example EDDCCRR vs EMRRR < EDDCCRR vs EMR, because in the former case the weaker side has zero leftover rabbits, whereas in the latter case the weaker side has negative two leftover rabbits.
Quote:| But I think EMHHDCR vs. EHDRRRR is enough advantage for me to be able to beat you even if you are the true World Champion. |
|
I agree with Ryan, and I would like to see this played out postally between the two of you. Are you game, Adanac? I don't think you will prefer the latter army any more after you have had to try to play with it for a while. Let's have all the pieces start on the back row on each side (following appropriate initial suicides) in the order RD*ERRHR for Adanac as Gold, opposite CM*HEDHR for Ryan as Silver. I don't think those piece oppositions give either player an advantage, apart from the significant material advantage for Ryan with the stronger pieces.
Perhaps FAME does undervalue rabbits relative to pieces in some endgames, but I believe popular opinion rather overvalues rabbits relative to pieces, and I would like to see at least Adanac's extreme proposal put to the test.
|
|
 IP Logged |
|
|
|
Adanac
Forum Guru
Arimaa player #892
Gender:
Posts: 635
|
|
Re: (no) absolute score values for pieces?
« Reply #46 on: Dec 18th, 2005, 11:12am » |
Quote Modify
|
on Dec 17th, 2005, 2:11pm, Fritzlein wrote:
I agree with Ryan, and I would like to see this played out postally between the two of you. Are you game, Adanac? I don't think you will prefer the latter army any more after you have had to try to play with it for a while. Let's have all the pieces start on the back row on each side (following appropriate initial suicides) in the order RD*ERRHR for Adanac as Gold, opposite CM*HEDHR for Ryan as Silver. I don't think those piece oppositions give either player an advantage, apart from the significant material advantage for Ryan with the stronger pieces.
Perhaps FAME does undervalue rabbits relative to pieces in some endgames, but I believe popular opinion rather overvalues rabbits relative to pieces, and I would like to see at least Adanac's extreme proposal put to the test.
|
|
I'd like to try it using Fritzlein's opening setup! I've struggled mightily with the stronger pieces (but with only 1 rabbit remaining) in the World Championship against PMertens and the Postal Championship against Belbo - it's because of games like those that I've come to highly value rabbits, but I rarely play endgames with the extra rabbits. The only game I can think of was my Postal Championship against 99of9 where I sacrificed my Camel for a rabbit and an opportunity to advance rabbits on both wings, rather than save my camel and leave my elephant without mobility. I won the game but I'm still not 100% convinced that it was necessarily a correct sacrifice.
If I'm wrong about my balanced rabbit theory, it's better to find out sooner rather than later!!
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: (no) absolute score values for pieces?
« Reply #47 on: Jan 22nd, 2006, 4:02am » |
Quote Modify
|
Since people have been wondering what the fame score was at certain times during the recent tournament games, I put together a page to calculate it.
http://www.janzert.com/fame.html
Janzert
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender:
Posts: 1413
|
|
Re: (no) absolute score values for pieces?
« Reply #48 on: Jan 22nd, 2006, 6:11am » |
Quote Modify
|
on Jan 22nd, 2006, 4:02am, Janzert wrote:Since people have been wondering what the fame score was at certain times during the recent tournament games, I put together a page to calculate it.
http://www.janzert.com/fame.html |
|
Wow, that's very cool.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: (no) absolute score values for pieces?
« Reply #49 on: Jan 22nd, 2006, 11:46am » |
Quote Modify
|
Thanks Janzert! I'm going to have that window open from now on when I'm watching any game.
|
|
IP Logged |
|
|
|
omar
Forum Guru
Arimaa player #2
Gender:
Posts: 1003
|
|
Re: (no) absolute score values for pieces?
« Reply #50 on: Jan 29th, 2006, 7:51pm » |
Quote Modify
|
Thanks Brian. For now I've linked it on the Downloads page so that we can find it easily.
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: (no) absolute score values for pieces?
« Reply #51 on: Jan 30th, 2006, 6:43pm » |
Quote Modify
|
Feel free to take and use it in the site itself if you want. I added a notice to the top of the html file releasing it into the public domain. I also moved the css into the page itself so there's no external stylesheet to deal with.
Janzert
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: (no) absolute score values for pieces?
« Reply #52 on: Jan 30th, 2006, 6:50pm » |
Quote Modify
|
One other thing on this subject. Fritzlein had mentioned possibly making some changes to FAME in order to fix some problems in certain situations.
If changes are made it might be nice to "normalize" the scores so that an initial free rabbit is worth something like 1, 10 or 100.
Janzert
|
|
IP Logged |
|
|
|
Swynndla
Forum Guru
Arimaa player #1821
Posts: 235
|
|
Re: (no) absolute score values for pieces?
« Reply #53 on: Mar 7th, 2006, 10:07pm » |
Quote Modify
|
Normalized values make a lot of sense to me ... not only are they easy for humans to understand, but if bots used a modular value lookup (and it makes sense to me that they should) then it would be easy to substitute that lookup for FAME, because the positional function (that deals with value increases and decreases based on a pieces position on the board and a pieces position relative to other pieces etc) wouldn't have to be changed. Am I making any sense at all?
PS - I really like FAME - great stuff Fritzlein.
|
| « Last Edit: Mar 7th, 2006, 10:12pm by Swynndla » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: (no) absolute score values for pieces?
« Reply #54 on: Mar 8th, 2006, 7:52am » |
Quote Modify
|
I just realized that the results of the Adanac / Ryan Cable never got posted to this thread. Ryan won both games.
http://arimaa.com/arimaa/gameroom/comments.cgi?gid=23643
http://arimaa.com/arimaa/gameroom/comments.cgi?gid=24845
So it may be that FAME undervalues rabbits, but a lone rabbit can still be plenty to win if backed by the more powerful army.
Another point worth noting is that both of these games were very short. If it ever seems that Arimaa is becoming too long and boring, it's good to know there is a quick fix in the form of starting with fewer pieces. I'm guessing that if each side removed HDCRRR to begin with (i.e. if we played with only ten pieces), Arimaa would be much shorter and more tactical. Also defensive play would become hopeless.
|
| « Last Edit: Mar 8th, 2006, 1:05pm by Fritzlein » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: (no) absolute score values for pieces?
« Reply #55 on: Mar 8th, 2006, 9:41am » |
Quote Modify
|
on Mar 7th, 2006, 10:07pm, Swynndla wrote:| Normalized values make a lot of sense to me ... not only are they easy for humans to understand, but if bots used a modular value lookup (and it makes sense to me that they should) then it would be easy to substitute that lookup for FAME, because the positional function (that deals with value increases and decreases based on a pieces position on the board and a pieces position relative to other pieces etc) wouldn't have to be changed. Am I making any sense at all? |
|
You are making sense to me. My concern is that the rabbit may not be the best thing to normalize on if the positional values are supposed to remain unchanged when a new material scheme is implemented.
For example, suppose I program my bot so that a perfectly positioned camel hostage is worth a dog. Taking the 99of9 material values R=1, C=2, D=3, H=5, M=8, E=13, I hard-code my camel hostage to also be worth 3. Then later on I discover that I was undervaluing the first rabbit, and revamp my system to have R=1, C=1.5, D=2, H=3, M=5, E=10. The first rabbit is still worth one, but by normalizing to the rabbit I just pulled the value of all the other pieces down, and now my hard-coded positional value for a camel is too high at 3, now the value of a horse. In this case my positional values would have stayed better if I had normalized to the total value of the army rather than normalizing to the value of its weakest unit.
Well, anyway, probably whenever a new material evaluation scheme is implemented, all the positional values have to be re-tuned by hand anyway, so it isn't such a big deal. Probably a more important factor is the convention in chess that a pawn is worth 1, which makes a computer chess evaluation of +2.36 immediately intelligible. I guess for the sake of going easy on everyone's intution in conversation, FAME should be scaled to make the first free rabbit worth 1. Developers can scale things internally however it most makes sense to them. Rabbit=1 is a convention that regular gamers will expect, even if they don't expect that rabbits will be worth a lot more than 1 in the endgame.
|
| « Last Edit: Mar 8th, 2006, 1:06pm by Fritzlein » |
IP Logged |
|
|
|
Swynndla
Forum Guru
Arimaa player #1821
Posts: 235
|
|
Re: (no) absolute score values for pieces?
« Reply #56 on: Mar 8th, 2006, 5:38pm » |
Quote Modify
|
As always, it's more complicated than I first realized!
But thinking about the bot that is programmed to think "that a perfectly positioned camel hostage is worth a dog", probably it would be better to program it to think that a perfectly positioned camel hostage is worth the value of a dog, and not just "3" (ie it would look up the value). That way, changing the piece values wont matter. But ... that value (of a dod) would change as the number of peices decreased as the games goes on, so would it still work? ... I'm not so sure, so I'm at a loss as to the what to do about that.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: (no) absolute score values for pieces?
« Reply #57 on: Mar 9th, 2006, 11:22am » |
Quote Modify
|
Well, I don't know how I would do it, now that I think about it some more. A camel hostage is worth way less late in the game than early in the game. Bomb can be tricked in the endgame because it still thinks camels and camel hostages are hugely important, and they're not. But the value of cats and dogs usually goes up between the opening and the endgame, so tying the value of the camel hostage to the value of another piece doesn't seem any cleverer than making it a fixed value.
I actually have no idea how to code dynamic positional values. It's hard enough thinking how relative material values change in light of exchanges.
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: (no) absolute score values for pieces?
« Reply #58 on: Mar 18th, 2006, 4:14pm » |
Quote Modify
|
Updated the FAME calculator today.
* You can now specify zero rabbits left.
* Normalized the scores so an initial rabbit is worth a score of 1. The raw score is also reported in parentheses for now.
But in testing this I found some discrepancies with other results in this thread. After rewriting the algorithm used, I still get the same results. Also checking with a local script I wrote in python when I originally implemented FAME I still get the same score. So either I'm misunderstanding some aspect of FAME or a few of the results posted are wrong. Could someone please double check the following:
Code:pieces = posted normalized score(raw posted), my normalized(my raw)
EDRR vs. ECCR = 0.24(8), 0.41(14)
EDRRR vs ECCRR = -0.02(-1), 0.20(7)
EMHHRRR vs. EHDRRRR = 5.67(190.9), 5.83(196)
EMHHDCR vs. ERRRRRR = 7(235.8), 7.44(251)
EHHDDCCR vs. EMHHDDCC = -6.62(-223), -4.36(-147)
ERRRRRRR vs. EMHHDDCC = -6.62(-223), -7.12(-240) |
|
Janzert
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: (no) absolute score values for pieces?
« Reply #59 on: Mar 18th, 2006, 4:49pm » |
Quote Modify
|
Hmm, I must have something wrong or FAME's a little wierder than I thought.
Try EMHHDDCCR vs EM and a variable number of R's.
For 8-6 silver r's the score goes up for gold as expected. But then 5-0 r's the score actually decreases for gold with each additional r captured. 
FYI, I'm using the modification (clarification?) by Fritzlein to allow negative rabbits left over.
Janzert
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print