Arimaa Forum
        (http://arimaa.com/arimaa/forum/cgi/YaBB.cgi)
      

        Arimaa >> General Discussion >> Re: Proposed New Material Evaluation Formula
        
(Message started by: Janzert on Dec 21st, 2006, 3:30pm)
Title: Re: Proposed New Material Evaluation Formula
Post by Janzert on Dec 21st, 2006, 3:30pm
Just a couple quick comments.

First just want to note that this isn't a material evaluator, or rather isn't just a material evaluator. Since it incorporates positional elements as well. At least if I'm understanding the description correctly.

Something that would probably help people comment on this is if you could provide some example situations and what it evaluates to. A couple of interesting ones might be; for pieces captured on each side:

M vs DCC
M vs HRR
M vs HC
H vs DR
H vs CC
H vs CR
DR vs CC

Also the initial value of each piece.

Of course since your evaluator also incorporates positional elements the score is going to vary by exact board positions but maybe you can show the range possible for each or simply use the assumption that each column is still filled.

Janzert
Title: Re: Proposed New Material Evaluation Formula
Post by The_Jeh on Dec 21st, 2006, 4:15pm
(I'm sorry about removing this earlier.  I thought I'd forgotten something.  Later I will add something that will take into account rabbit advancement, and I will give examples when I have time.  Thank you, Janzert)

I would like to propose a new material/positional evaluation formula.  Questions, comments, and criticisms are highly appreciated.

First, let me define the variables:

PieceValue = 9 - (# of pieces that can push or pull it)
Rabbits are evaluated in the same way as all other pieces.

SumPieceValue = the sum of the values of your pieces, including rabbits

RabDiff = (#ofFriendlyRabbits - #ofEnemyRabbits)
This is the rabbit difference

YC = (# of columns in which you have a piece or rabbit, or a piece higher than a rabbit in an adjacent column.)
OC = (" " " for opponent)

From these variables, your evaluation (E) is computed as follows:


E = ((SumPieceValue)(20/11)(1+RabDiff/9))^(1+YC/8-OC/8)


Under this system, both players start out with an initial E of 100.  If you find any major (or minor) flaws, please post them so I can refine this formula.

The_Jeh
Title: Re: Proposed New Material Evaluation Formula
Post by The_Jeh on Dec 21st, 2006, 4:16pm
The initial values of the pieces are:

Rabbit: 1
Cat: 3
Dog: 5
Horse: 7
Camel: 8
Elephant: 9
Title: Re: Proposed New Material Evaluation Formula
Post by chessandgo on Dec 22nd, 2006, 11:52am
Hello, The Jeh. Here are some clues from what we know or think to know about material, to help you designing your evaluator.

As initial trades :
- a cat is probably worth slighly more than a rabbit, but a lot less than two rabbits
- a dog is worth less than a cat and a rabbit, less than 2 rabbits as well in my opinion.
- a horse should be worth slightly less than cat + dog, maybe about the same as dog + rabbit.
- a camel should be worth about H+D, or equivalently H+C.

As pieces get exchanged :
- intrinseque value of rabbits goes up when the number of those go down (ie a rabbit is worth more when you have only 4 rabbits left than it is at the beginning, and as an extreme case the very last rabbit is worth a lot)
- I can't make up my mind concerning relative values of rabbits and pieces, and have no idea as to what the other think : when pieces get exchanged, does the relative value of a rabbit compared to a piece go up or down ? On one hand with less pieces left rabbits are more likely to goal, but the few remaining pieces get more important as well ...

Designing a material evaluator respecting those few things seems to be a very hard job, good luck ! Several other threads in the forum contain much more developped thoughts about the suject, enjoy the reading ;)

Jean
Title: Re: Proposed New Material Evaluation Formula
Post by chessandgo on Dec 22nd, 2006, 11:55am

on 12/22/06 at 11:52:58, chessandgo wrote:
and have no idea as to what the other think : Jean

*others
Title: Re: Proposed New Material Evaluation Formula
Post by The_Jeh on Dec 22nd, 2006, 1:05pm
On reflection, I think I weighted the rabbit difference too greatly, so here is the updated formula:

E = ((SumPieceValue)(20/11)(1+RabDiff/17))^(1+YC/8-OC/8)


ChessandGo, you said that the value of rabbits goes up when their number goes down.  This is absolutely correct; it is an economic principle.  The piece values here, though, are more of a measure of their threat, not their essentiality.  A rabbit is more essential when it is the last one you have, but it is not more of a threat because of it.  It is more of a threat when the number of pieces that can bully it goes down.  That is what this formula measures.

Also, my intention is for this formula to be effective holistically, with the more secondary goal that its individual components can be useful.

Thank you for your valuable insight, ChessandGo.  I will take all of it into consideration.
Title: Re: Proposed New Material Evaluation Formula
Post by Fritzlein on Dec 29th, 2006, 9:34pm

on 12/22/06 at 13:05:59, The_Jeh wrote:
On reflection, I think I weighted the rabbit difference too greatly, so here is the updated formula:

E = ((SumPieceValue)(20/11)(1+RabDiff/17))^(1+YC/8-OC/8)

That's an interesting idea, but it's hard for people to comment on any formula in the abstract.  I agree with Janzert's comment below: you will get more feedback if you show people some examples of how your formula is working.


on 12/21/06 at 15:30:44, Janzert wrote:
Something that would probably help people comment on this is if you could provide some example situations and what it evaluates to. A couple of interesting ones might be; for pieces captured on each side:

M vs DCC
M vs HRR
M vs HC
H vs DR
H vs CC
H vs CR
DR vs CC

Just this once, I will implement this in a spreadsheet for you, The_Jeh, but next time I'll leave the leg work to you.  Here are the values of Janzert's trades plus a few more.  I'll assume it is the opening, so all columns are blocked on each side.  Positive numbers favor the first column and negative number favor the second column

M vs DCC -> -33
M vs HRR -> -28
M vs HC -> -18
H vs DR -> -11
H vs CC -> -9
H vs CR -> -3
DR vs CC -> +2

R -> +13
C vs R -> +4
C vs RR -> -11
D vs RR -> -4
H vs RR -> +4
H vs RRR -> -11
M vs RRR -> -6
E vs RRR -> -3
M vs HR -> -12

My initial impression is that this formula does all right with initial trades of rabbits, cats, and dogs, but undervalues horses a bit, undervalues camels a lot, and undervalues elephants to an extreme.  We don't know how many rabbits for an initial elephant would be a fair trade, but it is more than three!  Even a camel is clearly worth more than three rabbits, although your formula favors the rabbits, and a camel is very likely worth more than a horse and a rabbit, although the formula favors the horse and rabbit.

The concept is interesting, though, if you could fix the heavy piece valuation somehow.  The elephant value hardly matters, since elephants are never traded, but the camel valuation is critical, and simply has to be fixed.
Title: Re: Proposed New Material Evaluation Formula
Post by IdahoEv on Dec 30th, 2006, 12:19pm
Am I the only one confused here by what seems to be the *second half* of a discussion thread about a new evaluation system?  This thread is titled "Re: ..." as if there were an earlier message, and Janzert responds with comments about an earlier message.   But I can't find the earlier message, so I am not sure what these people are talking about!
Title: Re: Proposed New Material Evaluation Formula
Post by Janzert on Dec 30th, 2006, 1:42pm
The second message is actually the original first message. :P

I think The_Jeh deleted the first message without realizing I had already posted a reply.

Janzert
Title: Re: Proposed New Material Evaluation Formula
Post by The_Jeh on Jan 1st, 2007, 1:21am

on 12/29/06 at 21:34:46, Fritzlein wrote:
My initial impression is that this formula does all right with initial trades of rabbits, cats, and dogs, but undervalues horses a bit, undervalues camels a lot, and undervalues elephants to an extreme.  We don't know how many rabbits for an initial elephant would be a fair trade, but it is more than three!  Even a camel is clearly worth more than three rabbits, although your formula favors the rabbits, and a camel is very likely worth more than a horse and a rabbit, although the formula favors the horse and rabbit.

The concept is interesting, though, if you could fix the heavy piece valuation somehow.  The elephant value hardly matters, since elephants are never traded, but the camel valuation is critical, and simply has to be fixed.


So the formula obviously undervalues pieces more and more the higher their rank.  I guess the piece values could be multiplied by or taken to the power of constants, a different one for each piece.  What I don't like about that idea is that these constants would be based on opinion, rather than having mathematically derived significance.  I'll try to think of another way to increase the weight of higher pieces.

Thank you very much Fritzlein.  Please don't spoil me with your recrementitious generosity.

The_Jeh
Title: Re: Proposed New Material Evaluation Formula
Post by Fritzlein on Jan 1st, 2007, 11:07am

on 01/01/07 at 01:21:12, The_Jeh wrote:
What I don't like about that idea is that these constants would be based on opinion, rather than having mathematically derived significance.

Yes, it would be very appealing if the formula had no arbitrary constants.  I think FAME had three arbitrary constants, so even getting that number down to two or one would be an improvement in beauty.  If you manage to create a reasonable formula with no arbitrary constants I will be tremendously impressed.


Quote:
Please don't spoil me with your recrementitious generosity.

Thanks for introducing me to a new vocabulary word.  I had to run off and look up "recrementitious".  I love learning new words.  I am sad, though, that it seems to mean "composed of waste matter or dross".  I guess my commentary left something to be desired in comparison to chessandgo's "valuable insight".  :-[
Title: Re: Proposed New Material Evaluation Formula
Post by The_Jeh on Jan 1st, 2007, 11:52am
My "recrementitious" is supposed to mean "excess" or "surplus." (Merriam-Webster's Collegiate Thesaurus)  That is a slightly more flattering connotation, I think.

The_Jeh
Title: Re: Proposed New Material Evaluation Formula
Post by The_Jeh on Jan 1st, 2007, 2:12pm
I have revised the PieceValue formula and tweaked the RabDiff component.  Here are all the variables and final formula again:

PieceValue = (9 - (# of pieces that can push it))(Rank) , where rabbits have a rank of 1, cats 2, elephants 6, etc.
SumPieceValue = sum of the friendly piece's PieceValues

RabDiff = (#FriendRabbits - #FoeRabbits)

YC = # of columns in which you have a piece or a rabbit, or a piece higher than a rabbit in an adjacent column
OC = " " " for opponent

E = ((SumPieceValue)(1/2)(1+(RabDiff/|RabDiff|)(RabDiff^2/192)))^(1+YC/8-OC/8)

So far, there are zero arbitrary numbers. 192 is 3(8^2).

Here are the initial values if White/Black start out down by the following hadicaps.  Positive differences favor white.

Handicap, Score, Difference
E/-, 73/114, -41
M/-, 80/113, -33
E/RRRR, 79/102, -23
E/RRRRRRR, 92/81, 11

The_Jeh
Title: Re: Proposed New Material Evaluation Formula
Post by The_Jeh on Jan 1st, 2007, 4:45pm
I forgot to mention,  if the RabDiff = 0, do not use that portion of the formula.  Otherwise, the answer will be undefined.  You know what my intention was.

Also, here are some more initial scores for various handicaps for White/Black.  Positive values favor white.

Handicap, Score, Difference
RRRRRRR/-, 72/126, -54
D/RR, 94/102, -8
DDCC/HH, 87/92, -5
R/-, 99/101, -2
DCC/HH, 98/86, 12
M/HH, 98/81, 17
EM/HHDDCC, 91/62, 29

The_Jeh
Title: Re: Proposed New Material Evaluation Formula
Post by The_Jeh on Jan 1st, 2007, 4:59pm
This formula is equivalent, simpler, and you don't have to worry about RabDiff being 0:

E = ((SumPieceValue)(1/2)(1+RabDiff|RabDiff|/192))^(1+YC/8-OC/8)
Title: Re: Proposed New Material Evaluation Formula
Post by Fritzlein on Jan 1st, 2007, 8:57pm
Sorry to fuss about "recrementitious"; I just wanted to make sure I hadn't offended you.

The heavy pieces seem to be valued more appropriately now.  If I understand correctly, a camel is worth exactly a horse and a cat as an initial trade, which is close enough to popular opinion.

It seems the formula now slightly prefers a horse over four rabbits, which is a bit too much in popular opinion.  I personally value a horse at exactly three rabbits, i.e. a bit less than FAME says, but even FAME still thinks the horse is worth less than four rabbits.  When you get up to a camel being worth five rabbits, that's close enough to accurate, because none of us really knows.

Squaring the difference in rabbits now seems to seriously undervalue just one or two rabbits.  For example, a cat is now considered worth a little more than two rabbits, but I think every Arimaa player would rather have the two rabbits.  In reality a cat is much closer in value to one rabbit than it is to two rabbits, to say nothing of it being worth more than two rabbits.

I think there is near unanimity among players that MR is better than HD, or at least equal, but the formula hardly notices the R and therefore distinctly prefers the HD.  Also DRR is clearly superior to H, but the formula likes the H.  Finally, the formula prefers D to CR, while most players would prefer CR, although this case is less clear-cut than the others.

In general I think rabbits are a little more than linearly additive.  Yes, two rabbits are worth more than twice one rabbit.  However, the effect is very small for the first few rabbits.  Your current formula thinks two initial rabbits are worth over three times as much as one initial rabbit, which doesn't correspond to anything about Arimaa that I am aware of.

Good luck with continued tweaking; for now, I don't trust that equation to protect my first rabbit enough.
Title: Re: Proposed New Material Evaluation Formula
Post by The_Jeh on Jan 2nd, 2007, 11:03pm
I think I can satisfy most of your stipulations, Fritzlein, but it is going to take another variable to do so.  Here are all the variables again for convenience:

PieceValue = (9-(# of pieces that can push it))(Rank), where rank is 1 for rabbits, 2 for cats, 6 for elephants, etc.
SumPieceValue = the sum of the PieceValue of all friendly pieces
RabDiff = #FriendRabbits - #FoeRabbits
HiCap = the Rank of the highest piece (or pawn) your opponent lost or started without
YC = the number of columns in which you have a piece or rabbit, or a piece higher than a rabbit in an adjacent column
OC = " " " for opponent


E = ((1/2)(SumPieceValue)(1+(RabDiff|RabDiff|^(1/8))/(3*8^(9/8)))+HiCap)^(1+YC/8-OC/8)

The result is rounded to the nearest integer.
Here are the critical initial handicaps for White/Black:

Handicap, Scores, Difference
MR/HD, 94/96, -2
DRR/H, 95/101, -6
CR/D, 101/102, -1
RR/C, 97/105, -8
R/C, 102/101, 1
RRRR/H, 93/100, -7
RRR/H, 98/97, 1

The_Jeh
Title: Re: Proposed New Material Evaluation Formula
Post by Fritzlein on Jan 13th, 2007, 9:09am

Quote:
E = ((1/2)(SumPieceValue)(1+(RabDiff|RabDiff|^(1/8))/(3*8^(9/8)))+HiCap)^(1+ YC/8-OC/8)

I tried to implement your latest formula, The_Jeh and I got different values than you are reporting.  For example, for DRR vs H, I have that the side missing DRR has a score of 97.25556682, while the side missing H has a score of 99.51518217.  Would you mind working through this example in detail, showing what value each component of your formula produces, so that I can find my error?
Title: Re: Proposed New Material Evaluation Formula
Post by The_Jeh on Jan 13th, 2007, 3:43pm
Certainly, Fritzlein:  
 
DRR/H  
For the side missing DRR:  

PieceValue = (9 - #PiecesThatCanPunuts )(Rank)
Rabbit = 2
Cat = 8
Dog = 18
Horse = 28
Camel = 40
Elephant = 54  

SumPieceValue=sum(Piecevalue*#ofPieces)  
SumPieceValue=(2*6+8*2+18*1+28*2+40*1+54*1)  
SumPieceValue=196  
 
Then multiply this by 1/2 to get 98.  
 
Now multiply 98 by {1+[RabDiff|RabDiff|^(1/8)]/[3*8^(9/8)]}, which is:  
 
{1+[(-2)(2)^(1/8)]/[3*8^(9/8)]}  
{1+[(-2)(1.090507733)]/[3*10.37471644]}  
{1+[-2.181015466]/[31.12414932]}  
{1-.0700747013}  
.9299252987  
 
98*.9299252987 = 91.13267927  
 
Then we have to add the hicap, which is 4, to get 95.13267927 .  
 
Finally we take this number to the power of (1+YC/8-OC/8), which is 1 initially.  Then, rounding to the nearest integer, the final answer is:  
 
E = 95  
 
I needn't go through the calculation for Black, which is similar.  Perhaps you had some order of operations issues?  Did you remember that White has only 6 rabbits and only 1 dog remaining?  
 
I have recently written a program that handles this formula with ease.  If you want any other specific combinations, I can churn them out quite quickly.    
 
The_Jeh
Title: Re: Proposed New Material Evaluation Formula
Post by The_Jeh on Jan 13th, 2007, 3:45pm
PieceValue = (9 - #PiecesThatCanPunuts
)(Rank)

A glitch is preventing me from writing this correctly.  I have no idea why.  I checked the non-smiley box.
Title: Re: Proposed New Material Evaluation Formula
Post by Fritzlein on Jan 13th, 2007, 4:21pm
OK, I see now.  I was taking the rabbit difference squared, then raising to the 1/8 power, even though the precedence of operations says to exponentiate first.
Title: Re: Proposed New Material Evaluation Formula
Post by The_Jeh on Jan 17th, 2007, 11:45pm
I have decided to lower the weight of column difference, although I still think it has value.  Here is the new formula:

E = {[1/2][SumPieceValue][1+(RabDiff|RabDiff|^(1/8))/(3*8^(9/8))]+HiCap}^{1+YC/24-OC/24}  


Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB © 2000-2003. All Rights Reserved.