Welcome, Guest. Please Login or Register.
Mar 28th, 2024, 3:53pm

Home Home Help Help Search Search Members Members Login Login Register Register
Arimaa Forum « (no) absolute score values for pieces? »


   Arimaa Forum
   Arimaa
   Bot Development
(Moderator: supersamu)
   (no) absolute score values for pieces?
« No topic | Next topic »
Pages: 1 ... 4 5 6 7 8  9 Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print
   Author  Topic: (no) absolute score values for pieces?  (Read 38078 times)
Janzert
Forum Guru
*****



Arimaa player #247

   


Gender: male
Posts: 1016
Re: (no) absolute score values for pieces?
« Reply #75 on: Jun 8th, 2009, 8:20pm »
Quote Quote Modify Modify

Added to my page; I think I've got it correct. aaaa, let me know if you see it producing any errors.
 
A few samples (all scores normalized to an initial rabbit):
 
E vs mhd = 0.16
M vs hcr = 0.74
H vs dc = 0.41
D vs rr = even
C vs rr = 0.73
 
Janzert
 
P.S. As long as 99 doesn't mind I really should reintegrate all of 99's work adding the different systems back into my page.
« Last Edit: Jun 8th, 2009, 8:21pm by Janzert » IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: (no) absolute score values for pieces?
« Reply #76 on: Jun 9th, 2009, 5:30am »
Quote Quote Modify Modify

Thanks Janzert.  One case where the aaaa evaluation impresses me is the camel for horse trade.  What is thrilling is not the exact number (1.90) by which the camel is worth more than the horse, but rather that the "even" trade of a pair of horses knocks down the advantage to 0.95.  This is strikingly close to my intuition that, after an initial M for H trade an addition horse swap is worth about a rabbit.  Kudos to bot_quad for knowing which side will benefit from this "even" trade, and by how much.
IP Logged

99of9
Forum Guru
*****




Gnobby's creator (player #314)

  toby_hudson  


Gender: male
Posts: 1413
Re: (no) absolute score values for pieces?
« Reply #77 on: Jun 9th, 2009, 6:03pm »
Quote Quote Modify Modify

on Jun 8th, 2009, 8:20pm, Janzert wrote:
P.S. As long as 99 doesn't mind I really should reintegrate all of 99's work adding the different systems back into my page.

Go ahead Janzert, I borrowed it all from you in the first place!
http://www.chem.usyd.edu.au/~hudson_t/arimaa/material_evals_new.html
IP Logged
aaaa
Forum Guru
*****



Arimaa player #958

   


Posts: 768
Re: (no) absolute score values for pieces?
« Reply #78 on: Jun 15th, 2009, 10:49am »
Quote Quote Modify Modify

If one considers the formula distinguishable enough to merit its own name, I was thinking of calling it "HarLog", a reference to the harmonic and logarithmic components of the system. Of course, that would be a blend rather than an acronym.
IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: (no) absolute score values for pieces?
« Reply #79 on: Jun 15th, 2009, 12:54pm »
Quote Quote Modify Modify

A catchy name was all that HarLog was missing to take its place as the premiere material evaluation function.  Perhaps you could even call it HarmLog for the benefit of the word play (i.e. a journal of the damage suffered by each side Wink).
 
Apparently FAME is now obsolete.  Yes, there are a few corner cases where my intuition agrees more with FAME than with HarLog (for example an initial trade of E for DCCRRRRRRR), but the main meat-and-potatoes exchanges that happen all the time seem to be handled a little bit better across the board by HarLog.
« Last Edit: Jun 15th, 2009, 12:55pm by Fritzlein » IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: (no) absolute score values for pieces?
« Reply #80 on: Jun 15th, 2009, 1:16pm »
Quote Quote Modify Modify

By the way, aaaa, I recall our discussion in which I maintained that two rabbits might always be worth more than a cat, and you proposed EC6R vs E8R as a possible counter-example.  I notice now that all four functions on Janzert's page disagree with me and prefer the cat to the two rabbits.  I'm afraid that my endgame play is so weak that the disagreement reflects badly on my intuition rather than reflecting badly on the (unanimous) material functions.
IP Logged

aaaa
Forum Guru
*****



Arimaa player #958

   


Posts: 768
Re: (no) absolute score values for pieces?
« Reply #81 on: Jun 16th, 2009, 3:41am »
Quote Quote Modify Modify

On the contrary, it was the very fact that you treated the question of which side would be better in this situation as being very much open to discussion that made me settle on the current incarnation of the function with its lack of lopsided evaluation for one side or the other (not much more than the advantage of an initial rabbit).
« Last Edit: Jun 16th, 2009, 6:01am by aaaa » IP Logged
Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: (no) absolute score values for pieces?
« Reply #82 on: Jul 3rd, 2009, 7:09am »
Quote Quote Modify Modify

on Jul 2nd, 2009, 10:39am, Janzert wrote:
I've finally finished integrating 99of9's additional evaluation systems back into the page and also added piece images. In doing so I used a few css and javascript techniques I've not used before so there is certainly a possibility for browser compatibility problems.
 
The location was also changed to better reflect that it shows multiple evaluation systems instead of just FAME. The new url is http://arimaa.janzert.com/eval.html (there is a redirect setup for the old url so old links should continue to work for now).
 
Let me know if you see any errors or have another eval system you'd like to see added.
 
Janzert

Thanks, Janzert.  You inspired me to try out some more material states to see how the evaluators agree with my intuition.  I noticed something interesting about the M for HD trade.
 
I had seen before that HarmLog gives a significant edge to HD over M, whereas FAME and DAPE put it about even.  I think my intuition is somewhere in between, so it wasn't much worth commenting on.  But what I just now noticed is the effect of further removing CC from each side.  My intuition is that even trades are slightly disadvantageous to the M side, because the more the board thins out, the more important sheer numbers are relative to having the strongest pieces.  FAME and DAPE both agree with me, slightly preferring HD over M after a trade of CC.  HarmLog, in contrast, thinks the trade helps the M side and reduces the advantage of the HD side.
 
Of course, my intuition about this case could well be wrong, as it has been wrong about other cases in the past.  If quad enters the 2010 Computer Championship, it will be interesting to see it in action against FAME bots, because there will be some equal trades that both sides will be angling for.
IP Logged

Hippo
Forum Guru
*****




Arimaa player #4450

   


Gender: male
Posts: 883
Re: (no) absolute score values for pieces?
« Reply #83 on: Feb 17th, 2010, 2:09am »
Quote Quote Modify Modify

Fritzlein is much further with his bot_Nimrod than me with bot_Hippo... I like both FAME and HarmLog evaluations. FAME is good in positions where you could put equally "ordered" pieces next one to the other, but as said it suffers in EHx against mdx or ccx giving equal results. I would probably go for something in between (considering not only equally ordered matches, but giving equally ordered matches more weigh).
 
I am planning to add a "quiet" multiplication factor to material evaluation not to be fixed too much on material in races. How to compute the factor is another question.
 
Last joke: HHCx against mccx is not as good as HDCx at least theoretically the difference is the 3 repetitions rule.  It would be difficult to find an example when it will change the game result so not including it in material evaluation would rarely cause any harm.
 
Hmm it would need a lot of work to set coefficiens well.
 
So far I have:
1) the FAME rabbit evaluation  
2) let SGNi,j is result of comparison of i-th strongest gold piece with j-th strongest silver piece.
I have matrix of coefficients Ci,j. Sum of coordinate-wise multiplication of these two matrices is second summand.
3) last summand contains 1 for each presented stone type and 10000 for presented rabbit (added for gold and subtracted for silver).
 
C is symmetric: first try
(250  27  10   3   1   0   0   0)
( 27  90  19   7   2   0   0   0)
( 10  19  60  13   6   1   0   0)
(  3   7  13  40   9   2   0   0)
(  1   2   6    9   30  3   1   0)
(  0   0   1   2   3  10   3   1)
(  0   0   0   0   1   3  10   3)
(  0   0   0   0   0   1    3   7)
 
BTW: Having C diagonal with diagonal  
256      85      57      38      25      17      11      7 gives original FAME (first 2 summands). May be I had to start much nearer to this matrix.
 
So Hippo Adjusted Fritz Arimaa Material Evaluation could be good name for it (HAFAME).
 
I suppose the C matrix would be changed either for improving the evaluation or for speedup reasons to make the computation easier.
« Last Edit: Feb 17th, 2010, 3:27pm by Hippo » IP Logged

Hippo
Forum Guru
*****




Arimaa player #4450

   


Gender: male
Posts: 883
Re: (no) absolute score values for pieces?
« Reply #84 on: Feb 22nd, 2010, 12:31pm »
Quote Quote Modify Modify

on Mar 18th, 2006, 4:49pm, Janzert wrote:
Hmm, I must have something wrong or FAME's a little wierder than I thought.
 
Try EMHHDDCCR vs EM and a variable number of R's.
 
For 8-6 silver r's the score goes up for gold as expected. But then 5-0 r's the score actually decreases for gold with each additional r captured. Sad
 
FYI, I'm using the modification (clarification?) by Fritzlein to allow negative rabbits left over.
 
Janzert

 
It seems to me gold score increases by each captured silver rabbit even in this case. So no need for fixed 30 points for "negative rabbits" (33 FAME (flored) points for initial captured rabbit is the minimal dynamics rabit value).
 
on Mar 18th, 2006, 7:45pm, Fritzlein wrote:

Here's a genuine weirdness with FAME:
 
EMHHDDCCR vs. ER = +652
EHHDDCCR vs. ER = +640
 
It only lowers Gold's evaluation by 12 points to throw away a camel.  Although it leaves Silver with only -5 rabbits instead of -6, it simultaneously weakens Gold's defense from 17 to 15, and against negative offense, FAME thinks a weaker defense is better!
 
To stop that silliness, each negative leftover rabbit should have a fixed value of, let's say, 40 points to the other team regardless of the size of the larger army.  Not that it matters much, but why not patch holes that are easy to patch?  There will still be enough unpatchable holes left. Smiley

 
Oh so this was the reason ... I am thinking of following "rabbits" scoring ... take difference of number of pieces (including rabbits) side with more pieces obtains points say the difference * 600/(opponent defense = 2*nonrabbits+rabbits).
 
It seems to me HarmLog overvalues the rabbits ... the log part is much higher than the other part.
« Last Edit: Feb 22nd, 2010, 12:42pm by Hippo » IP Logged

Fritzlein
Forum Guru
*****



Arimaa player #706

   
Email

Gender: male
Posts: 5928
Re: (no) absolute score values for pieces?
« Reply #85 on: Feb 22nd, 2010, 6:20pm »
Quote Quote Modify Modify

Ah, a matrix is a nice way to value non-direct comparisons, and it seems quite relevant.  I look forward to further iterations of HAFAME once the theory meets reality.
IP Logged

Hippo
Forum Guru
*****




Arimaa player #4450

   


Gender: male
Posts: 883
Re: (no) absolute score values for pieces?
« Reply #86 on: Feb 24th, 2010, 12:55pm »
Quote Quote Modify Modify

I have used matrix just as an easy way to describe the algorithm.
Of course it would be hard coded. Important thing is that only top left corner bounded by maximal "nonrabbit pieces size" is used.
(Probably jump table would be used for this matrix cut, fastest implementation of score += sgn (a-b) * Ci,j would be score' += (b-a)&C'i,j, score' -= (a-b)&C'i,j where ' values are shifted 3 bits left ... no conditional jumps).
 
With the other rabbit evaluation ... I would probably end with HAME acronym. ... I am not sure it's OK but initial exchange of 2 horses for 5 rabbits is considered advantage for player with more pieces in HAME.
 
Actually I use more FAME like
(213, 21,  4, 0, ...)
( 21, 54, 13, 2, 0, ...)
(  4, 13, 40,  7,  1,   0, ...)
(  0,  2,  7, 27,  4,   1,  0, ...)
(  0,  0,  1,  4, 18,   3,  0, ...)
(  0,  0,  0,  1,  3, 12,  2,  0)
(  0,  0,  0,  0,  0,  2,  7,  1)
(  0,  0,  0,  0,  0,  0,  1,  6)
« Last Edit: Feb 24th, 2010, 1:12pm by Hippo » IP Logged

aaaa
Forum Guru
*****



Arimaa player #958

   


Posts: 768
Re: (no) absolute score values for pieces?
« Reply #87 on: Feb 24th, 2010, 3:29pm »
Quote Quote Modify Modify

I was asked whether there is a theoretical justification for my material evaluation function and I can offer a partial one in the form of the fact that it contains two degrees of freedom; I consider this to be ideal on account of the fact that this number corresponds nicely with the three (main?) dimensions by which an army can be measured: the quantity of the pieces, their quality and their goal potential. I encourage anyone to suggest alternative parameter values for HarLog if the outputted values feel off in any of these respects. Rabbits overvalued, you say? Try lowering 'G' and see what values you get then. I'd personally be wary of adding several parameters on a whim though, because that runs the risk of overfitting.
IP Logged
Hippo
Forum Guru
*****




Arimaa player #4450

   


Gender: male
Posts: 883
Re: (no) absolute score values for pieces?
« Reply #88 on: Feb 25th, 2010, 7:38am »
Quote Quote Modify Modify

Sorry aaaa, I have had problems with log(0) term Smiley, but it was bad interpretation ... multiplied by friendly nonrabbits instead of all friendly pieces.
 
Actually when there is no log(0) problem, I valuate rabbits more Smiley ... (C6R/d3r), (C6R/dcr), (6R/c2r) or (2C7R/hdr) are advantages for gold in HAME and for silver in HAR(M)LOG (if I don't have a bug there).
 
It's good there are several evaluation functions Smiley, I am not sure with current "co"processors speed. Causes the often divisions and the logarithm problem with speed?
 
Follows stronger advantages for gold in HAME and stronger advantages for silver in HAR(M)LOG
(2HD2C8R/m2hr) and (2H2D2C7R/m2hr).
 
Similar comparison HAMExFAME (but roughly 7 times smaller advantages):
(MD2C8R/eh2dcr) (M2DC8R/e2hdcr) (M2DC8R/e2h2dr) (MH2C8R/em2dcr) (MH2D2C6R/em2hdr)
 
HAR(M)LOGxFAME:
(M2HR/2hd2c8r) (M2HR/2h2d2c8r)
 
Both (2DC8R/em2hr) (2H2D2CR/emhr) are considered slight advantage for gold in HAME, but I am really not sure with that.
In the former case ... does silver have enough pieces to take control of both home traps and prevent goaling?
In the later case ... it seems to me silver can go for elimination and the lot of weak nonrabbit pieces does not help gold.
« Last Edit: Apr 27th, 2013, 7:22am by Hippo » IP Logged

jdb
Forum Guru
*****



Arimaa player #214

   


Gender: male
Posts: 682
Re: (no) absolute score values for pieces?
« Reply #89 on: Jul 28th, 2010, 2:13pm »
Quote Quote Modify Modify

I have been doing some tests with the various material evaluators using Janzert's roundrobin program. The games are 10 sec per move, so a game takes around 10 minutes.  I'll post the results when there are enough games.
 
Assume there is a H for d trade. Who benefits from equal trades? The side with the extra H benefits from equal trades of camels or horses. This gets them closer to having the strongest piece. What about equal trades of dogs, cats or rabbits?  
 
Assume one side has an extra rabbit. Who benefits from equals trades? Trading rabbits eventually leads to 2 rabbits vs 1 rabbit. The extra rabbit becomes a huge advantage. What about trading cats, dogs, horses or camels? Eventually this leads to E vs e with an extra rabbit. This also looks like a big advantage for the extra rabbit.
 
Assume one side has an extra camel. Who benefits from equal trades? Fame/Harlog puts the initial advantage at 5.64/6.48. With everything but the rabbits traded off, leaving EM8R vs E8R, Fame/Harlog is 6.38/3.80. Finally EMR vs er, FAME/Harlog is 8.46/5.31. This looks like an area for improvement. What is correct in this case?  
 
The reason I am asking about this, is I was hoping to come up with a set of basic tests a material evaluator needs to pass. Things like a camel is worth more than a horse etc. It takes alot of time to get enough test games and I wanted a way to filter the different evaluators.
 
IP Logged
Pages: 1 ... 4 5 6 7 8  9 Reply Reply Notify of replies Notify of replies Send Topic Send Topic Print Print

« No topic | Next topic »

Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1!
YaBB © 2000-2003. All Rights Reserved.