Author |
Topic: Handicap Order - what beats what? (Read 7662 times) |
|
mistre
Forum Guru
Gender: 
Posts: 553
|
 |
Handicap Order - what beats what?
« on: Apr 17th, 2008, 3:50pm » |
 Quote  Modify
|
It took a while and was tedious, but I took the values from each of the 3 material evaluators - http://arimaa.janzert.com/fame.html and ranked each major type of handicap.
I say major, because I calculated there are actually 815 different handicap combinations (keeping at least 1 rabbit). For this exercise, I did not include any combination that included both non-rabbit pieces and rabbit pieces. This dropped the number to a more manageable 108 combinations.
My reasoning for not including the rabbit/non-rabbit combinations is that typically when handicap botbashing, you will start with the heavy pieces first and only add rabbits at the end. So I don't think there would be any reason to compare something crazy like HDCR and CRRRRRR. I did leave in rabbit-only handicaps to see how they compared.
After I ranked all 108 combinations for each evaluator, I summed the rankings to give a composite score. I think this is more accurate than just doing a 2/3 majority rule (this was the tiebreaker).
Some interesting findings:
H = CC. FAME and DAPE (opt) prefer CC, while DAPE prefers H. The average rankings for all 3 makes the comparison equivalent.
7 rabbits = MHC. This would make a neat asymmetrical handicap match.
In Arimaa, E = MHC2
Here is the entire list from lowest handicap to highest.
R
C
D
RR
RRR
CC
H
DC
DD
RRRR
HC
DCC
HD
M
DDC
RRRRR
HCC
HH
HDC
MC
HDD
MD
DDCC
HHC
RRRRRR
HHD
MCC
HDCC
MH
HDDC
MDC
MDD
HHCC
HHDC
MHC
RRRRRRR
MDCC
MHD
HHDD
HDDCC
MDDC
E
MHCC
MHH
HHDCC
MHDC
EC
HHDDC
MHDD
MDDCC
ED
MHDCC
MHHD
ECC
MHDDC
EH
HHDDCC
EDC
EDD
MHHCC
MHHDC
EHC
EHD
MHHDD
MHDDCC
EDCC
EDDC
EHH
EM
EHCC
MHHDCC
EHDC
EHDD
MHHDDC
EMC
EMD
EHHD
EHDCC
EHDDC
EMCC
EMH
EHHCC
MHHDDCC
EMDC
EMDD
EHHDC
EHHDD
EMHC
EHDDCC
EMDCC
EMHD
EMDDC
EHHDCC
EMHCC
EHHDDC
EMHDC
EMHH
EMDDCC
EMHDD
EMHHC
EMHDCC
EMHDDC
EHHDDCC
EMHHD
EMHHCC
EMHDDCC
EMHHDC
EMHHDD
EMHHDCC
EMHHDDC
EMHHDDCC
Edit: I found 3 more, there is 111 listed now. 3 more to find.
|
| « Last Edit: Apr 18th, 2008, 4:21pm by mistre » |
 IP Logged |
|
|
|
Arimabuff
Forum Guru
Arimaa player #2764
Gender:
Posts: 589
|
|
Re: Handicap Order - what beats what?
« Reply #1 on: Apr 18th, 2008, 6:45am » |
Quote Modify
|
You did a great job here!
maybe you should put a link in the handicap rules part of the botbasher page to this list.
|
|
IP Logged |
|
|
|
Arimabuff
Forum Guru
Arimaa player #2764
Gender:
Posts: 589
|
|
Re: Handicap Order - what beats what?
« Reply #2 on: Apr 18th, 2008, 6:55am » |
Quote Modify
|
on Apr 17th, 2008, 3:50pm, mistre wrote:| ...In Arimaa, E = MHC2... |
|
Only in relative terms. 
|
|
IP Logged |
|
|
|
Arimabuff
Forum Guru
Arimaa player #2764
Gender:
Posts: 589
|
|
Re: Handicap Order - what beats what?
« Reply #3 on: Apr 18th, 2008, 7:09am » |
Quote Modify
|
on Apr 17th, 2008, 3:50pm, mistre wrote:...Oops, I think I missed a few - EMHDDC for one. Please let me know if you see any others and I will re-do the analysis.
|
|
Actually there should be 2x2X3X3X3=108-1 = 107 (If you exclude "nothing" from the list) combinations, if you don't count the rabbits. Plus 1 to 7 rabbits (since you need at least one rabbit to win) = 107 + 7
= 114 combinations.
However, your list has only 108 items, therefore there must still be 6 combinations missing.
|
|
IP Logged |
|
|
|
chessandgo
Forum Guru
Arimaa player #1889
Gender:
Posts: 1244
|
|
Re: Handicap Order - what beats what?
« Reply #4 on: Apr 18th, 2008, 11:24am » |
Quote Modify
|
Thanks for this interesting list, Mistre.
I went trough the start of the list, and I disagree with a few things. For instance, it does not make sense, in my opinion, that H<CC while HH>HCC. Having a piece less makes the number of pieces even more important. I'm not sure what I'd prefer between H and CC, but I definitely prefer having HCC to HH.
Similarly, M and HD are probably roughly equal, but MD should be worth less than HDD, by the same principle that the less pieces you have, the more their number is important relatively to their strength.
Or are there theoretical arguments for the converse ?
|
|
IP Logged |
|
|
|
Arimabuff
Forum Guru
Arimaa player #2764
Gender:
Posts: 589
|
|
Re: Handicap Order - what beats what?
« Reply #5 on: Apr 18th, 2008, 11:35am » |
Quote Modify
|
on Apr 18th, 2008, 11:24am, chessandgo wrote:Thanks for this interesting list, Mistre.
I went trough the start of the list, and I disagree with a few things. For instance, it does not make sense, in my opinion, that H<CC while HH>HCC. Having a piece less makes the number of pieces even more important. I'm not sure what I'd prefer between H and CC, but I definitely prefer having HCC to HH.
Similarly, M and HD are probably roughly equal, but MD should be worth less than HDD, by the same principle that the less pieces you have, the more their number is important relatively to their strength.
Or are there theoretical arguments for the converse ? |
|
I my view the NUMBER of pieces outweighs their relative strengths in game endings when what's important is the ground you're covering. In the beginning of a game, it's pretty much the reverse. We all know how deadly it can be if you block someone's elephant even if you commit half of your pieces to the task! While in the end of a game, an elephant can't even outrun two distant rabbits. Therefore, you may try to think of these pieces as in the context of a cluttered board and it might change your perspective a little.
|
|
IP Logged |
|
|
|
mistre
Forum Guru
Gender:
Posts: 553
|
|
Re: Handicap Order - what beats what?
« Reply #6 on: Apr 18th, 2008, 11:59am » |
Quote Modify
|
Chessandgo - Thanks for your comments.
The results are only as good as the three current models. What I think this list does for the first time though is combine all 3 model evaluators into one composite ranking.
|
|
IP Logged |
|
|
|
Arimabuff
Forum Guru
Arimaa player #2764
Gender:
Posts: 589
|
|
Re: Handicap Order - what beats what?
« Reply #7 on: Apr 18th, 2008, 1:36pm » |
Quote Modify
|
on Apr 18th, 2008, 11:59am, mistre wrote:Chessandgo - Thanks for your comments.
The results are only as good as the three current models. What I think this list does for the first time though is combine all 3 model evaluators into one composite ranking. |
|
I think this is the right thing to do. If we try to make a list by taking each individual's opinion, it’ll take ten years to come up with an order that everyone will hate. So you might as well refer to an objective indicator like this one and stick to it.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Handicap Order - what beats what?
« Reply #8 on: Apr 18th, 2008, 2:09pm » |
Quote Modify
|
on Apr 18th, 2008, 11:59am, mistre wrote:| The results are only as good as the three current models. |
|
Yeah, and I freely admit that FAME stinks. The best I can say for it is that FAME is better than a static piece evaluation, which is what I was competing against when I designed it. Oh, and it is well-defined. Chessandgo is absolutely right that after a trade of H for CC, the side with the cats benefits more from trading H for H. I'm pretty sure H > CC but I don't know about HH for HCC. It's painful to imagine having that kind of discussion a hundred times to get the handicap order right. 
|
| « Last Edit: Apr 18th, 2008, 2:14pm by Fritzlein » |
IP Logged |
|
|
|
mistre
Forum Guru
Gender:
Posts: 553
|
|
Re: Handicap Order - what beats what?
« Reply #9 on: Apr 18th, 2008, 4:25pm » |
Quote Modify
|
on Apr 18th, 2008, 2:09pm, Fritzlein wrote:
Yeah, and I freely admit that FAME stinks. |
|
The worst thing about FAME is that once you get to multiple piece handicaps it thinks they are equivalent.
For example, it ranks the 4-piece handicaps EHCC through EMHH as equal. This first occurs with HDC and HDD.
|
| « Last Edit: Apr 18th, 2008, 4:27pm by mistre » |
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: Handicap Order - what beats what?
« Reply #10 on: Apr 18th, 2008, 5:11pm » |
Quote Modify
|
I'm planning to come up with my own material evaluator in the form of a multilayer perceptron based on game statistics. Here's hoping it will have better output than garbage.
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender:
Posts: 1413
|
|
Re: Handicap Order - what beats what?
« Reply #11 on: Apr 18th, 2008, 7:06pm » |
Quote Modify
|
on Apr 18th, 2008, 11:24am, chessandgo wrote:I went trough the start of the list, and I disagree with a few things. For instance, it does not make sense, in my opinion, that H<CC while HH>HCC. Having a piece less makes the number of pieces even more important. I'm not sure what I'd prefer between H and CC, but I definitely prefer having HCC to HH.
Or are there theoretical arguments for the converse ? |
|
Yes, I have a theoretical argument for the converse.
I agree with you that as both player's pieces are reduced, the number of pieces of pieces starts mattering more.
BUT, there is also something in the detail about which pieces you have. It's good to have a piece equal to the pieces your opponent has an excess of. One horse can partially neutralize an opponent with two horses, but zero horses cannot neutralize an opponent with one horse.
So after an H for CC trade, I would agree that it is beneficial for the CC player to trade rabbits, but it could well be beneficial for the H player to trade away one of his H's for the last of his opponent's.
(This is the reason for the "Equals" term in DAPE, which depreciates the value of a piece according to how many opponent's pieces are equal to it.)
|
| « Last Edit: Apr 18th, 2008, 7:07pm by 99of9 » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Handicap Order - what beats what?
« Reply #12 on: Apr 18th, 2008, 9:17pm » |
Quote Modify
|
on Apr 18th, 2008, 7:06pm, 99of9 wrote:| So after an H for CC trade, I would agree that it is beneficial for the CC player to trade rabbits, but it could well be beneficial for the H player to trade away one of his H's for the last of his opponent's. |
|
That's a good point, and one that would weigh even more heavily if the camels were missing. Maybe I'll have to reconsider my intuition.
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: Handicap Order - what beats what?
« Reply #13 on: Apr 18th, 2008, 10:39pm » |
Quote Modify
|
on Apr 17th, 2008, 3:50pm, mistre wrote:| After I ranked all 108 combinations for each evaluator, I summed the rankings to give a composite score. I think this is more accurate than just doing a 2/3 majority rule (this was the tiebreaker). |
|
While I don't want to step into the middle of the controversy over what order things should be ranked in, I do think this approach is flawed because of the way I currently present the material eval scores on that page.
The scores shown are normalized for the first rabbit captured (i.e. all the methods show 1.0)1. But the three methods still operate on different scales (i.e. once there is only a rabbit left FAME gives 162, DAPE 284 and DAPE(eo) 56). This means that for the high end sacrifices with the above summation method you are essentially giving DAPE 1.75 times the weight of FAME and 5 times the weight of DAPE(eo). So for the current numbers a simple count as 99 originally proposed is almost certainly better.
I have several times in the past thought about rescaling the numbers so the could be more directly compared. But the problem is that while it's very natural to say that the first rabbit should be 1.0 there isn't any similiar self apparent spot to pin the top end. Or looked at another way the current numbers directly mean DAPE thinks one rabbit remaining is like 284 initial rabbits while DAPE(eo) thinks it's like 56 initial rabbits. Scaling the methods to all go 1 through 100 or some such makes the numbers a little more abstract but more directly comparable. Maybe I'll just modify it to show both the current (initial rabbit) number and a new (scaled range) number.
Janzert
1 The actual raw values for the initial rabbit are 33.69 for FAME, 4.28 for DAPE and 10.44 for DAPE(eo). So had the raw numbers been left on the page DAPE would have had a smaller weighting while FAME and DAPE(eo) would have had larger weightings in the above calculation.
|
| « Last Edit: Apr 18th, 2008, 10:43pm by Janzert » |
IP Logged |
|
|
|
mistre
Forum Guru
Gender:
Posts: 553
|
|
Re: Handicap Order - what beats what?
« Reply #14 on: Apr 18th, 2008, 11:00pm » |
Quote Modify
|
Janzert,
Let me clarify my ranking method. I did not use overall raw numbers but rankings instead. I ranked all 108 (now 111) handicaps 1 through 111 for each of the three measures. I then summed that number for each measure. The handicaps were then ranked from lowest total score to highest total score.
So overall raw numbers didn't matter, just what order each of the 3 evaluators placed each handicap.
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print