Author |
Topic: Handicap Order - what beats what? (Read 7150 times) |
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender: 
Posts: 1413
|
 |
Re: Handicap Order - what beats what?
« Reply #15 on: Apr 18th, 2008, 11:22pm » |
 Quote  Modify
|
Ahh, thanks for the clarification mistre, that sounds like a much better method than what I (and Janzert) assumed you were doing. In fact I think I even agree with you that it is a better method than my "majority of evals" method (although yours is obviously much more time consuming).
|
|
 IP Logged |
|
|
|
chessandgo
Forum Guru
Arimaa player #1889
Gender:
Posts: 1244
|
|
Re: Handicap Order - what beats what?
« Reply #16 on: Apr 19th, 2008, 3:38am » |
Quote Modify
|
on Apr 18th, 2008, 1:36pm, Arimabuff wrote:
I think this is the right thing to do. If we try to make a list by taking each individual's opinion, it’ll take ten years to come up with an order that everyone will hate. So you might as well refer to an objective indicator like this one and stick to it. |
|
on Apr 18th, 2008, 11:59am, mistre wrote:Chessandgo - Thanks for your comments.
The results are only as good as the three current models. What I think this list does for the first time though is combine all 3 model evaluators into one composite ranking. |
|
Well, I wasn't suggesting to the least extent that the ranking should be any different, you did a great job, Mistre. Just wanted to profit from this thread to hear from other players how they approach unbalanced trades.
|
|
IP Logged |
|
|
|
chessandgo
Forum Guru
Arimaa player #1889
Gender:
Posts: 1244
|
|
Re: Handicap Order - what beats what?
« Reply #17 on: Apr 19th, 2008, 3:43am » |
Quote Modify
|
on Apr 18th, 2008, 7:06pm, 99of9 wrote:
Yes, I have a theoretical argument for the converse.
I agree with you that as both player's pieces are reduced, the number of pieces of pieces starts mattering more.
BUT, there is also something in the detail about which pieces you have. It's good to have a piece equal to the pieces your opponent has an excess of. One horse can partially neutralize an opponent with two horses, but zero horses cannot neutralize an opponent with one horse.
So after an H for CC trade, I would agree that it is beneficial for the CC player to trade rabbits, but it could well be beneficial for the H player to trade away one of his H's for the last of his opponent's.
(This is the reason for the "Equals" term in DAPE, which depreciates the value of a piece according to how many opponent's pieces are equal to it.) |
|
Yes, thanks for this point, Toby. But as Karl says, I would consider this to be significant only if the camels were gone. Moreover, if camels get traded, the number of pieces decreases again, making the extra piece even more interesting ... in my opinion. But I see your point.
So do you think an early exchange of HH vs HCC favorizes the side with hcc down ?
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: Handicap Order - what beats what?
« Reply #18 on: Apr 19th, 2008, 5:17am » |
Quote Modify
|
on Apr 19th, 2008, 3:43am, chessandgo wrote:So do you think an early exchange of HH vs HCC favorizes the side with hcc down ?
|
|
I'm not sure. It's close. All I'm saying is that there's something in each player's favour.
I agree that the factor is much stronger when the camels are missing (all 3 evals also agree on this). But even the fact that a camel trade is now favourable for the hcc player makes his position a little easier, because the other player has to watchfully prevent a camel trade.
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: Handicap Order - what beats what?
« Reply #19 on: Apr 19th, 2008, 5:51am » |
Quote Modify
|
on Apr 18th, 2008, 11:00pm, mistre wrote:| Let me clarify my ranking method. I did not use overall raw numbers but rankings instead. I ranked all 108 (now 111) handicaps 1 through 111 for each of the three measures. I then summed that number for each measure. The handicaps were then ranked from lowest total score to highest total score. |
|
Ahh, yes that does sound like a better method.
Janzert
|
|
IP Logged |
|
|
|
woh
Forum Guru
Arimaa player #2128
Gender:
Posts: 254
|
|
Re: Handicap Order - what beats what?
« Reply #20 on: Apr 19th, 2008, 6:50am » |
Quote Modify
|
Great list, mistre!
on Apr 17th, 2008, 3:50pm, mistre wrote:
I believe those are
MHHC
EHHC
EDDCC
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Handicap Order - what beats what?
« Reply #21 on: Apr 19th, 2008, 1:47pm » |
Quote Modify
|
on Apr 18th, 2008, 11:00pm, mistre wrote:| Let me clarify my ranking method. I did not use overall raw numbers but rankings instead. I ranked all 108 (now 111) handicaps 1 through 111 for each of the three measures. I then summed that number for each measure. The handicaps were then ranked from lowest total score to highest total score. |
|
So am I correct to state that we can't infer from your list whether EMHHDDCCRRRRR or EMHHDDCRRRRRRR is the greater handicap? I looks like, in order to get a relative ranking between the two, you would have to rerun with 113 handicaps in the list.
Would it also be correct to say that adding alternatives could change the relative rankings of items that are already in your list? For example, in the partial lists
ABC
ACB
ACB
your method would have C ahead of B overall, but if new alternatives came in like
ABDEFC
ADEFCB
ADEFCB
then the combined ranking would have B ahead of C, right?
I guess my point is that relative values of handicaps is an unstable way to do it, so to be fair one would presumably have to have _all_ handicaps in the list, or else raise the question of why one list and not another.
|
|
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: Handicap Order - what beats what?
« Reply #22 on: Apr 19th, 2008, 2:07pm » |
Quote Modify
|
Take the ranking generated by a Condorcet method with the chosen material evaluators being the voters and the 971 possible handicaps being the candidates. Problem solved.
|
|
IP Logged |
|
|
|
Arimabuff
Forum Guru
Arimaa player #2764
Gender:
Posts: 589
|
|
Re: Handicap Order - what beats what?
« Reply #23 on: Apr 19th, 2008, 2:43pm » |
Quote Modify
|
on Apr 19th, 2008, 1:47pm, Fritzlein wrote:
So am I correct to state that we can't infer from your list whether EMHHDDCCRRRRR or EMHHDDCRRRRRRR is the greater handicap? I looks like, in order to get a relative ranking between the two, you would have to rerun with 113 handicaps in the list.
Would it also be correct to say that adding alternatives could change the relative rankings of items that are already in your list? For example, in the partial lists
ABC
ACB
ACB
your method would have C ahead of B overall, but if new alternatives came in like
ABDEFC
ADEFCB
ADEFCB
then the combined ranking would have B ahead of C, right?
I guess my point is that relative values of handicaps is an unstable way to do it, so to be fair one would presumably have to have _all_ handicaps in the list, or else raise the question of why one list and not another. |
|
Karl I believe that for handicaps where the pieces are all but depleted we should see it the other way around, that is in this case whether having only CR in your camp is better than RRR. I think it makes things clearer to see. If we have only either an army of three rabbits to fight or a cat and a rabbit, we all know which one is best don't we?
I think when we consider it on the side of the handicap MHHDDCRRRRRRR versus MHHDDCCRRRRR the program will suffer from side effect miscalculations, due to the big accumulation of pieces that adds each its own uncertainty.
When you look it as RRR versus CR it suddenly makes it look plain and simple but maybe that’s not what we want?
I say it makes sense to look at things from their clearer and simpler perspective, that’s how we’ve been taught to organize our thoughts.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Handicap Order - what beats what?
« Reply #24 on: Apr 19th, 2008, 2:53pm » |
Quote Modify
|
If my previous post is correct, the question of whether CR beats RRR was not decided at the time that Arimabuff and 99of9 set their respective records against Gnobot2005P1. That means we now need to decide which is better, either directly, or by deciding on a methodology and accepting the result.
That is tricky enough, but there is an added complication that we are setting the finish line after the race has been run. Arimabuff stopped running after reducing the handicap to CR, because he thought the race was over. Unless Gnobot2005P1 can be beaten with only RR, the race is really over now, and decision we make will determine who holds the all time record. We can't decide that RRR was the greater feat now without being unfair to Arimabuff. On the other hand, we can't start the discussion knowing what we have to conclude, or it isn't a real discussion.
The more I think about whether CR or RRR is better, the more it seems to me that the two records are incommensurate, and the Hall of Fame should therefore reflect both. More generally, one handicap should not replace another in the Hall of Fame unless one handicap includes all the pieces of the other, plus more, or the two handicaps are equal in number of pieces and one is strictly better in strength.
To illustrate my reasoning, consider the following examples. In the first pair of boards (Silver to move) CR is clearly weaker than RRR
+---+---+---+---+---+---+---+---+
8 | | | | | | | | |
+---+---+---+---+---+---+---+---+
7 | | | | | | | | |
+---+---+---+---+---+---+---+---+
6 | | | * | | | * | | |
+---+---+---+---+---+---+---+---+
5 | c | | | M | E | | | r |
+---+---+---+---+---+---+---+---+
4 | R | | | H | H | | | R |
+---+---+---+---+---+---+---+---+
3 | | | * | D | D | * | | |
+---+---+---+---+---+---+---+---+
2 | | | | C | C | | | |
+---+---+---+---+---+---+---+---+
1 | | R | R | R | R | R | R | |
+---+---+---+---+---+---+---+---+
a b c d e f g h
+---+---+---+---+---+---+---+---+
8 | | | | | | | | |
+---+---+---+---+---+---+---+---+
7 | | | | | | | | |
+---+---+---+---+---+---+---+---+
6 | | | * | | | * | | |
+---+---+---+---+---+---+---+---+
5 | r | r | | M | E | | | r |
+---+---+---+---+---+---+---+---+
4 | R | | | H | H | | | R |
+---+---+---+---+---+---+---+---+
3 | | | * | D | D | * | | |
+---+---+---+---+---+---+---+---+
2 | | | | C | C | | | |
+---+---+---+---+---+---+---+---+
1 | | R | R | R | R | R | R | |
+---+---+---+---+---+---+---+---+
a b c d e f g h
In the second pair of boards (Silver to move), CR is clearly stronger than RRR
+---+---+---+---+---+---+---+---+
8 | | | | | | | | |
+---+---+---+---+---+---+---+---+
7 | r | c | | | | | | |
+---+---+---+---+---+---+---+---+
6 | R | R | * | | | * | | |
+---+---+---+---+---+---+---+---+
5 | | | | M | E | | | |
+---+---+---+---+---+---+---+---+
4 | | | | H | H | | | |
+---+---+---+---+---+---+---+---+
3 | | | * | D | D | * | | |
+---+---+---+---+---+---+---+---+
2 | | | | C | C | | | |
+---+---+---+---+---+---+---+---+
1 | | | R | R | R | R | R | R |
+---+---+---+---+---+---+---+---+
a b c d e f g h
+---+---+---+---+---+---+---+---+
8 | | | | | | | | |
+---+---+---+---+---+---+---+---+
7 | r | r | r | | | | | |
+---+---+---+---+---+---+---+---+
6 | R | R | * | | | * | | |
+---+---+---+---+---+---+---+---+
5 | | | | M | E | | | |
+---+---+---+---+---+---+---+---+
4 | | | | H | H | | | |
+---+---+---+---+---+---+---+---+
3 | | | * | D | D | * | | |
+---+---+---+---+---+---+---+---+
2 | | | | C | C | | | |
+---+---+---+---+---+---+---+---+
1 | | | R | R | R | R | R | R |
+---+---+---+---+---+---+---+---+
a b c d e f g h
We can't say in advance what combination of pieces is going to be most useful, because we don't know in advance what portion of the bot's army will be deployed, whether it will advance rabbits sooner or later, on one side of the board, or both, etc.
The material handicap formulas weren't designed for extreme situations such as a whole army against three little pieces. A handicap win is only possible because the bot doesn't use its whole army. If we use the formulas at all in this case, it might make more sense to ask whether RRR or CR is stronger against an opposing army of RRR. But then for a different bot that used its pieces differently, the true difficulty of each handicap would be reflected by a different formula.
Why not use a partial order (as we mathematicians call it) and keep both records where there is no clear order between them? One might say that we would have too many records per bot, but in practice I doubt it would get beyond two or three. Under my proposal we can't say whether a handicap of M or RRRR is better, but if both are possible then probably so is MRRR, which trumps them both. If we ended up with cases where MD was possible and also HH, but not MH, then why do we need to split hairs? Why not recognize both?
My hunch is that this will result in basically two handicap records per bot: a strength handicap and a numbers handicap. On the one side you'll be trying to win with just EMR, and on the other trying to win with just CRRRRRRR or something like that. For the bots that get extremely bashed, the two may even converge.
What do you guys think? Is it worth a try?
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Handicap Order - what beats what?
« Reply #25 on: Apr 19th, 2008, 3:02pm » |
Quote Modify
|
on Apr 19th, 2008, 2:07pm, aaaa wrote:| Take the ranking generated by a Condorcet method with the chosen material evaluators being the voters and the 971 possible handicaps being the candidates. Problem solved. |
|
Are you ranking each smaller army by how much it is less than a full army? That provides clarity, but as Arimabuff points out, the greater the mismatch, the more likely the numbers are to be meaningless. FAME, at least, wasn't designed for extreme situations.
Or do you mean ranking each army head to head? As Arimabuff suggests, we could enter CR vs. RRR directly into each evaluator. Unfortunately, FAME can have circular preferences, i.e. head-to-head preferences don't produce a strict ordering of all possible armies. For example, CCR > MR > DRR > CCR. So there may be more accuracy of the formulas that way, but less overall clarity.
|
| « Last Edit: Apr 19th, 2008, 3:10pm by Fritzlein » |
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: Handicap Order - what beats what?
« Reply #26 on: Apr 19th, 2008, 3:11pm » |
Quote Modify
|
on Apr 19th, 2008, 3:02pm, Fritzlein wrote:
Are you ranking each smaller army by how much it is less than a full army? That provides clarity, but as Arimabuff points out, the greater the mismatch, the more likely the numbers are to be meaningless. FAME, at least, wasn't designed for extreme situations. |
|
You might want to let additional material evaluators weigh in then, like optimized FAME, RabbitCurveABC, LinearAB, etc.
|
| « Last Edit: Apr 19th, 2008, 3:11pm by aaaa » |
IP Logged |
|
|
|
mistre
Forum Guru
Gender:
Posts: 553
|
|
Re: Handicap Order - what beats what?
« Reply #27 on: Apr 19th, 2008, 3:16pm » |
Quote Modify
|
Wow, a lot was said since I signed off yesterday...
After some more thought, I don't plan on re-working my ranking analysis to see if RRR or CR remaining is better. As Karl said, I can't just add those two scenarios to the list without adding all other 700+ scenarios that mix rabbits/non-rabbit pieces.
My original list is far from perfect, but I stand by using it over just 2/3 majority of evals. For 99% of the cases, it should work just fine. For a case like RRR vs CR remaining, it will not and we need another solution.
aaaa mentions using the condorcet model. I don't know how to use that, so if anyone wants to give it a go with all 971 combinations, be my guest.
|
|
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: Handicap Order - what beats what?
« Reply #28 on: Apr 19th, 2008, 3:20pm » |
Quote Modify
|
on Apr 19th, 2008, 3:02pm, Fritzlein wrote:| Or do you mean ranking each army head to head? |
|
No. We call it a handicap because one is playing with less than a full army against the full one of the opponent, so determining how large the handicap is would logically entail quantifying the resulting disadvantage between the two armies.
|
|
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: Handicap Order - what beats what?
« Reply #29 on: Apr 19th, 2008, 3:27pm » |
Quote Modify
|
on Apr 19th, 2008, 3:16pm, mistre wrote:
aaaa mentions using the condorcet model. I don't know how to use that, so if anyone wants to give it a go with all 971 combinations, be my guest.
|
|
Give me the evaluation functions you want to count (preferably a large and odd number, like, say, 5) and I'll try to give you the relative ranking as given by the Schulze(margins) method.
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print