Author |
Topic: Handicap Order - what beats what? (Read 7161 times) |
|
Janzert
Forum Guru
Arimaa player #247
Gender: 
Posts: 1016
|
 |
Re: Handicap Order - what beats what?
« Reply #60 on: Apr 22nd, 2008, 8:28pm » |
 Quote  Modify
|
on Apr 22nd, 2008, 7:31pm, Fritzlein wrote:
Read reply #24 in this thread to understand what I meant by incommensurate, and my proposal to deal with it. But I used the wrong word. I meant "incommensurable", meaning they can't be compared.
We can know for sure that CRR > CR > RR, but we can't compare CR to RRR or DRR to CCR. My idea is not to have a separate, fully-ordered ranking for each bot, but to treat as records all handicaps for a bot that aren't clearly beaten by some better handicap. |
|
Ahh, ok. First a minor nitpick, saying we can't compare two things is saying we can't tell whether green, water or 23 grams is better. I think all handicaps are comparable, whether we can figure out if a particular handicap is better, worse than or equal to another handicap is another matter. Having records for all handicaps that are equal, within the margin of error or plain undecidable on which is better is fine with me. In actual fact I care very little how the records are listed.
You say we know CRR > CR > RR and that we can't for CR-RRR and DRR-CCR. While I completely agree with you on the former and have no opinion at all on the latter, how do you decide this is the case? The method for deciding this is what interests me.
Janzert
|
|
 IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Handicap Order - what beats what?
« Reply #61 on: Apr 22nd, 2008, 8:30pm » |
Quote Modify
|
on Apr 22nd, 2008, 2:14pm, IdahoEv wrote:| One can argue that the game history doesn't actually represent the value of the pieces, and you might even be correct! But it's definitely an uphill argument. |
|
I think that playing more aggressively recently has given me an insight I didn't have before into unbalanced trades. I have always been a control player rather than a race player. That is to say, I don't want to capture more of your pieces than you capture of mine; I want to capture something of yours in return for nothing. Race players, on the other hand, are willing to give in order to get. They will slug it out as long as they are getting the best of it. In the past I have been unwilling to race even when I would get the best of the race, because I didn't want to lose control. This unwillingness to race often came back to haunt me, because it isn't always possible to keep everything under control, even with a positional or material advantage. Sometimes you have to race or lose your control, but if you are willing to race, you can trade control for a race that favors you. Being able to identify and accept those favorable races actually makes control more valuable to me.
In comparing M to HR as an opening trade, it is clear that the M is better for control, i.e. more likely to produce a free capture for nothing. The HR on the other hand, is better for racing and slugging it out, because every equal trade of pieces favors the HR side, most particularly a trade of H for H. The question of who the material balance favors overall can be translated into whether the M side can force the game into a control game, or the HR can force the game into a slugfest.
Naturally the answer is not absolute. One can always force the game into a race if one is willing to pay a price. So maybe a better way of putting it is in terms of tradeoffs: what will the HR player have to pay to force the game into a race, and/or what will the M player have to pay to keep the game a control game.
This language gives me a new way of expressing why I think M is superior to HR. Racing is, in essence, having the elephants apart. Each player tries to do damage with his own elephant rather than defend any damage the opposing elephant does. But when I have M and my opponent doesn't, I can fearlessly use my elephant to track his elephant, and defend anything his elephant tries to get started. I can become Mr. Tag-along. It doesn't bother me if both our elephants get bound up in a defensive deadlock, because I will have the strongest free piece (M) if that happens. Meanwhile my opponent's elephant can't afford to track my elephant and stop whatever I am up to. He will always have to leave, complicate, and threaten to trade.
I submit that races and slugfests are easier to understand than the control game, and that the weaker the players are, the better HR fares compared to M. Even a player as strategically strong as Omar has made the mistake of racing when he is ahead by M for H, even though he wouldn't have had to race. (I specifically remember scolding him for this.  ) In a race strength is not so important, and the advantage of extra strength evaporates.
I submit that the strongest players prefer M to HR, because that's what is more useful in games against each other. The more one understands the control game, the more one understands when and how not to race.
You may say that I have an uphill argument because I am arguing against the data, but you also have rather an uphill argument against the intuitions of strong players. If I pitted two random steppers against each other, I wouldn't be surprised if the game results favored R over E when the two players start with those respective handicaps. (Actually, that experiment wouldn't be too hard to run. 99of9 could tell us in an afternoon whether E or R is worth more to a random stepper.) But if R happens to be favored over E in that set of games, who cares? The strength of the players invalidates the data.
I argued in a different thread that an M handicap is only a moderate advantage between beginners, but is decisive among experts. The value of the extra piece increases with playing strength. I fully expect that the same is true of the value of M versus HR: the stronger the players are, the more the full force of the M will be felt. Of course, it wouldn't have to be this way; in theory it could be that the stronger the players are, the more effectively they are able to use an advantage in numbers. But I think in reality the control game is generally more subtle and difficult than the race game, and understood later in one's Arimaa career.
This is just my $0.02 that the "reverse causality" argument is not the only argument that undermines the game database. The "data" that will really convince me is when chessandgo uses his HR to beat my M. 
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Handicap Order - what beats what?
« Reply #62 on: Apr 22nd, 2008, 8:39pm » |
Quote Modify
|
on Apr 22nd, 2008, 8:28pm, Janzert wrote:| You say we know CRR > CR > RR and that we can't for CR-RRR and DRR-CCR. While I completely agree with you on the former and have no opinion at all on the latter, how do you decide this is the case? The method for deciding this is what interests me. |
|
What I propose is that handicap A is bigger than handicap B if and only if the pieces in A can be set against the pieces in B so that every piece in B is "covered" by at least an equal piece in A, plus
1) A has at least one piece left over
or
2) A has the stronger piece in at least one pair.
DRR can't cover both cats in CCR, and CCR can't cover a D at all, so the two are incommensurable, and both would have to be listed in the Hall of Fame. However, DCR can cover each of them while meeting criterion (2), so it would bump both of the previous handicaps out.
|
|
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: Handicap Order - what beats what?
« Reply #63 on: Apr 22nd, 2008, 8:59pm » |
Quote Modify
|
So ER=MRR or any handicap leaving 3+ pieces but without an E?
Janzert
|
| « Last Edit: Apr 22nd, 2008, 9:01pm by Janzert » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Handicap Order - what beats what?
« Reply #64 on: Apr 22nd, 2008, 10:09pm » |
Quote Modify
|
on Apr 22nd, 2008, 8:59pm, Janzert wrote:| So ER=MRR or any handicap leaving 3+ pieces but without an E? |
|
Yep. But don't use the equal sign. MCR beats one of those and not the other. EC beats the other but not the one. So they aren't equal, just unordered compared to each other. It takes ERR to beat them both.
|
| « Last Edit: Apr 22nd, 2008, 10:12pm by Fritzlein » |
IP Logged |
|
|
|
Janzert
Forum Guru
Arimaa player #247
Gender:
Posts: 1016
|
|
Re: Handicap Order - what beats what?
« Reply #65 on: Apr 22nd, 2008, 11:32pm » |
Quote Modify
|
I find it hard to believe botbasher's will be satisfied unable to distinguish between ER on the board and MHHDDCCRRRRRRRR on the board for a handicap. If the botbasher's are willing to agree to it though, I suppose I won't fight against it. Since what it does rank I think it probably ranks correctly I just think it leaves too much unranked.
[Edit: or maybe I'm being confused about which pieces are left on the board and which are taken off]
[Edit2: actually I think maybe it doesn't matter which way around you look at it, it works out the same]
Janzert
|
| « Last Edit: Apr 22nd, 2008, 11:38pm by Janzert » |
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender:
Posts: 1413
|
|
Re: Handicap Order - what beats what?
« Reply #66 on: Apr 23rd, 2008, 12:44am » |
Quote Modify
|
on Apr 22nd, 2008, 11:32pm, Janzert wrote:| I find it hard to believe botbasher's will be satisfied unable to distinguish between ER on the board and MHHDDCCRRRRRRRR on the board for a handicap... Since what it does rank I think it probably ranks correctly I just think it leaves too much unranked. |
|
You're right that one of those is clearly harder than the other. But that just means the easier one will be attacked first. Sooner or later, those two records will be broken up into (on the board):
- ER, and
- MDRR, and
- MHR, and
- HHCR, and
- HRRRR, and
- DRRRRR, and
- CCRRRRR, and
- RRRRRRRR, etc
The big question for me is how many unranked records will there be per bot in the long term, and is this unacceptably high?
|
| « Last Edit: Apr 23rd, 2008, 12:47am by 99of9 » |
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: Handicap Order - what beats what?
« Reply #67 on: Apr 23rd, 2008, 2:42am » |
Quote Modify
|
on Apr 22nd, 2008, 1:05pm, Janzert wrote:| Another aspect is that almost certainly different opponents are going to have different rankings in difficulty for various sacrifices. Almost certainly even to the point of handicap 1 being impossible and 2 being possible against opponent A but the reverse against opponent B. |
|
I'm sure that this is possible in principle, but I have not seen it yet.
Quote:| Of course it is beyond any current, or apparent near future, resources to construct and/or look at the whole game tree for even a single handicap. I wonder though if some sort of random sampling could produce useful results. |
|
No, Fritz is right that random will give us misguided results. It will favor rabbits over anything.
Quote:| Anyone have further thoughts on this or other ideas? |
|
I only see three viable choices:
- Use well defined materials evals in some combination.
- Resort to partial ordering only.
- Assume opponent bias is not a big factor and construct an order of handicaps based on what handicaps are possible in other bots.
The last one needs clarification. Say we are comparing handicap X with handicap Y. If handicap X (or self-evidently better) has been achieved against more bots than handicap Y (or self-evidently better) has been, then handicap Y is the better handicap (for the moment). In the case of a tie, both records remain. This list would be slightly dynamic, and would require a fair bit of overhead in terms of keeping it up to date. But it would (almost by definition) be a good way of ordering the handicaps fairly.
|
| « Last Edit: Apr 23rd, 2008, 2:45am by 99of9 » |
IP Logged |
|
|
|
mistre
Forum Guru
Gender:
Posts: 553
|
|
Re: Handicap Order - what beats what?
« Reply #68 on: Apr 23rd, 2008, 8:43am » |
Quote Modify
|
After listening to the discussion for a while, I thought I would pop back in with my 2 cents.
Since no one has come up with anything better yet, I think we should continue to use the list we have of the 3 combined material evaluators. Leaving many combinations unranked would just lead to a complete mess.
I really don't foresee too much of an issue, like I said, 99% of the cases for handicap game purposes shouldn't be a problem. Someone trying for a handicap should almost always pick something obvious (for example the handicap is EMHD, so I will try EMHH instead of something like EMHCRR.
As for general ordering for the sake of knowing which handicaps are better, we should continue to try to develop a method that is better than what we have now.
What if we have a test bot play against itself with different handicaps? Omar would have to set up a way to have piece exclusion in the set-up, but after that, it should be possible right?
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: Handicap Order - what beats what?
« Reply #69 on: Apr 23rd, 2008, 8:56am » |
Quote Modify
|
on Apr 23rd, 2008, 8:43am, mistre wrote:| What if we have a test bot play against itself with different handicaps? |
|
This won't help much with extreme handicaps, because none of the bots know how to attempt an extreme handicap, and some of the bots don't even know how to stop one!
For the smaller handicaps, each bot might give you a different result. E.g. bomb may do better against itself when handicapped with HC, but clueless may do better when handicapped with DD. What would that tell us?
|
|
IP Logged |
|
|
|
RonWeasley
Forum Guru
Harry's friend (Arimaa player #441)
Gender:
Posts: 882
|
|
Re: Handicap Order - what beats what?
« Reply #70 on: Apr 23rd, 2008, 11:27am » |
Quote Modify
|
I don't know if this has been said, but we might order these simply by the bot-bashing record result. Is the metric still number of moves?
The order may not be the same for each bot, but the trend among bots should provide an indication. Anyone who thinks a handicap is rated too hard can prove otherwise by bashing it better, or challenging the bashing specialists.
Such a ranking would not necessarily tell us what a certain handicap means 1) against a good muggle player or 2) in a position where there's been equal attrition.
|
|
IP Logged |
|
|
|
Arimabuff
Forum Guru
Arimaa player #2764
Gender:
Posts: 589
|
|
Re: Handicap Order - what beats what?
« Reply #71 on: Apr 23rd, 2008, 1:27pm » |
Quote Modify
|
BTW, Aamira2006P1 has just entered the CRN family, that is the bots that can be beat by a C and one or several Rs. That family now contains Gnobot2005P1 that is a CR1, ShallowBlue a CR2, ArimaaScoreP1 a CR3, Arimaalon a CR4 and our latest addition Aamira2006P1 a CR5. 
|
|
IP Logged |
|
|
|
Arimabuff
Forum Guru
Arimaa player #2764
Gender:
Posts: 589
|
|
Re: Handicap Order - what beats what?
« Reply #72 on: Apr 26th, 2008, 6:37am » |
Quote Modify
|
A little off topic: I beat bot_ArimaaScoreP2 with both colors with only a dog and eight rabbits. I suppose that cat and 8 rabbits is within the realm of possibilities but it seems so unlikely (instead of four pieces that can kill you, you have six and the enemy's cats become brick walls to you) that I don't expect my record to be beat within this century (7 rabbits is out of the question with only a single dog to defend the fort!).
But of course you are welcome to try.
|
|
IP Logged |
|
|
|
Arimabuff
Forum Guru
Arimaa player #2764
Gender:
Posts: 589
|
|
Re: Handicap Order - what beats what?
« Reply #73 on: Apr 26th, 2008, 9:06am » |
Quote Modify
|
on Apr 26th, 2008, 6:37am, Arimabuff wrote:A little off topic: I beat bot_ArimaaScoreP2 with both colors with only a dog and eight rabbits. I suppose that cat and 8 rabbits is within the realm of possibilities but it seems so unlikely (instead of four pieces that can kill you, you have six and the enemy's cats become brick walls to you) that I don't expect my record to be beat within this century (7 rabbits is out of the question with only a single dog to defend the fort!).
But of course you are welcome to try. |
|
Well, either I am a hundred years older or I was completely wrong on that one. I just cracked bot_ArimaaScoreP2 with a cat and eight rabbits with silver. It seemed even easier than with a dog. Now for gold, I am adamant cat and eight rabbits is completely utopist!
|
|
IP Logged |
|
|
|
aaaa
Forum Guru
Arimaa player #958
Posts: 768
|
|
Re: Handicap Order - what beats what?
« Reply #74 on: Apr 27th, 2008, 5:57pm » |
Quote Modify
|
Here is a proposal: If there are handicap games such that one does not unambiguously dominate the other (i.e. it is not the case that one can get from one handicap to the other by any combination of adding pieces and promoting a piece which is stronger than a rabbit), then the record which stands is the one whose handicap against that particular bot has the slowest fastest result. In case of a tie, the earliest one stands. So if you have multiple players achieving for the first time certain incomparable handicaps against a particular bot, then in order to get or keep the record it would be in their interest to try to improve on the speed of the handicaps picked by the other contenders. That way you have an objective measurement of handicap achievement and as a bonus a nice way of keeping the contest going.
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print