Author |
Topic: Cat-for-rabbit handicap experiment (Read 2303 times) |
|
Fritzlein
Forum Guru
Arimaa player #706
Gender: 
Posts: 5928
|
 |
Cat-for-rabbit handicap experiment
« on: Apr 29th, 2006, 1:18am » |
 Quote  Modify
|
In another thread, I tried my best to explain away the stronger correlation between winning and initial rabbit capture than exists between winning and initial cat capture. Every time I persuaded myself, IdahoEv's other statistic came back to haunt me. The number I can't explain away is that, after an initial cat-for-rabbit trade, the side with the extra rabbit wins 55% while the side with the extra cat wins only 45%. I just couldn't get to sleep tonight with that statistic buzzing around in my head, so here I am at 1:00 in the morning typing this post.
It seems that one could make a controlled experiment whereby the cat-for-rabbit exchange happens before the game. That would prevent the exchange from being caused by any type of positional advantage. To put it another way, beginning the game with a material difference would prove than any difference in winning percentages was caused by the material difference and nothing else. The only trouble is that a minimum of 1000 games are necessary to statistically distinguish 55% winning chances from 50%, and 4000 games would be nice.
http://arimaa.com/arimaa/forum/cgi/YaBB.cgi?board=talk;action=display;nu m=1065901453;start=0
Then I remembered from the rating deflation thread that clauchau coded some dumb bots for rapid play against each other offline, including Random Stepper, Random Mover, Infiltrator, Flooder, etc. up to ScoreP1. These are bots that could each self-play 1000 games in a reasonable time frame. I don't imagine it would be too hard to alter them to include a built-in cat-for-rabbit handicap, alternating with a rabbit-for-cat handicap.
Admittedly, any results from this experiment could be dismissed as inconclusive because these bots are so dumb, but I promise that if they all show a 55% advantage to the side with the extra rabbit, I will start backing away from my claim that an initial cat is worth more. I will be especially impressed if the percentage is nearly constant across all the dumb bots, rather than, for example, showing a preference for the rabbit among aggressive evaluators, but a preference for the cat in the random bots and S+K-K.
Is there anyone here I could beg, wheedle, or cajole into running this experiment? It would literally help me sleep better at night.
|
|
 IP Logged |
|
|
|
IdahoEv
Forum Guru
Arimaa player #1753
Gender:
Posts: 405
|
|
Re: Cat-for-rabbit handicap experiment
« Reply #1 on: Apr 29th, 2006, 3:31am » |
Quote Modify
|
OMG OMG what have I done
I have wrecked the sleep of our finest player!
Will Fritzlein's rating begin to droop from the stress?
|
|
IP Logged |
|
|
|
IdahoEv
Forum Guru
Arimaa player #1753
Gender:
Posts: 405
|
|
Re: Cat-for-rabbit handicap experiment
« Reply #2 on: Apr 29th, 2006, 3:32am » |
Quote Modify
|
I do think a good part of the statistic can probably be accounted for by the profligate use of bait-and-tackle vs. bomb. I will re-run the analysis with only HVH games tomorrow to see what difference it makes.
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender:
Posts: 1413
|
|
Re: Cat-for-rabbit handicap experiment
« Reply #3 on: Apr 29th, 2006, 3:33am » |
Quote Modify
|
[Edit IdahoEv posted while I was writing  ]
I claim this particular statistic is an anomaly due to people baiting bot_bomb. They offer it a cat for an elephant blockade, then they win the first rabbit, and go on to win.
Maybe IdahoEv could run that same comparison excluding all bot games. Are there sufficient H-H stats?
|
| « Last Edit: Apr 29th, 2006, 3:34am by 99of9 » |
IP Logged |
|
|
|
jdb
Forum Guru
Arimaa player #214
Gender:
Posts: 682
|
|
Re: Cat-for-rabbit handicap experiment
« Reply #4 on: Apr 29th, 2006, 8:29am » |
Quote Modify
|
The offline match player, available in the downloads section, would work for this experiment. It is possible to specify the initial position.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: Cat-for-rabbit handicap experiment
« Reply #5 on: Apr 29th, 2006, 10:09am » |
Quote Modify
|
on Apr 29th, 2006, 3:33am, 99of9 wrote:| Maybe IdahoEv could run that same comparison excluding all bot games. Are there sufficient H-H stats? |
|
I tried to run this myself. I got 942 rated games (no move 2 suicide) with an initial cat for rabbit trade. The side with the extra rabbit won 488/942 = 52%. This may disagree with IdahoEv's 55% number because I didn't insist that the material configuration had to last any specified amount of time.
The HvH and BvB games don't exist in sufficient quantity to be statistically significant. However, the side with the extra rabbit was favored in each case. For HvH games it was 28/49 = 57% and for the BvB games it was 76/144 = 53%. This is a statistic to prolong my sleepless nights.
On the other hand, there was some good news for my insomnia: I split the HvB games into those in which the human got the rabbit and those in which the bot got the rabbit. If the human won the rabbit, then the extra rabbit won 309/557 = 55%, but if the bot won the rabbit, then the extra rabbit won 75/192 = 39%. That is to say, humans are winning against bots no matter which side of the cat-for-rabbit exchange they end up on, and the reason the rabbit comes out looking better overall is that humans were much more often taking than giving the rabbit against bots.
In other words, the bomb-baiting theory seems to be a pretty good explanation of how the cat could be worth more and yet be on the losing side of the trade overall.
Whew. I think I'll go back to bed.
|
|
IP Logged |
|
|
|
chessandgo
Forum Guru
Arimaa player #1889
Gender:
Posts: 1244
|
|
Re: Cat-for-rabbit handicap experiment
« Reply #6 on: Apr 29th, 2006, 10:57am » |
Quote Modify
|
Anyway this is an interesting problem, maybe we could play some games ourselves (ok Fritz, unrated, not to biaisedly put your rating in danger ) where the first move of one side is to kamikaze a cat and the other a rabbit ... after which the game begins.
Not that it would solve the problem, but maybe it would give us a hint of what's going on with an initial rabbit value. I would be as surprised as Fritz if a cat were not better than a rabbit, but ... who knows ?
If anyone wants to make the test, feel free to challenge me !
|
|
IP Logged |
|
|
|
chessandgo
Forum Guru
Arimaa player #1889
Gender:
Posts: 1244
|
|
Re: Cat-for-rabbit handicap experiment
« Reply #7 on: Apr 29th, 2006, 10:58am » |
Quote Modify
|
(well, feel free to challenge me anyway, but not in dry 15 mn per game please kami )
|
|
IP Logged |
|
|
|
IdahoEv
Forum Guru
Arimaa player #1753
Gender:
Posts: 405
|
|
Re: Cat-for-rabbit handicap experiment
« Reply #8 on: Apr 29th, 2006, 3:27pm » |
Quote Modify
|
Well, I reran my analysis using only HvH, and as suspected the numbers drop into statistical insignificance. We need to play more games people!
But, here's what I've found.
If I still only consider games where the losing player had a rating > 1650, then:
| R capture | 112228-112227 | 63.6% | (171/269) | | C capture | 112228-112218 | 65.1% | (66/101) | | RC trade | 112227-112218 | 41.7% | (8/20) |
Considering all rated HvH games regardless of player ratings:
| R capture | 112228-112227 | 66.4% | (197/400) | | C capture | 112228-112218 | 72.9% | (140/190) | | RC trade | 112227-112218 | 42.7% | (13/31) |
The third line on both of those is so small as to be essentially useless statistically. The others are not particularly significant, but they do both show a cat more valuable than an initial rabbit.
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print