Author |
Topic: First Move Advantage vs. Second Setup Advantage (Read 16564 times) |
|
Fritzlein
Forum Guru
Arimaa player #706
Gender: 
Posts: 5928
|
 |
First Move Advantage vs. Second Setup Advantage
« on: Nov 15th, 2006, 10:07pm » |
 Quote  Modify
|
A year ago, in another thread, I tried to calculate the advantage of moving first. I was (and am!) quite convinced that Gold has an advantage from moving first. However, my naive method of measuring Gold's advantage showed that it was negligible. I therefore did a much more careful study, using only pairs of games where the same two opponents matched up twice with reversed colors. The results are as below:
on Dec 4th, 2005, 1:41pm, Fritzlein wrote:
Game Type Pairs Gold Wins Mismatch Gold Adv. # Std. Dev.
--------- ----- --------- -------- --------- -----------
ALL . 4692 4725 . 189 . 3.23 . 0.79
H v B . 3839 3851 . 192 . 1.45 . 0.32
B v B . 608 630 . 152 . 15.1 . 1.38
H v H . 245 244 . 237 . -2.19 . 0.12
|
|
At that time, the only explanation I could think of was that there are too few games to make a judgment, especially in HvH games, so the slight bias to Silver must be a statistical fluke, which will work out in the long run as a bias to Gold.
When I was chatting with Omar today, however, a different explanation suddenly occurred to me. Gold has the advantage of the first move, but Silver has the advantage of setting up second. I have always thought of the setup advantage as being negligible compared to the first move advantage, but what if I am wrong? What if setting up second is in fact more important?
I think my bias comes from the fact that I use the 99of9 setup no matter what. All the asymmetrical setups recently, however, suggest that it would be very nice to be able to respond to the other guy's setup, either to defend where he is attacking, or to attack where he isn't defending.
If the second setup is actually a greater advantage than the first move, then you would expect that to show up only in human games, because none of the bots adjust their setup based on what the other player has done (except to keep elephants in different files). By the same logic, the bot vs. bot games would measure purely the first move advantage.
I therefore quickly resurrected my naive query to determine Gold advantage. The way it works is that I calculate over all the rated games how many games Gold was expected to win, and how many games Gold did win. If Gold won more games than expected, I would add a few points to the ratings of all Gold players and calculate again, until their expected number of wins came up to their actual number of wins. It turns out the magic number is 3.5 rating points. If I add 3.5 rating points to every Gold player's rating in the database, then Gold's expected wins are equal to the actual wins.
Look, however, what happens when I break that down according to game type:
Game Type Games Gold Advantage
--------- ----- --------------
ALL . 32680 3.5
B v B . 4566 10.0
B v H . 12830 8.4
H v B . 13070 0.8
H v H . 2214 -25.7
1600+ HvH 1391 -26.5
For HvH games, the Silver advantage is huge! The effect is so pronounced, I am going to have to revisit my more careful methodology, and evaluate games only in pairs.
Before I jump to the conclusion that the setup advantage outweighs the first-move advantage, I should note that the BvH and HvB data somewhat contradict my theory. If the bots have a setup that is not reactive, then humans playing Gold should have move advantage plus setup advantage, while bots playing Gold should have move advantage minus setup advantage. Yet for some reason, there appears to be a greater advantage for bots playing Gold against humans than for humans playing Gold against bots.
So maybe I will eventually land back at my idea that humans doing better with Silver than with Gold is just a fluke. More data is needed, especially given the trend towards asymmetrical setups and away from the 99of9 setup, before we can determine whether move advantage or setup advantage is greater.
|
|
 IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #1 on: Nov 16th, 2006, 1:02am » |
Quote Modify
|
Luckily, I had the old code lying around with which I did the paired-games experiment before. I repeated it now on a larger set of data. The results are as follows:
Game Type Pairs Gold Wins Mismatch Gold Adv. # Std. Dev.
--------- ----- --------- -------- --------- -----------
ALL . 11222 11231 . 212 . 0.4 . 0.14
B v B . 1676 1684 . 190 . 2.2 . 0.32
H v B . 8972 9001.5 . 213 . 1.6 . 0.53
H v H . 574 545.5 . 258 . -30.3 . 2.2
Here I did not calculate the Gold advantage by tweaking the ratings of players to match expected results to actual results. Instead I calculated the Gold advantage from the number of extra rating points needed, at the size of the average mismatch, to tip the winning percentage as much as it was tipped. Thus if the players had inaccurate ratings it won't throw off the result by much; the ratings were only used to calculate roughly how mismatched the players are.
I also used the number of game pairs to calculate how statistically significant the results are. Generally one says that a result is statically significant if there is less than a 5% probability that it could happen by chance. Since the HvH result is 2.2 standard deviations from the mean, it has less than a 5% chance of being a fluke. In all probability, Silver has an advantage over Gold in HvH games, and it seems to be worth about 30 rating points. In HvB games, or BvB games, or in all games combined, the results are not statistically significant, and we might as well say the sides are even.
I don't believe that the second setup advantage is greater than the first move advantage, certainly not by 30 rating points, but that's what the data seems to be saying.
|
| « Last Edit: Nov 16th, 2006, 8:40am by Fritzlein » |
IP Logged |
|
|
|
RonWeasley
Forum Guru
Harry's friend (Arimaa player #441)
Gender:
Posts: 882
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #2 on: Nov 16th, 2006, 7:27am » |
Quote Modify
|
When muggle players invite each other, does the inviter usually offer to play silver as a courtesy? This is what I see, but I haven't played enough games to really know. If this is true, do inviters tend to be winners? Avid players gain more experience and tend to improve. If an improving player tends to be silver, he would play better than his rating, possibly accounting for some of the advantage you are measuring.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #3 on: Nov 16th, 2006, 8:01am » |
Quote Modify
|
on Nov 16th, 2006, 7:27am, RonWeasley wrote:| When muggle players invite each other, does the inviter usually offer to play silver as a courtesy? |
|
It was my suspicion as well that the stronger player is more often Silver. I personally have played as Silver in 54.4% of my games. That is why I used the second methodology, in which a single game can't count by itself. In this more careful methodology, I only count pairs of games in which the same two players compete with colors reversed in the second game. If I play someone twice with Silver and once with Gold, one of my games with Silver is disregarded, and only the other two are counted.
|
| « Last Edit: Nov 16th, 2006, 8:43am by Fritzlein » |
IP Logged |
|
|
|
Microbe
Forum Newbie
Arimaa player #1977
Gender:
Posts: 4
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #4 on: Nov 16th, 2006, 8:42am » |
Quote Modify
|
I remember once reading that you believed Gold had an advantage Fritz, and I wondered if Silver had a way of counteracting and balancing it by setting up second. But I knew I could never prove it and that you probably had data or other ideas to back you up.
However, after seing this data it's nice to see that I may have at least been a little right. Very interesting results there. Looking forward to further studies and possible results.
|
|
IP Logged |
|
|
|
PMertens
Forum Guru
Arimaa player #692
Gender:
Posts: 437
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #5 on: Nov 16th, 2006, 10:07am » |
Quote Modify
|
Thats a nice excuse to remind Fritzl that I got a 50% winning ration against him ... when I play silver ...
Quote:| It was my suspicion as well that the stronger player is more often Silver. I personally have played as Silver in 54.4% of my games |
|
hmm .. I played silver 58.2% (HvH) ... (and won 70% of silver games ... while only 53% if gold games)
Counting all my opponents with a 2-digit number of games only against Adanac, 99of9 and Omar I do have a better ration with gold than with silver.
But more questions to the actual question:
How can a purely defensive player gain advantage from first move ?
How can an agressive gold setup (EHH, phant behind trap) reduce silvers options to pure reaction ?
(While only a 08/15 gold setup actually gives silver all options to choose and reduce gold's advantage)
I personally think that the advantage is very dependant on the players/styles and not to safely calculated as a sum over everything.
|
| « Last Edit: Nov 16th, 2006, 10:19am by PMertens » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #6 on: Nov 16th, 2006, 10:24pm » |
Quote Modify
|
on Nov 16th, 2006, 10:07am, PMertens wrote:| I personally think that the advantage is very dependant on the players/styles and not to safely calculated as a sum over everything. |
|
Certainly the advantage of going first depends on the styles of the players. If both players use the 99of9 setup every game, like I have a tendency to do, it seems to me the first player must have the advantage.
But for every player with a Gold advantage of 10 rating points, there must be a player with a Gold advantage of -70 rating points to make it average out. The fact that the effect varies from player to player means that for some it is even more pronounced than it is for the average, which is even more startling to me.
Quote:| How can a purely defensive player gain advantage from first move ? |
|
There are no purely defensive players. I'm about the most defensive player there is, and I can gain an advantage from the first move by dragging a rabbit first.
Quote:| (While only a 08/15 gold setup actually gives silver all options to choose and reduce gold's advantage) |
|
What is a 08/15 gold setup?
One possible explanation of the Silver advantage is that current opening theory recommends doing something bad. For example, suppose it is bad to launch an EH attack, but everyone tries to do it because they think it is good. Since Gold has the first move, Gold is more often successful getting an EH attack going, and therefore loses more often than Silver. That could account for my results above where BvH games have an 8.4 point Gold rating advantage whereas HvB games have an 0.4 point Gold rating advantage: the humans are using their first move to do something counter-productive.
I like this explanation because it allows me to claim that Gold theoretically has the true first-move advantage and claim that rabbit-dragging is the only sound opening strategy, and claim that I am respecting the evidence rather than stubbornly clinging to old-fashioned ideas that have already been proven wrong. 
|
| « Last Edit: Nov 16th, 2006, 10:30pm by Fritzlein » |
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender:
Posts: 1413
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #7 on: Nov 16th, 2006, 10:38pm » |
Quote Modify
|
on Nov 16th, 2006, 8:01am, Fritzlein wrote:
It was my suspicion as well that the stronger player is more often Silver. I personally have played as Silver in 54.4% of my games. That is why I used the second methodology, in which a single game can't count by itself. In this more careful methodology, I only count pairs of games in which the same two players compete with colors reversed in the second game. If I play someone twice with Silver and once with Gold, one of my games with Silver is disregarded, and only the other two are counted. |
|
What if I play silver 10 times against someone while they're learning. Then we play 10:10 when they have become as strong a player as me.
(The first?) 10 silver games count, I have a very good record as silver. 10 gold games count, I have an even record as gold.
Therefore it is better to have the silver pieces than the gold?
I don't mean to cut you down on this, but removing statistical bias is a hard job!
[btw... this isn't just hypothetical, from memory I behaved very like this against some learners (Belbo springs to mind)]
|
| « Last Edit: Nov 16th, 2006, 10:40pm by 99of9 » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #8 on: Nov 16th, 2006, 11:36pm » |
Quote Modify
|
on Nov 16th, 2006, 10:38pm, 99of9 wrote:What if I play silver 10 times against someone while they're learning. Then we play 10:10 when they have become as strong a player as me.
(The first?) 10 silver games count, I have a very good record as silver. |
|
Good question, but I did anticipate this. I only count a game if the very next game between the two players has the colors reversed. So if you play Silver ten times against someone, and then Gold ten times against the same person, only the middle two games count from all twenty games. Here is a sample of which games I would count from a string of your games against a particular opponent, with only your color given:
S
S
S *
G *
S
S *
G *
G
G *
S *
G *
S *
G
where the asterisks indicate the paired games.
Quote:| I don't mean to cut you down on this, but removing statistical bias is a hard job! |
|
I would be grateful if you would find a flaw in my methodology, because I don't want to believe the result!
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #9 on: Nov 17th, 2006, 12:04am » |
Quote Modify
|
on Nov 16th, 2006, 11:36pm, Fritzlein wrote:
Good question, but I did anticipate this. I only count a game if the very next game between the two players has the colors reversed. |
|
Excellent, I'll have to think harder then.
Quote:| because I don't want to believe the result! |
|
 neither do I.
|
|
IP Logged |
|
|
|
woh
Forum Guru
Arimaa player #2128
Gender:
Posts: 254
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #10 on: Nov 17th, 2006, 9:11am » |
Quote Modify
|
You might calculate the advantage of either side by giving each player two ratings, one for when he's playing gold and one for silver. Then you calculate their current ratings applying their gold or silver rating as appropriate. If more players end up with a higher gold rating then you might have an indication that the first move advantage is greater than the second setup advantage.
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #11 on: Nov 17th, 2006, 3:18pm » |
Quote Modify
|
that's a nice alternative woh
|
|
IP Logged |
|
|
|
IdahoEv
Forum Guru
Arimaa player #1753
Gender:
Posts: 405
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #12 on: Nov 17th, 2006, 4:31pm » |
Quote Modify
|
on Nov 17th, 2006, 9:11am, woh wrote:| You might calculate the advantage of either side by giving each player two ratings, one for when he's playing gold and one for silver. Then you calculate their current ratings applying their gold or silver rating as appropriate. If more players end up with a higher gold rating then you might have an indication that the first move advantage is greater than the second setup advantage. |
|
It would be interesting to chart how separate gold vs. silver ratings change over time. It may be that the advantage has varied as Arimaa opening theory has evolved.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #13 on: Nov 17th, 2006, 11:32pm » |
Quote Modify
|
I realized that I could calculate the standard deviation more accurately than by estimating it based on the average rating difference. I simply summed the variance of each game and took the square root. Unfortunately, for HvH games, the Silver advantage still comes out to 2.12 standard deviations, which means there is only a 3.4% chance of such an extreme result if Arimaa is really fair.
I may calculate the Gold and Silver ratings of each player if the fancy strikes me, but that not only seems like a less accurate way to measure, it also seems almost certain to confirm some level of Silver advantage.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: First Move Advantage vs. Second Setup Advantag
« Reply #14 on: Nov 17th, 2006, 11:37pm » |
Quote Modify
|
on Nov 17th, 2006, 4:31pm, IdahoEv wrote:| It would be interesting to chart how separate gold vs. silver ratings change over time. It may be that the advantage has varied as Arimaa opening theory has evolved. |
|
Yes, it seems that the Silver advantage has only recently materialized, whereas before it was more even. That could roughly correspond to the emergence of unbalanced setups whereas before symmetrical (or at least balanced) setups predominated. However, the smaller the subsets we break the data into, the less statistically significant the results on each piece. I'm just happy we're getting enough HvH games to to statistics on the whole set. (even though the results seem to say everything I know about Arimaa is wrong)
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print