Author |
Topic: 2006 WC Prediction Contest (Read 5358 times) |
|
99of9
Forum Guru
Gnobby's creator (player #314)
 
Gender: 
Posts: 1413
|
 |
Re: 2006 WC Prediction Contest
« Reply #30 on: Nov 24th, 2005, 10:49pm » |
 Quote  Modify
|
on Nov 24th, 2005, 10:18pm, Ryan_Cable wrote:| I was a little surprised by how high the crowd's upset numbers turned out to be; maybe there are not as many overly aggressive predictors heading for a comeuppance in round 2 as I thought. |
|
From game 1 of round 2 it seems this is not the case anymore!
|
|
 IP Logged |
|
|
|
Ryan_Cable
Forum Guru
Arimaa player #951
Gender:
Posts: 138
|
|
Re: 2006 WC Prediction Contest
« Reply #31 on: Nov 25th, 2005, 1:32am » |
Quote Modify
|
on Nov 24th, 2005, 10:49pm, 99of9 wrote:
From game 1 of round 2 it seems this is not the case anymore! |
|
Amazing! Comeuppance galore!
14 for PMertens in 49; 80% sure [!!!]
7 for Adanac in 46; 66% sure
64.7% for PMertens. At the time of the game, Adanac actually had a 26 point rating advantage, predicting a 0.53735 chance of winning! Also, Adanac had a 4-5 record against PMertens and went 1-1 against him just a week ago.
PMertens had told me he thought he would win if he didn’t blunder, and I predicted 65% for him, thinking I was being rather bold. With these crowd numbers, I was a fool to not predict 55% for Adanac and grab all of the free variance. Oh well, it just supports the point I made about previous experience. 
|
|
IP Logged |
|
|
|
PMertens
Forum Guru
Arimaa player #692
Gender:
Posts: 437
|
|
Re: 2006 WC Prediction Contest
« Reply #32 on: Nov 25th, 2005, 3:53am » |
Quote Modify
|
on Nov 25th, 2005, 1:32am, Ryan_Cable wrote:
PMertens had told me he thought he would win if he didn’t blunder ... |
|
well ... unfortunately I got a bad habit there ...
Quote:
... and I predicted 65% for him ... |
|
since I did bet 100% on me I do not really feel guilty 
Quote:
... I was a fool to not predict 55% for Adanac ... |
|
looks like we got plenty of fools in our community 
|
| « Last Edit: Nov 25th, 2005, 3:56am by PMertens » |
IP Logged |
|
|
|
Ryan_Cable
Forum Guru
Arimaa player #951
Gender:
Posts: 138
|
|
Re: 2006 WC Prediction Contest
« Reply #33 on: Nov 25th, 2005, 4:41am » |
Quote Modify
|
on Nov 25th, 2005, 3:53am, PMertens wrote:| since I did bet 100% on me I do not really feel guilty |
|
I don’t blame you. I would have predicted 55%+ for you anyway. I just wish your game had been scheduled after a few other games had shown me just how incredibly aggressive people were being. Then I would have bet against you for strategic reasons.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2006 WC Prediction Contest
« Reply #34 on: Nov 25th, 2005, 12:20pm » |
Quote Modify
|
on Nov 25th, 2005, 1:32am, Ryan_Cable wrote:14 for PMertens in 49; 80% sure [!!!]
7 for Adanac in 46; 66% sure
[...]
With these crowd numbers, I was a fool to not predict 55% for Adanac and grab all of the free variance |
|
Grabbing the free variance was my strategy exactly when I predicted 55% for Adanac. I considered the game an absolute coin flip, but guessed (correctly) that more people would bet on PMertens. (Well, I admit that hearing about PMertens' playing conditions influenced me a little too.)
I also considered the game Paul vs. MrBrain to be a near coin flip. I had the advantage of having played MrBrain in the first round, and having sweated bullets to win despite everyone predicting 100% on me. Had the seeding been otherwise I might have bet 55% on Paul, but with MrBrain vastly underrated I opted for all the free variance that a 55% wager on him would provide.
Now that I've called two upsets, I am probably close to the lead, so it behooves me to get more conservative. As long as my position in the prediction contest remains realtively good, I'm going to stay as near to true percentages as I possibly can, hopefully thereby forcing folks who are trailing to get crazy with their predictions.
By the way, Ryan, I somehow missed your excellent post in this thread from Monday. I like your analysis of the penalty for players in the game, particularly the average penalty for underdog losers. That makes it seem appropriate to have a moderate compensation to big underdogs who lose, based on how heavily the betting went against them. Of course betting favorites who lose would still get nothing, and small underdogs who lose would get only a little.
Here's another formula to go with my previous proposal: Have the point value compensation be simply (average bet on favorite) - (average bet on underdog), where each predictor is thought to be betting partly on each player. So for example in round one these bets
19 for 99of9 in 33; 99% sure
2 for acheron in 50; 78% sure
make 99of9 the favorite by (19*99 + 2*22)/21 = 92 percent average prediction versus 8 percent average prediction on Acheron. Since Acheron lost as the underdog, the compensation would be 92 - 8 = 84 points. Those 84 points are still less than Acheron is going to lose on average by being constrained in the predicting.
This creates an incentive for an underdog to lose, but recall that all incentives are relative. The bonus for losing is nowhere near the bonus for pulling off the upset, namely gaining ~350 points on most of the field. Even in closer games, the bonus only kicks in when the betting underdog loses, and I'm guessing that the betting underdog will alway gain more relative points by winning than by losing.
Well, we'd have to look at it more closely, but it seems a reasonable way to level the playing field.
|
| « Last Edit: Nov 25th, 2005, 6:01pm by Fritzlein » |
IP Logged |
|
|
|
Adanac
Forum Guru
Arimaa player #892
Gender:
Posts: 635
|
|
Re: 2006 WC Prediction Contest
« Reply #35 on: Nov 25th, 2005, 3:42pm » |
Quote Modify
|
This is my first ever prediction contest, and I really like the way it adds an extra level of excitement to an already exciting tournament. Are there any other prediction contests? i.e. Human vs. Bot Challenge Match or Bot Championships?
on Nov 25th, 2005, 12:20pm, Fritzlein wrote:
Here's another formula to go with my previous proposal: Have the point value compensation be simply (average bet on favorite) - (average bet on underdog), where each predictor is thought to be betting partly on each player. So for example in round one these bets
19 for 99of9 in 33; 99% sure
2 for acheron in 50; 78% sure
make 99of9 the favorite by (19*99 + 2*22)/21 = 92 percent average prediction versus 8 percent average prediction on Acheron. Since acheron lost as the underdog, the compensation would be 92 - 8 = 84 points. Those 84 points are still less than Acheron is going to lose on average by being constrained in the predicting.
This creates an incentive for an underdog to lose, but recall that all incentives are relative.
|
|
Since players don’t have a lot of flexibility in predicting their own games anyway, and they appear to be at a disadvantage compared to the spectators, and weaker players are at a further disadvantage, why not just award both players a number of points equal to the average [mean] score from all the other participants. So far example, in the Paul vs. Mr. Brain game both players would receive a score (I’ll take a wild guess here) of roughly -110 while in the Fritzlein vs. Mr. Brain game both players would receive a score of roughly +185. It’s not perfect, but it achieves a few objectives:
(1) Reduces/Removes incentive for a heavily favoured player to bet 55% and throw a match to *relatively* gain points on the rest of the field. (I can't imagine anyone would actually do this anyway)
(2) Removes underdog disadvantage
(3) Removes spectator advantage
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2006 WC Prediction Contest
« Reply #36 on: Nov 25th, 2005, 10:36pm » |
Quote Modify
|
on Nov 25th, 2005, 3:42pm, Adanac wrote:| This is my first ever prediction contest, and I really like the way it adds an extra level of excitement to an already exciting tournament. Are there any other prediction contests? |
|
I think the WC prediction contest is the only one, but I agree it is great fun.
Quote:| [...] why not just award both players a number of points equal to the average [mean] score from all the other participants. |
|
I didn't think of that, but now that you say it, it seems obvious and best. On a game you are involved in, you don't get to bet, and you get the mean of what everyone else got. Even if you are in, say, seven of the thirty games of the WC, you still get to bet on the other 23, so you have enough chance to have fun there. Great idea.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2006 WC Prediction Contest
« Reply #37 on: Nov 26th, 2005, 4:52am » |
Quote Modify
|
on Nov 25th, 2005, 12:20pm, Fritzlein wrote:| Now that I've called two upsets, I am probably close to the lead, [...] |
|
Easy come, easy go. JDB's upset of Naveed surely gives JDB a commanding lead at this point.
|
|
IP Logged |
|
|
|
Adanac
Forum Guru
Arimaa player #892
Gender:
Posts: 635
|
|
Re: 2006 WC Prediction Contest
« Reply #38 on: Nov 26th, 2005, 6:53am » |
Quote Modify
|
on Nov 25th, 2005, 10:36pm, Fritzlein wrote:
I didn't think of that, but now that you say it, it seems obvious and best. On a game you are involved in, you don't get to bet, and you get the mean of what everyone else got. Even if you are in, say, seven of the thirty games of the WC, you still get to bet on the other 23, so you have enough chance to have fun there. Great idea. |
|
I just thought of a problem with my own suggestion. Omar wanted the incentive for a player to fight on as long as possible, even if the game is certainly lost (which is a great feature of the existing system). We would have to reward the players for number of moves but then my system no longer works because it gives an advantage to the higher ranked player over both the underdog and the spectators... 
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: 2006 WC Prediction Contest
« Reply #39 on: Nov 27th, 2005, 4:25pm » |
Quote Modify
|
on Nov 26th, 2005, 4:52am, Fritzlein wrote:
Easy come, easy go. JDB's upset of Naveed surely gives JDB a commanding lead at this point. |
|
So it seems - but he must've been picking the other upsets too, since his game was the only one I missed.
|
|
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2006 WC Prediction Contest
« Reply #40 on: Nov 29th, 2005, 5:29pm » |
Quote Modify
|
What a turnaround for the field. The same twenty-one predictors who averaged a gain of 985 points over the first seven games, averaged a gain of only 122 points over the next eight games. One more upset, and we might have been negative as a group for round two!
|
|
IP Logged |
|
|
|
Ryan_Cable
Forum Guru
Arimaa player #951
Gender:
Posts: 138
|
|
Re: 2006 WC Prediction Contest
« Reply #41 on: Dec 1st, 2005, 7:30pm » |
Quote Modify
|
The round 2 scores:
Name . . Points StDev
1 jdb . . 923 1.747
2 Fritzlein . 765 1.379
3 99of9 . . 731 1.299
4 Paul . . 664 1.143
5 Elmo . . 533 0.838
6 Adanac . . 498 0.756
7 MrBrain . . 492 0.742
8 nbarriga . . 417 0.567
9 RonWeasley . 195 0.050
10 naveed . . 154 -0.046
11 grey_0x2A . 82 -0.213
12 BlackKnight . 77 -0.225
13 mohabbatse . 53 -0.281
14 Ryan_Cable . 32 -0.330
15 Belbo . . 19 -0.360
16 robinson . -34 -0.484
17 acheron . . -158 -0.773
18 PMertens . -297 -1.097
19 omar . . -426 -1.398
20 fritzlforpresident -523 -1.624
21 Aamir . . -552 -1.691
22 carolaina . -1029 -2.803
Median 82.0
Mean 173.6
StDev 429.0
The total scores after round 2, with ~48% of the non-forfeit games finished:
Name . . Points StDev
1 jdb . . 2088 1.733
2 99of9 . . 1867 1.339
3 Fritzlein . 1754 1.137
4 Paul . . 1698 1.038
5 Elmo . . 1661 0.972
6 Adanac . . 1599 0.861
7 MrBrain . . 1381 0.473
8 RonWeasley . 1371 0.455
9 naveed . . 1240 0.222
10 Ryan_Cable . 1233 0.209
11 nbarriga . . 1133 0.031
12 BlackKnight . 1115 -0.001
13 Belbo . . 1091 -0.044
14 grey_0x2A . 808 -0.548
15 acheron . . 773 -0.610
16 robinson . . 679 -0.778
17 omar . . 520 -1.061
18 PMertens . . 517 -1.066
19 fritzlforpresident 516 -1.068
20 Aamir . . 476 -1.139
21 mohabbatse . -94 -2.155
22 carolaina . -279 -2.484
Median 1133.0
Mean 1115.5
StDev 561.3
The following tables list round 1 scores, round 2 scores, total scores. (Note that mohabbatse was included in the round 2 numbers only.)
Round 1 winners:
Median 1030.5 25.5 1162.0
Mean 996.5 161.0 1157.5
StDev 166.6 452.4 549.0
Round 1 losers:
Median 1034.0 154.0 1240.0
Mean 981.3 319.1 1300.4
StDev 145.4 383.7 473.2
Spectators:
Median 1033.5 53.0 824.5
Mean 972.8 -129.4 813.0
StDev 194.1 580.0 710.9
People with 2005 experience:
Median 1033.0 115.5 1177.5
Mean 1003.4 115.8 1119.3
StDev 131.2 603.0 698.6
People without 2005 experience:
Median 1034.0 67.5 1133.0
Mean 959.7 122.6 1090.0
StDev 197.6 343.3 419.6
People with 2004 experience:
Median 1072.0 154.0 1240.0
Mean 1025.8 194.0 1219.8
StDev 103.7 445.5 487.7
People without 2004 experience:
Median 1031.0 77.0 1115.0
Mean 971.8 96.8 961.0
StDev 174.6 514.2 631.3
Undefeated players:
Median 1045.0 614.5 1676.5
Mean 984.8 490.0 1474.8
StDev 191.7 368.9 541.8
Players with one loss:
Median 1038.0 32.0 1115.0
Mean 1017.9 117.1 1135.0
StDev 142.5 461.0 538.6
Eliminated players:
Median 982.5 118.0 1024.0
Mean 944.3 185.5 1129.8
StDev 159.1 345.8 434.3
The spectators are having the worst performance as I expected, but the round 1 losers now have strong evidence that they are the most skilled group of predictors. Experience seems to have been somewhat less valuable in round 2. The undefeateds are doing quite well but there are only 4 of them, and I think it is mostly due to coincidence. Two of the eliminated players were round 2 favorites, so that might account for a small portion of their good scores, but I think it mostly comes from them being a subset of the round 1 losers.
The round 2 scores have a 0.442 correlation with the round 1 scores. The round 1 scored have a 0.623 correlation with the total scores. The round 2 scores have a 1.178 correlation with the total scores. These correlations are obviously inflated due to the fact that the scores are significantly more spread out than a normal distribution, but I think that they should still be fairly accurate relative to each other.
The crowd predicted 2.144 upsets. Weighting down the upset predictions to remove the forced predictions, the crowd predicted 1.990 upsets. My ratings estimates predicted 2.03035 upsets, which is very close. However, the round 2 mean is much less than what could have been had by a score expectation maximization strategy.
The people I predicted to win after round 1 are all still in the running, but after round 2, I think the most likely winners are:
1 99of9
2 Fritzlein
3 jdb
|
|
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: 2006 WC Prediction Contest
« Reply #42 on: Dec 1st, 2005, 7:49pm » |
Quote Modify
|
on Dec 1st, 2005, 7:30pm, Ryan_Cable wrote:The round 2 scores:
The total scores after round 2, with ~48% of the non-forfeit games finished: |
|
Wow, we still have half the prediction comp to go! Maybe I'll have a chance to make up for my 85% Omar blunder .
|
| « Last Edit: Dec 1st, 2005, 7:49pm by 99of9 » |
IP Logged |
|
|
|
Fritzlein
Forum Guru
Arimaa player #706
Gender:
Posts: 5928
|
|
Re: 2006 WC Prediction Contest
« Reply #43 on: Dec 1st, 2005, 9:18pm » |
Quote Modify
|
Ryan, thanks again for this interesting statistical analysis. The results of this year will definitely be influential in next year's betting. However, I still believe that the final ordering is going to come down to who is lucky in guessing the coin-flip games. There's no mathematical analysis that's going to work as well as simply guessing right on tossup games. Come to think of it, we're already seeing it: 99of9 and I are near the top mostly because of guessing right on the two second-round tossups.
on Dec 1st, 2005, 7:49pm, 99of9 wrote:| Maybe I'll have a chance to make up for my 85% Omar blunder . |
|
99of9, I was only 65% for Omar, so that puts us in a virtual tie for second place, unless Paul or Adanac bet on BlackKnight and leapfrogged us.
|
| « Last Edit: Dec 1st, 2005, 9:23pm by Fritzlein » |
IP Logged |
|
|
|
99of9
Forum Guru
Gnobby's creator (player #314)
Gender:
Posts: 1413
|
|
Re: 2006 WC Prediction Contest
« Reply #44 on: Dec 1st, 2005, 9:47pm » |
Quote Modify
|
on Dec 1st, 2005, 9:18pm, Fritzlein wrote:| Come to think of it, we're already seeing it: 99of9 and I are near the top mostly because of guessing right on the two second-round tossups. |
|
By my reckoning they were not tossups nor upsets. I know this is counter to the majority opinion, but my predictions were for who I thought had better chances (based on their previous games and knowledge of their playing styles etc), and had nothing to do with exploiting variance. In fact I bet over 55% on one of them. I think the fact that the majority were betting ~80% for the loser indicates that we were either extremely lucky, or we had some better knowledge than the majority. Even thinking that it was a tossup shows your knowledge was superior. Perhaps my *favouritism* of the eventual winners was overconfidence in the lucky direction, but I'd say it was much closer to the truth than 80% on the losers!
However, you're right that the next half of the comp will have lots more coin flips in it. So perhaps that will determine the eventual winner.
|
|
IP Logged |
|
|
|
|
Home
Help
Search
Members
Login
Register
Reply
Notify of replies
Send Topic
Print