|
||||||||||
|
Title: 2006 WC Prediction Contest Post by 99of9 on Nov 14th, 2005, 8:26pm If anyone is looking for the page to enter your predictions, I found it here: http://arimaa.com/arimaa/wc/2006/predict/predictions.cgi 99 |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Ryan_Cable on Nov 14th, 2005, 8:35pm Thanks :) |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 15th, 2005, 12:15pm on 11/10/05 at 21:56:28, 99of9 wrote:
I think the main reason to keep it at 5% intervals is that you get integral rewards that way. Predicting 95% wins you 99 points, while predicting 100% wins you 100 points, so we'd be getting 99.XX points on the in-between percentages. Also, how much does 1% really matter at the extremes? Let's say the true odds on the winner are 98%. The average prediction payouts are: 100% prediction pays 92 98% prediction pays 92.16 95% prediction pays 91.8 We're talking fractions of a point on average. When you consider the sources of noise, it just doesn't matter down to the last digit. In all likelihood the contest will go to whoever predicts best on the tossup games. The difference between a 95% prediction and a 100% prediction is almost negligible in comparison to the difference between a 45% prediction and a 55% prediction, because in the latter case one of them gets the move prediction bonus and the other doesn't. The noise introduced by lucky guesses on coin-flip games will quite probably overwhelm any statistical nuances that one could use to eke out an extra point here and there. Frankly, that's the way I like it. The contest is more fun if gut-level intuition isn't overwhelmed by mathematics. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Nov 15th, 2005, 2:52pm You're assuming that the player always *wins* the 98% games. The difference between 95% and 98% becomes quite large whenever there is an upset victory. I think predicting actual probabilities of upsets is an interesting thing in itself rather than guessing coin flip games, especially in light of all the stuff that Ryan has been working on. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 15th, 2005, 4:59pm on 11/15/05 at 14:52:14, 99of9 wrote:
No, I'm not assuming that the favorite always wins. I'm assuming that your prediction of 98% is totally accurate (i.e. there is exactly a 2% chance of upset) and calculating how much you lose on average by being forced to pick 100% or 95% instead. If you are right about your 98% prediction, the chance of big swing due to an upset is quite small. Here's the complete calculation: 100% prediction pays 92 = 0.98*100 - 0.02*300 98% prediction pays 92.16 = 0.98*99.84 - 0.02*284.16 95% prediction pays 91.8 = 0.98*99 - 0.02*261 I agree with you that predicting probabilities of upsets is very interesting. You get the highest possible payoff on average by being as accurate as possible in your percentages. On the other hand, one might accept a small penalty in average payoff for an increased variance, on the theory that only the top three places pay out, so it is better to have a lower chance of an extremely good result than a higher chance of a merely above average result. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Nov 15th, 2005, 6:26pm Ok sorry, I didn't read your post with enough care. I still think it would help to have smaller increments (especially near the top). You are right that on *average* the difference will be negligible, but that is because in 98% of cases the expected winner wins, and the difference matters not at all. But when upsets happen, it matters a lot, especially given that the 1st-2nd placings in last year's prediction comp were decided by less than 30 points: [tt] Aamir 1042 omar 1014 mouse 942 Fritzlein 917 99of9 874 [\tt] Quote:
Interesting theory. My gut feeling is that humans gamble this way too much, and thus over many games, a steady rational performer will eventually come through ;-). (Although your theory certainly has weight if you are a few points behind and nearing the end of the WC) |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 15th, 2005, 7:38pm on 11/15/05 at 18:26:15, 99of9 wrote:
Last year Aamir won by predicting 100% on every game, except that he didn't predict once or twice and took a minimum penalty for those games. This might encourage one to subscribe to the "high variance" theory. On the other hand, last year there were a total of 11 games on which to wager. This year, barring forfeits, there will be 30 or 31. That might encourage one to play the percentages, confident that wild guessers will eventually stumble. But as you point out, it will also depend on your standing in relation to the leaders, and on how much time remains in the contest. The nearer the contest is to the end, the more the leaders should be inclined to hedge their bets, and the less the trailers have to lose by being radical. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Ryan_Cable on Nov 16th, 2005, 12:44am on 11/15/05 at 14:52:14, 99of9 wrote:
Making up an entire March Madness style prediction for a FDE tournament would be a fascinating exercise if someone comes up with a good way of scoring it. Maybe give people the opportunity to redo their predictions after each round but with a penalty for each change. Just predicting what round each player goes out in is difficult, much less predicting who they play along the way. Definitely, something to think about for next year’s prediction contest if we keep FDE. The tournament has 30 or 31 games in it so there will be <2^31~=2*10^9 possible tournaments. My highly un-optimized python script does about 1000 tournaments per second, so a similar script that explicitly calculated the probability of every possible tournament would take 23+ days or about half the tournament. :-( However, I think a clever algorithm that makes efficient use of space and time should be able to calculate exactly the odds of each player going out in each round and exactly the odds of each player winning the game given a set of StDev 0 ratings. Coding this algorithm would be very tricky though. For a 16 player single elimination tournament, the chance that seed 1 survives round 2 is: P(1sr2) = P(1b16) * (P(8b9) * P(1b8) + P(9b8) * P(1b9)) where P(1b16) is the probability that seed 1 beats seed 16. The chance that seed 1 survives round 3 can be simplified to: P(1sr3) = P(1sr2) * (P(4sr2) * P(1b4) + P(5sr2) * P(1b5) + P(12sr2) * P(1b12) + P(13sr2) * P(1b13)) The number of required calculations roughly doubles each round, and for N players the tournament has log2(N) rounds. The sum of 2^x from x = 0 to log2(N) - 1 is equal to N - 1. Thus to calculate the odds for each of the N players in the tournament takes O(N^2) calculations, which is much smaller than the 2^(N - 1) possible tournaments. Thus it would take a negligible amount of time to calculate the odds for a 128 person FIDE knockout tournament. Most importantly it should be fairly easy to implement the necessary equations in loops, rather than writing them all out by hand. The situation for the 16 player FDE is much more complicated. In round 2, seed 1 can play any of seeds 8 through 15. If seed 1 won in round 1, then he plays the lowest ranked person who won in round 1; if seed 1 lost in round 1, he plays the lowest ranked person who lost in round 1. After round 2 and all subsequent rounds, each seed is associated with three probabilities for the number of losses he could have. Thus the chance of seed 1 being eliminated in round looks like: P(1L2r2) = P(16b1) * (P(2b15) * P(15b1) + P(15b2) * P(3b14) * P(14b1) + P(15b2) * P(14b3) * P(4b13) * P(13b1) + … + P(15b2) * P(14b3) * P(13b4) * P(12b5) * P(11b6) * P(10b7) * P(8b9) * P(9b1) + P(15b2) * P(14b3) * P(13b4) * P(12b5) * P(11b6) * P(10b7) * P(9b8) * P(8b1) which can be simplified by factoring. The probability of seed 1 going undefeated is the mirror image of this. The probability of just 1 loss, is P(1L1r2) = 1 - P(1L0r2) - P(1L2r2) The probabilities for the other seeds are probably somewhat more complicated. However, there is a big problem with this approach; the requirement that people play the people they have played least. This means that the tournament is path dependent. Also you have to keep track of byes. It should be possible to handle all of this and still have the calculations run in a reasonable amount of time, but I don’t have any idea how to translate it into code that doesn’t take thousands of lines of code or more. Unless someone sees some clever way to make this work without huge effort, I will stick with the simulations, which are already significantly more precise than the ratings are accurate. The only big advantage of a program that makes exact calculations is the ability measure the sensitivity of the tournament to small perturbations, like directly making changes to the odds for pairs of players to account for Achilles’ heels. on 11/15/05 at 16:59:41, Fritzlein wrote:
The goal of the prediction contest is not to maximize your score per se, it is to maximize the probability of having the top score or at least within the top 3. For the 98% case, I think it is best to predict 95%. The 1 point you give up if the favorite wins will most likely vanish into the move per game noise. If that very rare upset does happen the 39 point advantage might be decisive. It depends on how many people are in the contest, but if there are as many as there are players, then I believe slow and steady is unlikely to win the race. I think the winners will be people who go against the grain and are right at least once. Or maybe someone who is very good/lucky at estimating moves per game. on 11/15/05 at 19:38:52, Fritzlein wrote:
Yes, the horizon effect is substantial. Imagine a final game for the title between 99of9 and PMertens. One could potentially make relative gains of 300+ points by predicting 100% for PMertens, and if you are 100+ points out of the money what do you have to loose? |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by PMertens on Nov 16th, 2005, 2:59am statistically I will win against 99of9 anyway ;-) |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 16th, 2005, 9:00am on 11/16/05 at 00:44:52, Ryan_Cable wrote:
Predicting the whole tourney in advance would be mind-boggling, and probably a lot of fun, but I'll bet Omar wouldn't go for it. He would probably rather force the spectators to come back each week and stay actively involved, rather than allow folks to predict and come back two months later to see how they did. Quote:
This cuts both ways, though. If there are ten matchups in the tourney that are 98%, then there probably won't be an upset among them, and the extra ten points from predicting 100% could be decisive too. This isn't an automatic hedging situation in my mind. Of course, towards the beginning of the tourney you don't know whether the standings will be close or not later on, so it is really a matter of style right now. If the true winning probability is 98% I'd rather go with 100% to increase my variance if it is early in the tourney, or if it is late and I am trailing. On the other hand, if I am ranked near the top, I'd go with 95% early and even lower than 95% late. Quote:
It's a good point that the number of contestants matters too. There were 19 contestants last year, so we might be close to 30 this year. The larger number of games this year favors slow and steady, but if there is a larger pool of competitors, surely one of them will get lucky with outrageous guessing. Quote:
Excellent example. I keep forgetting that variance is relative to the field. If everyone else is betting 100% on one side of a contest, then for me to make the "wild" 100% bet that everyone else is making provides no variance at all. On the other hand, as I recall from last year, the field usually spread out substantially in their predictions. Even in coin flip games there would be plenty of people predicting 100% on both ends. Probably no matter what predictions you make, you'll be different from some segment of the playing pool, and have a chance to gain on them. From my conversations with other people last year, I inferred that 99of9 and myself were the most conservative betters, hedging against upsets all the time. That stood us in good stead against most of the field, but that darned Aamir had picked Naveed to upset 99of9 (at 100%, no less!) and that gain ultimately proved unsurmountable. We got a consolation prize for 4th and 5th place last year, but if we take 4th and 5th again this year, we will walk away empty-handed. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Adanac on Nov 16th, 2005, 10:58am on 11/16/05 at 09:00:29, Fritzlein wrote:
Are the predictions private, meaning that we only see the total scores but not the individual predictions? I was looking forward to seeing how everyone predicted each game, but it's understandable that some people might want to have hidden predictions in order to avoid offending players. And, sorry for steering away from the main discussion, but I have a question about the scoring: Players cannot select the number of moves on their own games. If a player wins they will get 100 points minus the number of moves it took to win. If the player loses they will get points equal to the number of moves the game took. This encourages a winning player to try and win as fast as possible and it encourages a losing player to try and prolong the game as much as possible. Although after 70 moves a losing player will not get more points for prolonging the game. This is so that a player does not get more points for losing a long (over 70 move) game than for winning. In case of a draw both players get 25 points. So if 2 players each predict themselves with 55% likelihood, then the winner of a 100 move game gets 30 for winning + 21 prediction points (total 51) versus 70 minus 19 (total 51) for the loser. But they each get 25 points for a draw. Doesn't that mean that if the losing player had seen a potential drawing move, it's actually better not to make it? I'm going to predict all 8 first round games to end in a draw, so I want everyone to have as much incentive as possible. ;) |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 16th, 2005, 12:54pm on 11/16/05 at 10:58:21, Adanac wrote:
Yep, we only see aggregate predictions, so individual predictions can only be inferred from the standings, or from what other people tell you. Quote:
Can we predict draws? I would advise against such predictions, because the tournament rules say draws are not allowed. Capturing all opposing rabbits is a win within the WC. Even in the esoteric situation that the game oversteps time limit and is therefore decided by score, which happens to be tied, there is no draw, because Silver wins that case. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Ryan_Cable on Nov 16th, 2005, 3:19pm on 11/16/05 at 12:54:35, Fritzlein wrote:
Are the standings published after every game or just every round? on 11/16/05 at 10:58:21, Adanac wrote:
When it comes to making gameplay decisions based on the prediction contest, I think we mostly have to trust in people’s honesty. The rules prevent people from personally gaining from betting against themselves, but strong favorites can still make 300+ point relative gains by predicting 55% for themselves and throwing the game in ~70+ moves. Of course if you predict 55% for yourself against a very weak opponent, it is very strong evidence you are up to no good. Still, I think the prohibition on predicting against oneself mostly just penalizes weak underdogs, who in 2 games will probably loose ~200 relative points by betting on themselves when it is wise not to. A computer savvy cheater could just set up sock puppets anyway. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 16th, 2005, 5:34pm on 11/16/05 at 15:19:28, Ryan_Cable wrote:
Just every round. Otherwise you could deduce fairly well what individuals had predicted. Quote:
If you are strong enough that everyone is predicting 100% on you, then throwing the game probably costs you more in expected share of the $500/$200 tournament prize pool than it gains you in expected share of the $50/$30/$20 prediction prize pool. For example Belbo has at least $15 of equity from, say, 1% chance of winning the Championship and a 5% chance of being runner up. The only game he could throw as a favorite is the first one, and it's hard to see him gaining $15 of prediction prize equity in that way. It's only one game of thirty, and in any case conservative folks like me are not predicting him anywhere near 100%, so he wouldn't gain much on us. And of course BlackKnight would gain even more than Belbo! Bigger favorites could make a bigger average gain by intentionally losing, but they would be throwing away even more tournament equity, so I don't see this ever paying off. In a few years Arimaa will have become sufficiently popular that we can bar the participants in the World Championship from also participating in the prediction contest. In the mean time, I think you are right that we have to basically trust people to behave honsetly, but I also think Omar has done a decent job of removing blatant incentives to dishonesty. Quote:
This handicap is moderated by the fact that everyone except the eventual champion will lose twice when forced to bet on himself, but it's true that the biggest underdogs stand to lose the most by this rule, because when stronger players lose, more of the field was betting on them too. Another moderating factor is that, although betting on underdogs rarely pays off, when it does pay off, it pays off big. A final moderating factor is that the favorites who keep winning have an offsetting handicap in not being able to predict their own game length. Looking at the six games in the semifinals and finals from last year, the predictors averaged about 75 points per game from the game length portion (when they called the winner correctly), whereas the winners of the games averaged only 52 points for speed of victory. If I survive into the late rounds, I'll be giving the field more than 20 prediction points per game I play. Still, I admit it does work out badly for the low seeds on the balance. Here's a small idea to compensate underdogs: In recompense for being forced to bet on yourself, if you lose when the average bet was against you, you get (average bet) - 50 points for free. For example if one guy bets for you at 55% and nine folks bet against you at 95%, the average bet was 90% against you, so if you lose you get 40 free prediction points. This will never turn into an incentive to lose on purpose, it is just a modest consolation prize. The net result of favorites not being able to predict number of moves and underdogs not being able to bet against themselves is that non-players have a slight advantage over players in predicting. This is as it should be. Quote:
Tell me more about this way of cheating. I'm curious. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Nov 16th, 2005, 6:31pm on 11/16/05 at 17:34:30, Fritzlein wrote:
I think he just means setting up extra accounts so you can have extra shots at the prize pool. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Ryan_Cable on Nov 16th, 2005, 7:07pm Sock puppets are one of the standard internet trolling techniques: http://en.wikipedia.org/wiki/Internet_sock_puppet It refers to maintaining multiple user accounts designed to appear as a different people. The idea in this case would be for a player (or non-player) to start additional non-player accounts which then enter the prediction contest. This could be used to support a strategy of betting against oneself or just to multiply ones chances at getting a prize. For instance, one could be slow and steady with one account and wild and wacky with others. Entering a sock puppet in the tournament would be much harder to disguise. To avoid getting caught requires having enough technical savvy to keep your IP from giving away that the accounts are being operated from the same physical location. Also, you would have to invest the effort of playing enough games to make the sock puppet accounts appear plausible. I don’t have any actual experience with network security, so I don’t know what sort of countermeasures Omar could take against this. Analysis of login times would at least give some indication. On the whole, I think there are many simpler ways to scam money, both on the internet and in meat space. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Nov 20th, 2005, 4:11pm I guess now we get to see who was predicting upsets! |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Nov 20th, 2005, 7:09pm on 11/16/05 at 17:34:30, Fritzlein wrote:
What's more, of the top three predictors from round 1: * One is a non-player * One is a winning favourite * One is a losing underdog So it seems that any biases to do with status are less than the biases due to your ability to predict ;-). JDB has obviously predicted extremely accurately given that he took a loss and the consequent penalty in this round. Oh... and none were sock puppets! (Although maybe suspicion should fall on Elmo and co. - how easy would it be for Fritz to create a fictional wife and family of inlaws ;-)) [Humble apologies for that vicious jibe Katie... i couldn't help it. Nice predicting by the way, you've obviously recanted on your strategy of betting against Karl in every game?] |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 20th, 2005, 7:18pm Thanks, Omar, for being so quick in posting the prediction contest results and the pairings for round two! I was quite surprised to see only 21 people placing bets on each game, as I had expected registration to be closer to 30. But now I see there were indeed 28 people registered and 7 didn't enter anything. This is proof in my mind that the refundable registration fee for the WC itself is a good thing. I see my overly cautious bets dropped me to 13th even though I was right on seven of seven. Apparently JDB bet 100% on every game but his own, and this strategy paid off. If he gets knocked out of the championship next round, I'd pick him as the favorite to win the prediction contest even if there are, say, four players ahead of him, if those four predictors are all still in the championship and constrained in the betting on their own games. I've got a great idea: Let's place bets about who is going to win the prediction contest. We could have prizes and standings and everything. ;-) |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 20th, 2005, 7:28pm on 11/20/05 at 19:09:13, 99of9 wrote:
Here's a question, though. If Katie goes to the library so she can log in and watch and chat with you guys in the chat room while I play, will you believe her that she isn't sitting next to me passing on your advice? :o |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Nov 20th, 2005, 7:44pm on 11/20/05 at 19:28:22, Fritzlein wrote:
Well if that happens we'll probably pepper her with so many questions about the process of freezing liquids that there won't be any time for advice anyway! |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by PMertens on Nov 20th, 2005, 8:05pm cosidering some of our advices has yet to be determined wether it would help or damage you :-P |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Ryan_Cable on Nov 21st, 2005, 4:27am After round 1, with ~24% of the non-forfeit games finished, we have: Name . . Points StDev 1 Ryan_Cable . 1201 1.353 2 RonWeasley . 1176 1.196 3 jdb . . 1165 1.128 4 99of9 . . 1136 0.946 5 Elmo . . 1128 0.896 6 Adanac . . 1101 0.727 7 naveed . . 1086 0.634 8 Belbo . . 1072 0.546 9 fritzlforpresident 1039 0.340 10 BlackKnight . 1038 0.333 11 Paul . . 1034 0.308 12 Aamir . . 1028 0.271 13 Fritzlein . 989 0.027 14 omar . . 946 -0.242 15 acheron . . 931 -0.336 16 MrBrain . . 889 -0.598 17 PMertens . . 814 -1.067 18 carolaina . 750 -1.467 19 grey_0x2A . 726 -1.617 20 nbarriga . . 716 -1.680 21 robinson . . 713 -1.699 Median 1034.0 Mean 984.7 StDev 159.9 Fot the winners: Median 1030.5 Mean 996.5 StDev 166.6 For the losers: Median 1034.0 Mean 981.3 StDev 145.4 For the specators: Median 1033.5 Mean 972.8 StDev 194.1 All 15 active tournament players are in the prediction contest, giving 8 winners, 7 losers, and 6 spectators. It seems like those three groups are as close to each other as predicted by chance if not closer. on 11/20/05 at 19:18:13, Fritzlein wrote:
Strongly agree, having no-shows in a floating multiple elimination tournament is significantly worse than in a non-floating single elimination tournament, where there are plenty of first round byes to reach 2^n anyway. But if we just threw people out after a single forfeit, it would make the tournament pairing much harder to predict. And in our specific case, it would ruin the great 16 player structure. on 11/20/05 at 19:18:13, Fritzlein wrote:
I didn’t predict anyone below 85%, but I was probably more cautious than you in relative terms. I was never more than 15% away from the average prediction and often within 10% of it. If the average prediction is 90%, a 100% prediction is much less bold than a 70% one. Of course, the average prediction is not available when making a prediction, but a guesstimate of it should always be in your mind. on 11/20/05 at 19:18:13, Fritzlein wrote:
Not if he keeps predicting everything at 100%. With me @ 1800, omar +100, and 99of9 +75 and ignoring the forfeit, there was a 0.7115 chance of at least one upset, and the expectation was for 1.096 upsets. If there had been just one upset, people who boldly gave low predictions like Fritzlein would be near the top, and those who boldly gave high predictions would be near the middle. However, most of the people who actually predicted upsets were likely to end up near the bottom regardless of the overall number of upsets. on 11/20/05 at 19:18:13, Fritzlein wrote:
Well, if you assumed that the points from the future games would be distributed with some relation to the distribution above you could assign probabilities to everyone. However, it would be very hard to separate the skill from the luck. Even guessing is difficult without knowing people’s strategic approaches. Still, I will go on record with: 1 RonWeasley 2 99of9 3 Ryan_Cable on 11/20/05 at 20:05:07, PMertens wrote:
I heard that the peanut gallery supported the H blunder that finished off jdb in my game. Two interesting statistics I would like to see after the contest is over are the number of predictions each person had for the right player and the split in each person’s points between those from win prediction and those from move prediction. I don’t think that this would violate the promise that Individual predictions are kept confidential, but the overall prediction statistics are displayed to everyone. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Nov 21st, 2005, 6:36am on 11/21/05 at 04:27:45, Ryan_Cable wrote:
Just Omar, oops. We corrected him before you made your move. Then again, at another time I wanted you to make a move that was impossible! |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 21st, 2005, 10:05am Nice statistical analysis, Ryan! That sure seems to bust my idea that non-players have a significant advantage over players. I was quite confident that non-players have an advantage of about 20 points per game, which should show up as about 140 points over the seven games so far. It's funny, though, that I can't give up my intuition even in the face of mathematical evidence to the contrary. Thanks for pointing out that my "conservative" guesses were probably as radical, relative to the field, as guessing 100% on every game. In that light, it wasn't so much that didn't gamble enough as that I gambled and lost. I think that one's relation to the field is not the only relevant factor, though. One's relation to the true percentages is also very important, as 99of9 is trying to prove. I bet on all the favorites, but cumulatively predicted an average of 1.65 upsets in the first eight games. If the true average number of upsets was going to be 1.096, then I cost myself points, on average, relative to an accurate predictor, and I can only justify it in terms of being willing to accept a lower average in exchange for a higher variance. I agree with your intuition that people who actually predicted upsets were giving away too many points, on average, to justify their huge increase in variance, but that's only because the tournament has so many games this year. As the number of remaining games dwindles, wild predictions get more appropriate. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by RonWeasley on Nov 21st, 2005, 11:34am It's a bit early to draw conclusions. This first round had the most easily predicted games. The pairings are much closer in the rest of the tourney. We should see lower absolute scoring and more variance. The total number of games is interesting because it's in the region where the central limit theorem takes effect and a percentage strategy does as well as an upset stategy. Ryan, you put so much pressure on me. You know how I fall apart under pressure! But I'm off to a good start and I hope my non-player advantage pays off. The player who wins all his games gets this advantage too. An earned benefit. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 22nd, 2005, 10:09am Now I am feeling that JDB was right about betting 100% on the early rounds, because being behind puts the pressure on. I already feel that I have to go ballistic in the second round of the tournament. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 22nd, 2005, 3:05pm Heads up everyone: When Omar corrected the colors on the game Grey_0x2A vs. Megamau, it wiped out my prediction for that game. If you entered your predictions early this week, you had better check them again before game time. On the other hand, it is likely Megamau will forfeit the game, in which case the predictions don't matter. Futhermore, it is the last game of the week, so everyone will probably notice the problem when they check to see how they are doing mid-round. And finally, the people who don't look at their predictions mid-round probably aren't going to be reading the forum either, so this message is 99.73% useless, but I've written it, so I'm going to post it anyway. :P |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Nov 22nd, 2005, 4:06pm Thanks for that tip off Fritz |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Ryan_Cable on Nov 24th, 2005, 10:18pm on 11/21/05 at 10:05:34, Fritzlein wrote:
Just to be clear, if each player in a game loses ~20 move points relative to people not in the game, ~280 points total are lost by all players relative to the spectators, but that still shows in the averages as just a ~19 point disadvantage. (I think the real move penalty is more like ~18 for winners and ~32 for losers.) However, if each underdog looser bets 55% on himself, while everyone else predicts 85% for the winner, each underdog looses 112 relative points, which should put the looser average ~132 points below the spectators and ~110 points below the winners. The winner average should only be ~19 points below the spectators. All of the players have constraints on their win predictions that the spectators don’t have, and rational actors can’t benefit from having their decision space reduced. The two players in the game can collectively get 100 move points, and they get even less if the game goes beyond 70 moves. To get less than 50 move points, people not in the game must miss the move number by >50% low or >100% high, which requires either bold move predictions or near record move numbers. However, the players still get move points in their games even if they lose, which could be a ~10 point advantage in a tossup game. All together, spectators should be advantaged relative to players. While I think this is important to individual players, especially ones who are underdog losers, it is masked in the group averages by several effects. The groups of round one winners, losers, and spectators were far from randomly chosen. Collectively, I think the round 1 winners are likely to have advantages over the round 1 losers in several useful traits, such as Arimaa experience, prediction experience, strategic acumen, and familiarity with the players. I think the spectators are likely to fall below the losers in many of these traits. For an example, 5 winners, 4 losers, and 3 spectators took part in the 2005 prediction contest, and 3 winners and 2 losers took part in the 2004 prediction contest. This whole discussion is plagued by small number statistics, but the distribution of prediction experience is in the claimed direction. In round 1, we already see a substantial advantage for people with previous experience: People with 2005 experience: Median 1033.0 Mean 1003.4 StDev 131.2 People without 2005 experience: Median 1034.0 Mean 959.7 StDev 197.6 People with 2004 experience: Median 1072.0 Mean 1025.8 StDev 103.7 People without 2004 experience: Median 1031.0 Mean 971.8 StDev 174.6 Yet, strangely it is the losers who have substantially over-performed this round when corrected for the penalty of predicting for themselves. My guess is that the losers have over-performed this round by being aggressive (bold high) while others were passive (bold low) or cautious, and will suffer next round. Of course it could be that the losers just happen to be collectively the most skilled predictors, despite the odds against it. Also, I would guess that the unforced upset predictions were mostly made by spectators. And the spectators lost 100+ points (17+ points from the mean) by not predicting 1 game. I think that skill will play a greater role in future rounds, and will lead to a greater differentiation between the groups of round 1 winners, losers, and spectators. Tracking these three round 1 groups all the way to the end will probably be marred by people loosing interest and not making predictions, but my guess is that we will see at least 0.2 StDev in means between winners over losers and between losers over spectators. Note that next round, the winners’ bracket games will have about as many underdogs as the losers’ bracket games. on 11/21/05 at 10:05:34, Fritzlein wrote:
Good point, I have been implicitly assuming we have a wise crowd without realizing it. Including the Adanac vs. megamau game, the crowd gave an expectation for 1.367 upsets. If we weight down the upset predictions to remove the forced predictions, the crowd predicts 1.207 upsets. If we remove all the upset predictions, the crowd predicts 1.02 upsets. These numbers are pretty close to those predicted by my modified ratings, and crowd numbers could very well be more correct. I was a little surprised by how high the crowd's upset numbers turned out to be; maybe there are not as many overly aggressive predictors heading for a comeuppance in round 2 as I thought. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Nov 24th, 2005, 10:49pm on 11/24/05 at 22:18:36, Ryan_Cable wrote:
From game 1 of round 2 it seems this is not the case anymore! |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Ryan_Cable on Nov 25th, 2005, 1:32am on 11/24/05 at 22:49:44, 99of9 wrote:
Amazing! Comeuppance galore! 14 for PMertens in 49; 80% sure [!!!] 7 for Adanac in 46; 66% sure 64.7% for PMertens. At the time of the game, Adanac actually had a 26 point rating advantage, predicting a 0.53735 chance of winning! Also, Adanac had a 4-5 record against PMertens and went 1-1 against him just a week ago. PMertens had told me he thought he would win if he didn’t blunder, and I predicted 65% for him, thinking I was being rather bold. With these crowd numbers, I was a fool to not predict 55% for Adanac and grab all of the free variance. Oh well, it just supports the point I made about previous experience. ;-) |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by PMertens on Nov 25th, 2005, 3:53am on 11/25/05 at 01:32:00, Ryan_Cable wrote:
well ... unfortunately I got a bad habit there ... Quote:
since I did bet 100% on me I do not really feel guilty :P Quote:
looks like we got plenty of fools in our community ;D |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Ryan_Cable on Nov 25th, 2005, 4:41am on 11/25/05 at 03:53:00, PMertens wrote:
I don’t blame you. I would have predicted 55%+ for you anyway. I just wish your game had been scheduled after a few other games had shown me just how incredibly aggressive people were being. Then I would have bet against you for strategic reasons. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 25th, 2005, 12:20pm on 11/25/05 at 01:32:00, Ryan_Cable wrote:
Grabbing the free variance was my strategy exactly when I predicted 55% for Adanac. I considered the game an absolute coin flip, but guessed (correctly) that more people would bet on PMertens. (Well, I admit that hearing about PMertens' playing conditions influenced me a little too.) I also considered the game Paul vs. MrBrain to be a near coin flip. I had the advantage of having played MrBrain in the first round, and having sweated bullets to win despite everyone predicting 100% on me. Had the seeding been otherwise I might have bet 55% on Paul, but with MrBrain vastly underrated I opted for all the free variance that a 55% wager on him would provide. Now that I've called two upsets, I am probably close to the lead, so it behooves me to get more conservative. As long as my position in the prediction contest remains realtively good, I'm going to stay as near to true percentages as I possibly can, hopefully thereby forcing folks who are trailing to get crazy with their predictions. By the way, Ryan, I somehow missed your excellent post in this thread from Monday. I like your analysis of the penalty for players in the game, particularly the average penalty for underdog losers. That makes it seem appropriate to have a moderate compensation to big underdogs who lose, based on how heavily the betting went against them. Of course betting favorites who lose would still get nothing, and small underdogs who lose would get only a little. Here's another formula to go with my previous proposal: Have the point value compensation be simply (average bet on favorite) - (average bet on underdog), where each predictor is thought to be betting partly on each player. So for example in round one these bets 19 for 99of9 in 33; 99% sure 2 for acheron in 50; 78% sure make 99of9 the favorite by (19*99 + 2*22)/21 = 92 percent average prediction versus 8 percent average prediction on Acheron. Since Acheron lost as the underdog, the compensation would be 92 - 8 = 84 points. Those 84 points are still less than Acheron is going to lose on average by being constrained in the predicting. This creates an incentive for an underdog to lose, but recall that all incentives are relative. The bonus for losing is nowhere near the bonus for pulling off the upset, namely gaining ~350 points on most of the field. Even in closer games, the bonus only kicks in when the betting underdog loses, and I'm guessing that the betting underdog will alway gain more relative points by winning than by losing. Well, we'd have to look at it more closely, but it seems a reasonable way to level the playing field. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Adanac on Nov 25th, 2005, 3:42pm This is my first ever prediction contest, and I really like the way it adds an extra level of excitement to an already exciting tournament. Are there any other prediction contests? i.e. Human vs. Bot Challenge Match or Bot Championships? on 11/25/05 at 12:20:37, Fritzlein wrote:
Since players don’t have a lot of flexibility in predicting their own games anyway, and they appear to be at a disadvantage compared to the spectators, and weaker players are at a further disadvantage, why not just award both players a number of points equal to the average [mean] score from all the other participants. So far example, in the Paul vs. Mr. Brain game both players would receive a score (I’ll take a wild guess here) of roughly -110 while in the Fritzlein vs. Mr. Brain game both players would receive a score of roughly +185. It’s not perfect, but it achieves a few objectives: (1) Reduces/Removes incentive for a heavily favoured player to bet 55% and throw a match to *relatively* gain points on the rest of the field. (I can't imagine anyone would actually do this anyway) (2) Removes underdog disadvantage (3) Removes spectator advantage |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 25th, 2005, 10:36pm on 11/25/05 at 15:42:19, Adanac wrote:
I think the WC prediction contest is the only one, but I agree it is great fun. Quote:
I didn't think of that, but now that you say it, it seems obvious and best. On a game you are involved in, you don't get to bet, and you get the mean of what everyone else got. Even if you are in, say, seven of the thirty games of the WC, you still get to bet on the other 23, so you have enough chance to have fun there. Great idea. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 26th, 2005, 4:52am on 11/25/05 at 12:20:37, Fritzlein wrote:
Easy come, easy go. JDB's upset of Naveed surely gives JDB a commanding lead at this point. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Adanac on Nov 26th, 2005, 6:53am on 11/25/05 at 22:36:26, Fritzlein wrote:
I just thought of a problem with my own suggestion. Omar wanted the incentive for a player to fight on as long as possible, even if the game is certainly lost (which is a great feature of the existing system). We would have to reward the players for number of moves but then my system no longer works because it gives an advantage to the higher ranked player over both the underdog and the spectators... :-/ |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Nov 27th, 2005, 4:25pm on 11/26/05 at 04:52:06, Fritzlein wrote:
So it seems - but he must've been picking the other upsets too, since his game was the only one I missed. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Nov 29th, 2005, 5:29pm What a turnaround for the field. The same twenty-one predictors who averaged a gain of 985 points over the first seven games, averaged a gain of only 122 points over the next eight games. One more upset, and we might have been negative as a group for round two! |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Ryan_Cable on Dec 1st, 2005, 7:30pm The round 2 scores: Name . . Points StDev 1 jdb . . 923 1.747 2 Fritzlein . 765 1.379 3 99of9 . . 731 1.299 4 Paul . . 664 1.143 5 Elmo . . 533 0.838 6 Adanac . . 498 0.756 7 MrBrain . . 492 0.742 8 nbarriga . . 417 0.567 9 RonWeasley . 195 0.050 10 naveed . . 154 -0.046 11 grey_0x2A . 82 -0.213 12 BlackKnight . 77 -0.225 13 mohabbatse . 53 -0.281 14 Ryan_Cable . 32 -0.330 15 Belbo . . 19 -0.360 16 robinson . -34 -0.484 17 acheron . . -158 -0.773 18 PMertens . -297 -1.097 19 omar . . -426 -1.398 20 fritzlforpresident -523 -1.624 21 Aamir . . -552 -1.691 22 carolaina . -1029 -2.803 Median 82.0 Mean 173.6 StDev 429.0 The total scores after round 2, with ~48% of the non-forfeit games finished: Name . . Points StDev 1 jdb . . 2088 1.733 2 99of9 . . 1867 1.339 3 Fritzlein . 1754 1.137 4 Paul . . 1698 1.038 5 Elmo . . 1661 0.972 6 Adanac . . 1599 0.861 7 MrBrain . . 1381 0.473 8 RonWeasley . 1371 0.455 9 naveed . . 1240 0.222 10 Ryan_Cable . 1233 0.209 11 nbarriga . . 1133 0.031 12 BlackKnight . 1115 -0.001 13 Belbo . . 1091 -0.044 14 grey_0x2A . 808 -0.548 15 acheron . . 773 -0.610 16 robinson . . 679 -0.778 17 omar . . 520 -1.061 18 PMertens . . 517 -1.066 19 fritzlforpresident 516 -1.068 20 Aamir . . 476 -1.139 21 mohabbatse . -94 -2.155 22 carolaina . -279 -2.484 Median 1133.0 Mean 1115.5 StDev 561.3 The following tables list round 1 scores, round 2 scores, total scores. (Note that mohabbatse was included in the round 2 numbers only.) Round 1 winners: Median 1030.5 25.5 1162.0 Mean 996.5 161.0 1157.5 StDev 166.6 452.4 549.0 Round 1 losers: Median 1034.0 154.0 1240.0 Mean 981.3 319.1 1300.4 StDev 145.4 383.7 473.2 Spectators: Median 1033.5 53.0 824.5 Mean 972.8 -129.4 813.0 StDev 194.1 580.0 710.9 People with 2005 experience: Median 1033.0 115.5 1177.5 Mean 1003.4 115.8 1119.3 StDev 131.2 603.0 698.6 People without 2005 experience: Median 1034.0 67.5 1133.0 Mean 959.7 122.6 1090.0 StDev 197.6 343.3 419.6 People with 2004 experience: Median 1072.0 154.0 1240.0 Mean 1025.8 194.0 1219.8 StDev 103.7 445.5 487.7 People without 2004 experience: Median 1031.0 77.0 1115.0 Mean 971.8 96.8 961.0 StDev 174.6 514.2 631.3 Undefeated players: Median 1045.0 614.5 1676.5 Mean 984.8 490.0 1474.8 StDev 191.7 368.9 541.8 Players with one loss: Median 1038.0 32.0 1115.0 Mean 1017.9 117.1 1135.0 StDev 142.5 461.0 538.6 Eliminated players: Median 982.5 118.0 1024.0 Mean 944.3 185.5 1129.8 StDev 159.1 345.8 434.3 The spectators are having the worst performance as I expected, but the round 1 losers now have strong evidence that they are the most skilled group of predictors. Experience seems to have been somewhat less valuable in round 2. The undefeateds are doing quite well but there are only 4 of them, and I think it is mostly due to coincidence. Two of the eliminated players were round 2 favorites, so that might account for a small portion of their good scores, but I think it mostly comes from them being a subset of the round 1 losers. The round 2 scores have a 0.442 correlation with the round 1 scores. The round 1 scored have a 0.623 correlation with the total scores. The round 2 scores have a 1.178 correlation with the total scores. These correlations are obviously inflated due to the fact that the scores are significantly more spread out than a normal distribution, but I think that they should still be fairly accurate relative to each other. The crowd predicted 2.144 upsets. Weighting down the upset predictions to remove the forced predictions, the crowd predicted 1.990 upsets. My ratings estimates predicted 2.03035 upsets, which is very close. However, the round 2 mean is much less than what could have been had by a score expectation maximization strategy. The people I predicted to win after round 1 are all still in the running, but after round 2, I think the most likely winners are: 1 99of9 2 Fritzlein 3 jdb |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Dec 1st, 2005, 7:49pm on 12/01/05 at 19:30:35, Ryan_Cable wrote:
Wow, we still have half the prediction comp to go! Maybe I'll have a chance to make up for my 85% Omar blunder ;-). |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Dec 1st, 2005, 9:18pm Ryan, thanks again for this interesting statistical analysis. The results of this year will definitely be influential in next year's betting. However, I still believe that the final ordering is going to come down to who is lucky in guessing the coin-flip games. There's no mathematical analysis that's going to work as well as simply guessing right on tossup games. Come to think of it, we're already seeing it: 99of9 and I are near the top mostly because of guessing right on the two second-round tossups. on 12/01/05 at 19:49:31, 99of9 wrote:
99of9, I was only 65% for Omar, so that puts us in a virtual tie for second place, unless Paul or Adanac bet on BlackKnight and leapfrogged us. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Dec 1st, 2005, 9:47pm on 12/01/05 at 21:18:31, Fritzlein wrote:
By my reckoning they were not tossups nor upsets. I know this is counter to the majority opinion, but my predictions were for who I thought had better chances (based on their previous games and knowledge of their playing styles etc), and had nothing to do with exploiting variance. In fact I bet over 55% on one of them. I think the fact that the majority were betting ~80% for the loser indicates that we were either extremely lucky, or we had some better knowledge than the majority. Even thinking that it was a tossup shows your knowledge was superior. Perhaps my *favouritism* of the eventual winners was overconfidence in the lucky direction, but I'd say it was much closer to the truth than 80% on the losers! However, you're right that the next half of the comp will have lots more coin flips in it. So perhaps that will determine the eventual winner. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Dec 2nd, 2005, 10:05am on 12/01/05 at 21:47:20, 99of9 wrote:
I think we were definitely lucky to be right, and my knowledge of the situation consisted only of knowing that I should bet 55% on a game when I'm not sure who to bet on. My only extra information was that PMertens was playing late at night in a hotel lobby, which clinched the choice when I was wavering. Given that Mr. Brain was losing his game until he won, and that while I watched the PMertens-Adanac game I changed my mind five or six times about who was going to win, I hardly feel I'm in a position to say, "I knew it all along," when the final results happen to match my guesses. Of course, maybe you were right and the winners of those games were actually the favorites. There is no way to distinguish luck from skill in the short run, so you are perfectly justified in thinking you are skilled. ;-) And you've done so consistently well in prediction contests (4th, 5th, and now ~3rd) it isn't really the short run any more. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Adanac on Dec 2nd, 2005, 11:01am on 12/01/05 at 21:18:31, Fritzlein wrote:
I used the predict-the-opposite-of-everyone-else-in-a-close-matchup theory to pick BlackKnight at 55% even though Omar probably had a better than 50% chance of winning. I could pretend it was a brilliant pick, but to be honest, I didn't even research the W-L record of the 2 players before the game. I still haven't, for that matter :) |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Dec 2nd, 2005, 11:15am on 12/02/05 at 11:01:00, Adanac wrote:
Welcome to second place in the prediciton standings! |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Dec 3rd, 2005, 9:39am Three upsets in three games in round three! Does anyone care to make a side bet on whether we will collectively lose points in the third round of predicting? In the middle of the second round of the tournament, I guessed I was briefly in second place in the prediction standings, but now I'm bleeding prediction points with every game, and mentally demoting my rank as we go. We'll see how bad it gets by the end of the round. I'm shocked that the average of the five predictors for Adanac was 78%. Somebody took a chance and hit the jackpot, and it sure wasn't me. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Adanac on Dec 3rd, 2005, 10:24am on 12/03/05 at 09:39:42, Fritzlein wrote:
I’ve estimated the aggregate score for round 3 by conservatively guessing the distribution of predictions, excluding those that didn’t predict: it’s -6920!! To answer your question, yes I would :) ESTIMATES: 13 players predicted Omar: Estimated Total Score = -2080 5 players predicted BlackKnight: Estimated Total Score = +705 19 players predicted Belbo: Estimated Total Score = -4370 14 players predicted 99of9: Estimated Total Score = -1890 5 players predicted Adanac: Estimated Total Score = +715 Estimated Total Score = -6920 It's too bad for Megamau that he didn't participate. He could be in first place! on 12/03/05 at 09:39:42, Fritzlein wrote:
It sure wasn’t me. I’ve predicted myself at 55% in each of the past 2 games. That means that 4 players averaged 83.75% on me, likely players lower in the standings that needed to gamble to climb back up the ladder. If those same 4 players also predicted BlackKnight, we could be in for a surprise when the round 3 standings are published. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Dec 11th, 2005, 7:51pm Wow, all those upsets in round 3 didn't spread things out very much. In fact, it is tighter at the top than it was after round 2. Fifth place is currently 217 pionts behind first, whereas after round two, the gap between first and fifth was double that. With eight or nine games left, and lots of close matchups among those games, the contest is still wide open. Adanac jumped into first, while JDB dropped a spot to second, but I'd have to say that JDB is the favorite now since he is eliminated while Adanac is undefeated. On the other hand, my theory that non-players have an advantage is somewhat undermined by the fact that the highest observer is Elmo in 7th place. The closer it gets to the end, the less anything counts other than guessing right! |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Ryan_Cable on Dec 12th, 2005, 11:58am The Round 3 scores: Name Points StDev 1 robinson . . 92 . 1.365 2 Ryan_Cable . 86 . 1.345 3 Adanac . . 72 . 1.299 4 nbarriga . -43 . 0.922 5 grey_0x2A . -46 . 0.912 6 acheron . . -80 . 0.801 7 Paul . . -126 0.650 8 Fritzlein . -179 0.477 9 omar . . -237 0.287 10 MrBrain . . -265 0.195 11 Belbo . . -305 0.064 12 99of9 . . -313 0.038 13 PMertens . -404 -0.261 14 Elmo . . -405 -0.264 15 jdb . . -426 -0.333 16 Aamir . . -466 -0.464 17 BlackKnight . -511 -0.611 18 naveed . . -651 -1.070 19 mohabbatse . -855 -1.738 20 carolaina . -862 -1.761 21 RonWeasley . -890 -1.853 22 fritzlforpresident -963 -2.092 Median -305.0 Mean -324.5 StDev 305.2 The total scores after round 3, with ~71% of the non-forfeit games finished: Name Points StDev 1 Adanac . . 1671 1.245 2 jdb . . 1662 1.232 3 Fritzlein . 1575 1.110 4 Paul . . 1572 1.106 5 99of9 . . 1554 1.080 6 Ryan_Cable . 1319 0.750 7 Elmo . . 1256 0.661 8 MrBrain . . 1116 0.464 9 nbarriga . . 1090 0.427 10 Belbo . . 786 0.000 11 robinson . . 771 -0.021 12 grey_0x2A . 762 -0.034 13 acheron . . 693 -0.131 14 BlackKnight . 604 -0.256 15 naveed . . 589 -0.278 16 RonWeasley . 481 -0.430 17 omar . . 283 -0.708 18 PMertens . . 113 -0.947 19 Aamir . . 10 -1.092 20 fritzlforpresident -447 -1.735 21 mohabbatse . -949 -2.442 22 carolaina . -1141 -2.712 Median 771.0 Mean 786.2 StDev 710.6 The following tables list round 1 scores, round 2 scores, round 3 scores, and total scores. (Note that mohabbatse was included in the round 2 and round 3 numbers only.) Round 1 winners: Median 1030.5 25.5 -208.0 1052.5 Mean 996.5 161.0 -148.5 1009.0 StDev 166.6 452.4 202.5 608.0 Round 1 losers: Median 1034.0 154.0 -265.0 762.0 Mean 981.3 319.1 -300.7 999.7 StDev 145.4 383.7 233.8 457.4 Spectators: Median 1033.5 53.0 -855.0 245.5 Mean 972.8 -129.4 -640.6 208.2 StDev 194.1 580.0 342.6 920.7 People with 2005 experience: Median 1033.0 115.5 -404.5 695.0 Mean 1003.4 115.8 -418.7 700.6 StDev 131.2 603.0 190.7 815.5 People without 2005 experience: Median 1034.0 67.5 -63.0 771.0 Mean 959.7 122.6 -198.9 879.1 StDev 197.6 343.3 389.6 644.4 People with 2004 experience: Median 1072.0 154.0 -305.0 786.0 Mean 1025.8 194.0 -354.2 865.6 StDev 103.7 445.5 168.7 489.6 People without 2004 experience: Median 1031.0 77.0 -404.0 766.5 Mean 971.8 96.8 -353.3 749.4 StDev 174.6 514.2 365.5 807.9 Undefeated players: Median . . . 82.0 1221.0 Mean . . . 82.0 1221.0 StDev . . . 14.1 636.4 Players with one loss: Median . . . -313.0 1319.0 Mean . . . -264.2 1033.0 StDev . . . 230.6 647.6 Eliminated players: Median . . . -251.0 774.0 Mean . . . -267.0 932.9 StDev . . . 199.6 481.4 The spectators have fallen behind by more than a StDev. The round 1 losers and winners are about equal. People with 2005 experience did >200 points worse in round 3 than those without experience, which gives an idea how much all of these numbers are effected by noise. The round 3 scores where negatively correlated with the round 1 scores at -0.221. This is likely because round 1 rewarded aggressive predicting, but round 3 rewarded passive predicting. Round 3 has 0.693 correlation with the total score, putting it midway between round 2 at 1.133 and round 1 at 0.397. This is probably because predictions were more bunched up in round 3, with everyone predicting for Belbo for instance. The crowd predicted 1.539 upsets. Weighting down the upset predictions to remove the forced predictions, the crowd predicted 1.399 upsets. My ratings estimates predicted 1.02072 upsets. Since there were actually 3 upsets, this could be an indication that my ratings estimates are underestimating the chance of upsets, or it could be that desperation is causing people to predict upsets to have a chance to recover from poor performance in previous rounds. My winning predictions from round 2 still look good, but RonWeasley does seem to have cracked under the pressure from round 1. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Dec 12th, 2005, 12:56pm on 12/12/05 at 11:58:50, Ryan_Cable wrote:
The penalty for not predicting is 21 points per game, so it looks suspiciously like Paul did not enter any predictions, and out-performed the field by doing so. His overall position stayed steady at fourth place, but now he's only 100 points out of first place, whereas he was almost 400 points out of first before the round. If the rest of us have another debacle predicting on round 4, an absent Paul could not only move up into the money, he could climb all the way to the top! |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by omar on Dec 13th, 2005, 1:17am I've noticed that those of us at the bottom have almost no incentive now to keep predicting. I have about a zero chance of catching up to the top predictors :-) To keep things interesting for everyone next year, maybe we can pool a registration fee which is redistributed based on performance. Kind of like how we did with the postal tournament. So even at the bottom I would still have an incentive to try and get back as much of my registration fee as possible. Perhaps I would also not have made such wildly confident perdictions to begin with :-) |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Dec 13th, 2005, 2:47pm on 12/13/05 at 01:17:59, omar wrote:
I would continue to enter predictions to see how well I could do the rest of the way, but I can understand if people quit when they have no more chance to win. It is important to note that quitting doesn't spoil the fun for other people who are still in contention. Quote:
I doubt I will play in future prediction constests if the prize money comes from pooled entry fees. The prediction contest is fun, but I'm not here to gamble, I'm here to play Arimaa. Also I wonder whether spectators with a casual interest in the game (like my brother-in-law fritzlforpresident) would play if there were a registration fee. I feel quite differently about the postal tournament. In a round-robin tournament it is unpleasant if someone quits because the remaining players benefit unequally. Probably someone who would have otherwise lost will get a win for free when someone leaves. Furthermore, a postal game can require quite a bit of time in total, and after I've invested a month or two in a game, I would be disappointed to have it be decided by dropout. These factors make a "commitment fee" seem very appropriate. In other events, however, I'd much rather play with no fees and no prize money than have the money come from entry fees. I will try just as hard in the prediction contest next year if there are no prizes offered. I will play just as hard in next year's World Championship if there is no prize money. (Although I admit, I might not get high-speed Internet access just for the occasion.) And already there is no money for successfully defending humanity's honor in the Arimaa Challenge, but I nevertheless would consider it a privilege to play, and do my best to shut out the bots. As Arimaa matures and the playing population grows, I expect you will be able to phase out the prize money you are presently paying out of your own pocket. The prizes are helping to promote the game at present, but eventually lots of people will play just for the love of the game, and try to win events just for the fame. When you reduce your own monetary contribution, however, my personal preference would be to see it replaced with non-money games and events, rather than building the prize pool from entry fees. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by RonWeasley on Dec 13th, 2005, 4:09pm In an opinion representing only myself, I would be happy with no fees and no prizes. The small fees for tournament entry don't bother me, but I might not enter the whimsical prediction contest if the fee was too high. Motivation for playing well is less money motivated than it is to avoid the mortification of blundering in full view of the Arimaa community. Even in a losing position, it would be bad manners to abandon a game. Comical last moves are an exception, though, because they don't waste the opponent's time or respect. This from a minor wizard who can play only owl games. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Ryan_Cable on Dec 15th, 2005, 6:20am on 12/13/05 at 01:17:59, omar wrote:
Yes, but I think this is more than offset by making the game more exciting for the people at or near the top and more interesting for everyone in the beginning. To me, the tradeoff between variance and expectation is the whole attraction of the prediction contest. Simple score maximization would bore me. Of course, if you are wanting the prediction contest to help attract spectators, I can see why you are a little bit disappointed with their performance. on 12/13/05 at 01:17:59, omar wrote:
I agree with Fritzlein and RonWeasley. I have spent far more than $50 worth of my time thinking about who is going to win the WC, and trying to find a good strategy for the prediction contest. And I can assure you I am not in the WC for the 23 cents the sims said I could expect ;-) |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Adanac on Dec 15th, 2005, 7:18am on 12/15/05 at 06:20:24, Ryan_Cable wrote:
The current prediction format is fantastic and I completely agree with Ryan. The excitement of trying to be #1 is far more intriguing than a mathematical exercise in trying to maximize one's slice of the pie. I agree that it would be ideal if everyone had an incentive to play until the very end (and consistent with the Arimaa philosophy of never quit, scratch & claw 'til the bitter end), but I still prefer the current system. Incidentally, I think that we'll have lots of spectators for the finals, regardless of how many players are still active in the prediction contest. The contest really only needs to hook us into spectating during lopsided early-round games. Later in the tournament, the excitement of the games is enough incentive to watch :D |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by omar on Dec 16th, 2005, 5:38pm Sorry I didn't explain everything in my previous post. I didn't mean that the prize money would come from the registration fees. I'll continue having a prize for the best predictors as long as I can afford it. I just thought that my proposal would make the contest even more intersting regardless of how you've done so far in the contest. The registration fee would be very nominal like $5. But, I agree that since there is no need for a commitment, the registration fee should be optional. The registration fees would only be redistributed back to people who paid it and the performance would be determined limited to this set of people. So in a way it's like a second optional contest independent of the contest to be the top predictor. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by jdb on Dec 16th, 2005, 6:25pm I view the prediction contest as being something for fun. Having prize money and registration fees are not required. I also feel the current scoring method for the prediction contest is great. It provides a nice balance between agressive and conservative predictions. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Dec 17th, 2005, 12:51pm on 12/16/05 at 17:38:58, omar wrote:
I like the idea of an optional parallel contest, where the money of everyone who is involved would be redistributed in proportion to final point total. The current format where only the top three pay out tips the balance slightly towards aggressive predicting, but a proportional payout would tip the balance slightly towards conservative predicting. I wouldn't play in the parallel game, but I would enjoy watching it unfold nonetheless. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Dec 17th, 2005, 12:53pm I see 8 predicted for Ryan and 7 predicted for Paul. Among the eight or so people still in contention, the prediction contest will be won by whoever guesses right in tossups like this and the games to come. Just being right will overwhelm the difference between predicting 55% or 65% when it is close. (I assume that anyone betting 100% at this point is already way behind and shooting for a miracle comeback.) |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Dec 18th, 2005, 4:55pm Well, I said that this would be decided by whoever guesses right, but 99of9 passed me up for third place even though we were each right three times and wrong once. He must have been a little more aggressive on the favorites, or a little more accurate on the moves. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Dec 18th, 2005, 8:16pm on 12/18/05 at 16:55:37, Fritzlein wrote:
I did pretty badly on the moves. My main aggressiveness was on PMertens (70%), because although I'm his achilles heel, I know that deep down he's actually very good! We are over a full game behind adanac now, with roughly 5 games left (although I agree a lot of those will be close). I'm still not betting 100%, but I'm considering it. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Dec 18th, 2005, 9:39pm I remembered belatedly that 99of9 also gained 21 points on me by winning his game 21 moves faster than I won mine! I'm going to have to rev up my playing style a bit if I'm going to creep back into third place. That, or else I'll have to lose this round so I'm no longer forbidden to predict the moves in my own games. ;) Adanac could actually fall out of the money entirely, with four or five tossups remaining, but if people are betting heavily on me in an effort to make up ground, and he beats me, then he'll have it locked up for sure. JDB may be 72 points behind, but being unconstrained in the betting makes his chances of winning the contest at least as good as Adanac's. If JDB bets 100% on me, then his probability of taking the lead is as high as my true probability of winning. (Which I'm hoping is at least 50%, but you never know.) |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Adanac on Dec 19th, 2005, 9:52am on 12/18/05 at 21:39:45, Fritzlein wrote:
If you alter your playing style to win the prediction contest rather than the tournament, I'll be very amused indeed :) You can attack with E+M on the left side, and I'll do likewise on the right and the winner will get 75 move points! |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Adanac on Dec 19th, 2005, 12:58pm on 12/16/05 at 17:38:58, omar wrote:
That's a good idea for keeping people interested until the end. You could even simultaneously save money and keep more people interested by awarding small weekly prizes. For example, reduce the top 3 overall prizes to $20, $10 & $5 plus a $5 weekly prize for the top score. It's less money from you, with greater or equal excitement for everyone. It accomplishes the same goals as your propsoal since players in contention for 1st place must consider the alternate prizes as well. Multi-levelled strategizing - like the game itself. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Dec 19th, 2005, 8:33pm A weekly prize would be a bit silly when there is only one game per week at the end of the tournament. It would remove the incentive to find a balance between aggressive and conservative betting, because only first wins, and minus a jillion is just as good as second place. In pursuit of a weekly prize, everyone would bet 100% on coin-flip games, probably even in rounds with two or three games left. A payout proportional to the number of points above zero (across the entire tournament) would make for a much more interesting contest IMHO, and be a more effective way of keeping people engaged until the end. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Dec 20th, 2005, 6:08am Everyone should check that their predictions are still recorded for the next round. For some reason mine were reset, even though I'd entered them a few days ago. I hope we get game times soon... and that my game is not early tomorrow morning! |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by PMertens on Dec 20th, 2005, 10:35am do not worry ... if it is early tomorrow morning we will both lose by forfeit ;-) |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Ryan_Cable on Dec 21st, 2005, 5:26am The Round 4 scores: 1 PMertens . . 607 1.591 2 Adanac . . 503 1.163 3 jdb . . 440 0.903 4 Ryan_Cable . 431 0.866 5 99of9 . . 429 0.858 6 omar . . 420 0.821 7 Belbo . . 418 0.812 8 Elmo . . 400 0.738 9 RonWeasley . 389 0.693 10 Fritzlein . 373 0.627 11 carolaina . 360 0.573 12 Paul . . 325 0.429 13 naveed . . 229 0.034 14 acheron . . 137 -0.346 15 grey_0x2A . 25 -0.807 16 robinson . . 12 -0.861 17 MrBrain . . -84 -1.256 18 Aamir . . -84 -1.256 19 fritzlforpresident -84 -1.256 20 mohabbatse . -84 -1.256 21 BlackKnight . -134 -1.463 22 nbarriga . -169 -1.607 Mean 342.5 Median 220.9 StDEv 242.6 The total scores after round 4, with ~85% of the non-forfeit games finished: 1 Adanac . . 2174 1.403 2 jdb . . 2102 1.323 3 99of9 . . 1983 1.190 4 Fritzlein . 1948 1.150 5 Paul . . 1897 1.093 6 Ryan_Cable . 1750 0.929 7 Elmo . . 1656 0.824 8 Belbo . . 1204 0.318 9 MrBrain . . 1032 0.126 10 nbarriga . . 921 0.001 11 RonWeasley . 870 -0.055 12 acheron . . 830 -0.100 13 naveed . . 818 -0.114 14 grey_0x2A . 787 -0.148 15 robinson . . 783 -0.153 16 PMertens . . 720 -0.223 17 omar . . 703 -0.242 18 BlackKnight . 470 -0.503 19 Aamir . . -74 -1.111 20 fritzlforpresident -531 -1.623 21 carolaina . -781 -1.902 22 mohabbatse . -1033 -2.184 Mean 850.0 Median 919.5 StDEv 894.0 It is impressive that Adanac and I are in the top 4 for this round despite loosing our games. Did anyone pick all 4 games besides PMertens? Counting Adanac as a favorite, the crowd predicted 1.136 upsets. Weighting down the upset predictions to remove the forced predictions, the crowd predicted 1.075 upsets. My ratings estimates predicted 0.79597 upsets. I should be back in time to watch round 6, but just in case I would appreciate if someone would post the scores after round 5 to the forum, so we keep a record of every round. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Jan 15th, 2006, 10:46am What a finish to the prediction contest! On the last game I bet moderately on Adanac to protect myself from 99of9 betting 100% on Adanac and passing me that way. When I saw that only two people had bet on Robinson, and I had inside information that Elmo was one of the two, I assumed that Robinson was the other player betting on himself, and I was therefore in good shape to hang on to third place. Imagine my shock to see the final standings, with 99of9 gaining a ton of points to pass me for third, and actually coming within 35 points of first! Apparently Robinson didn't bet at all in the final round, so it looks like I have to settle for my second 4th place in two prediction contests. I had great fun the whole way, but I think the prediction contest was clearly marred by people dropping out or betting crazily because they felt they had nothing to play for in the later rounds. Next year, instead of giving $100 to the top three spots, I suggest that the entire prize money be split proportional to score among all the participants who end up with a positive score. That's the best way to foster the "never resign" mentality. Putting any special bonus at the top fosters all-or-nothing bets, which in turn tend to eliminate people from contention and make them lose heart. |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Jan 17th, 2006, 12:32am on 01/15/06 at 10:46:25, Fritzlein wrote:
You are still to be congratulated - you were a very strong adversary, and forced me to make the gamble first (I bet 90% in case you gambled 100% :-) and we both lost). Quote:
I think special bonuses are ok, as long as those further down the table still have *some* incentive. I liked the 2005 system where there was (less) money for the top 3, but also a pool divided between the rest who scored positively. I think those that many of those who bet all or nothing were generally making a mistake. I think a better way to seek the top 3 when you are significantly behind is to bet 55% on a slight underdog. If (s)he wins, you get around +140 points relative to the leaders who predicted the favourite. 140 points is enough to really add up if you get a couple of wins. Going for 100% means you are totally out of the comp if you miss out even once! (Of course, in the final round I did exactly what I'm saying NOT to do, but it was the final round, and I could have a shot at 2nd place that way.) |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Jan 17th, 2006, 12:46am on 11/15/05 at 18:26:15, 99of9 wrote:
I can't help a bit of a gloat... you can't say I didn't give you warning about my strategy! |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by Fritzlein on Jan 17th, 2006, 11:06am on 01/17/06 at 00:46:19, 99of9 wrote:
Indeed you did give a warning. My initial bet was 65% on Robinson, trying to cover myself against any possible bet you would make on Robinson. I was 48 points ahead. Even if you bet 100% on Robinson and he wins, that nets you only 31 points. Conversely if you bet 55% on Robinson and he loses, that only nets you only 32 points on me. The only way I could lose with a 65% bet on Robinson is if you bet on Adanac. But then I considered switching to 100% on Robinson in an effort to pass Adanac. As it turns out, if I had had the guts to make this bet, I would not only have passed Adanac into second place, but even JDB for first place! I had neglected to consider that JDB might place a moderately large wager on Adanac to protect himself from Adanac, so I mistakenly completely left first place money out of my reckoning. Anyway, when I realized that I had a decent shot at second place, I psyched myself out. I thought that for me to bet on Robinson was the obvious move, and you would anticipate that I had bet on Robinson. I had entered my bet on Robinson early in the week, so I had days to imagine that you were reading my mind, and guessing exactly what I had done. I thought that you would first and foremost try to bet differently than me (on the theory that Adanac and JDB were out of your reach, so you would have no motivation to bet differently than them), and therefore you would counter my obvious Robinson bet by betting on Adanac yourself. And since I truly thought Adanac was better than 50% chance to win, I didn't want to give you better than even odds at passing me simply because I bet on the underdog. Finally, I switched to 70% on Adanac. Clearly the moral of the story is not to think too much, for fear of outsmarting myself! |
||||||||||
|
Title: Re: 2006 WC Prediction Contest Post by 99of9 on Jan 17th, 2006, 3:23pm Remember Gnobot is also the current reigning champion of Roshambo/rock-scissors-paper. I relied on its expert advice on how to outsmart you ;-). |
||||||||||
|
Arimaa Forum » Powered by YaBB 1 Gold - SP 1.3.1! YaBB © 2000-2003. All Rights Reserved. |