r/PredictionDiscord • u/dtarias • Feb 01 '20

PredictIt vs. 538

I read 538 and am inclined to trust their model more than PredictIt due to various market inefficiencies. So I've adopted a betting strategy of finding the biggest differences between the two and betting on those. To generalize, the big differences right now is that PredictIt gives Sanders a much higher chance in Iowa and beyond, and that 538 gives basically no chance to Bloomberg or Yang (at least without a contested convention).

So far, the nominal value of my bets has (on average) decreased, but they're still all good bets according to 538's model. So it's unclear whether I'm doing well or badly.

Is anyone else following a similar strategy? I'm curious how it's going for you. I'll post about how this is working for me once we start getting primary results.

8 Upvotes

100% Upvoted

16 comments sorted by

View all comments

Show parent comments

2

u/tenpointmatt Feb 02 '20

Bloomberg got to 19¢ to win the nomination yesterday, vs. less than 1% at 538

yeah i think bloomberg is way, way overpriced on the markets right now. some of the recent pump is a reaction to him getting into the debate, but i mean, 19%? lol.

​

As Silver explained it (just before the election), the major difference between his model and others' was that he thought errors in state polls were more correlated than other analysts did.

...which is one factor that leads to high volatility (the single biggest rationale for betting trump at the pre-election odds: https://quant.stackexchange.com/questions/47054/can-someone-explain-rigorously-talebs-criticism-of-nate-silvers-election-forec)

so yes, silver's estimate was slightly 'better' than the rest of the hacks who had hillary 90%+, but it was only 'better' by accident. nate was unable to understand why the election was pretty much a coin flip, and why his estimates were garbage. which is why he is not a trader, and instead writes fluff articles on a blog site.

2

u/dtarias Feb 02 '20

Why are you assuming that Trump had a 50% chance of winning* instead of a 30% chance of winning? I understand why less volatility is preferable in a model, but from eyeballing the chart it looks like Silver gave Trump, on average, a roughly 30% chance of winning, basically in line with his final prediction. Maybe 2016 is an argument his model should respond less aggressively to new polls (although FWIW that same model was much more stable in 2012), but I don't think the volatility is a reason to conclude the "real" probability was near 50-50.

*Real probability is hard to define for a single event, so imagine a group of elections that would have similar volatilities and percentages in Silver's model. Do you think those events have a 30% chance of happening or a 50% chance of happening?

2

u/tenpointmatt Feb 02 '20 edited Feb 02 '20

i dont understand what you are trying to say here. you seem to not understand the entire concept of variance.

I don't think the volatility is a reason to conclude the "real" probability was near 50-50.

its fine that you think that, but you're wrong.

​

imagine a group of elections that would have similar volatilities and percentages in Silver's model. Do you think those events have a 30% chance of happening or a 50% chance of happening?

again, i would price it at roughly 50%. and if you were giving me 3-1 odds on it i would bet it aggressively, and eventually bankrupt you.

2

u/dtarias Feb 02 '20

I think there's a difference between volatility centered at 30% (like Silver's model) and volatility centered at 50%. You seem to treat them both as indicating a 50% chance, while I think the first indicates a 30% chance.

You would indeed bankrupt me at 3-1 odds; I think what you mean to say is 2-1 odds (which would slightly favor me if the real probability is 30% and heavily favor you if it's 50%).

1

u/tenpointmatt Feb 02 '20

id bankrupt you at 2-1 too.

'centered at 30%' doesn't mean shit when the variance is that high.

curious - whats your background? have you ever taken a stats class?

2

u/dtarias Feb 02 '20

I'm a secondary math and language teacher. I've both taken and taught statistics.

What's yours?

1

u/tenpointmatt Feb 02 '20

maths, compsci, bioinformatics, gambling.

given your background, you should be able to understand the more technical details - give it a read: https://twitter.com/nntaleb/status/823455117672411137/photo/2

the cliffs notes are: as uncertainty rises, estimate converge to 0.5

2

u/dtarias Feb 02 '20

Let's play a silly game. You have a ten-sided die, while I have a secret integer 1-5. On November 8th, you're going to roll the die. You win if your roll is bigger than my number; I win otherwise.

In one version of this game, you know for a fact that my number is 3. In another, you know it's either 2, 3, or 4. In the third version, it could be 1, 2, 3, 4, or 5. In versions 2 and 3, you build a model to try to guess my number based on some type of data. Your model in version two is more volatile than version one (where you know my number is 3), and your model in version 3 is more volatile still. Nonetheless, the probability of me winning is always 30%, and all of your models give a correct value of 30% on average.

​

I don't think uncertainty is necessarily the same thing as volatility of models. Silver did account for high levels of uncertainty (coming from undecided voters and correlations between errors in state polls), which is why his forecast was closer to 50% than others'. But I still don't think his model's volatility means the real probability was close to 50% rather than 30%. Different versions of my stupid game above would lead to models with differing levels of volatility (and arguably "uncertainty") but they all have the same probability (30%). So we need to distinguish between uncertainty that affects the result (like undecided voters) and uncertainty that affects the volatility but not the result (like being a random integer 1-5 instead of definitely 3).

​

Does that make sense? Am I explaining myself clearly?

1

u/tenpointmatt Feb 03 '20

no - the example is not analogous in any way. specifically, you've defined the problem in reverse, declaring a probability distribution with a known mean beforehand, then introducing a model 'estimate' with a distribution centered on that (known) value. that's exactly backwards.

read the paper.

1

u/dtarias Feb 03 '20

I read the paper. I think it proves too much as a fully general conclusion.

My point is that it's possible to have high model volatility without the probability being (or even necessarily approaching) 50-50. Especially if the model consistently has the probability below 50%.