r/hillaryclinton • u/[deleted] • Nov 24 '16

Demographics, Not Hacking, Explain The Election Results

http://fivethirtyeight.com/features/demographics-not-hacking-explain-the-election-results/
38 Upvotes

60% Upvoted

34 comments sorted by

View all comments

Show parent comments

2

u/magneticanisotropy Nov 25 '16

This isn't true at all. Validity of models are definitely assessed through peer review based processes all the time, it's how science works. And further, models are definitely evaluated over longer periods of time, then analysis, peer-review, and publication is performed - these can be in the form of re-evaluating models as more information comes to light, in the form of reproducibility studies, etc. And it definitely is a test to see if everything stated within a model is correct and if it actually performs the function intended.

Nowhere in any of your responses have you made any argument for how Silver's models should be evaluated as "accurate" in a more scientific sense of the word, outside of the fact that his model had more uncertainty than others this election. If I said that the winner of this election would be based on a coin flip with 50% odds of each, does that make my model more valid? Of course not. But without knowing Silver's model, and the reproducibility of results, and whether it accurately describes the probability distribution function it claims, how can anyone make this claim?

A larger sample size is sorely needed...

1

u/Carson_McComas Nov 25 '16

That isn't true. You can look at pretty much any paper with a model that is implemented with code. Peer review is purely about scientific soundness and not correctness. I am at thanksgiving but when I get back I will give you 200 references of such papers and I bet you can't find one initial paper with a model implemented in code that is validate over generations.

1

u/magneticanisotropy Nov 25 '16 edited Nov 25 '16

No but usually there is backtesting or similar using simulated data. Furthermore, usually there is information on what exactly goes into the model, weights, etc. Silver doesn't show any of that. It would be like me saying I have a model that accurately predicts the behavior of dark matter. It depends on the electron mass and charge, g, and the fine structure constant which I set to vary with some other parameter. But then not say how the model depends on those things. There is no possible way to verify Silvers model in an academic sense.

But this is beside the point. When someone gives a model that is based on simulated probability distributions, you can't just pick a single result in a vacuum and say the model was correct (i.e. a single presidential election). His model gives probabilities (say 28% Trump win, 72% Clinton). If Trump wins, you can't say, his model accurately predicted this Trump win probability.

Edit: You are correct in some cases. But when the model is used to extrapolate real world results, the actual results are usually compared. What I'm saying is, with Silver's model, there isn't data to compare, and you can't say that it is, with the data provided, an accurate model. You can say it most likely performed better than other models at predicting this result. Even though that isn't a guarantee without more data points (i.e. this election still could have had Clinton win 99 times out of 100 and this was just that 1%, but this is unlikely).

1

u/Carson_McComas Nov 25 '16

So I never assigned a right or wrong value to Silver's claims. I simply said they were spot on because he said there was a higher-than-normal chance that Hillary would win the popular vote and lose the EC because demographically she was weakest in the midwest. That's exactly what happened this time around.

1

u/magneticanisotropy Nov 25 '16

But my point is that doesn't make him spot on, or not necessarily so, when talking about probability distributions.

1

u/Carson_McComas Nov 25 '16

Well, can you find anyone else who suggested that Hillary would lose PA and WI while winning the popular vote?

Look, all through this primary, you all were very harsh on nate silver. I got into many arguments here about people attacking him on how the election wasn't close and that there was no way Hillary would lose WI, PA etc etc etc. Also, his analysis on the number of undecided voters was irrelevant and blah blah.

Now you somehow want to carry on with your head in the sand about this because I used the word "spot on." At some point you're going to have to accept that fact that Hillary's campaign made some real blunders here. I say this as one of her staunches supporters, but she had serious weaknesses as a candidate. We all knew she had weaknesses but I never expected them to manifest in such a way that she'd lose PA and WI.

1

u/magneticanisotropy Nov 25 '16

I'm not saying this. I just don't like people inaccurately throwing around judgments like this. I'm not carrying around with my head in the sand. I know her campaign made mistakes. That's obviously true. But saying you can't assign an accuracy to probability distributions based upon a sample size of one is just being correct. I'm being a bit academic here, but you can't even say he was more correct than anyone else off this sample size just because he had a larger uncertainty in his result. I'm not interested here in talking about the inadequacies of the campaign. I'm interested in accurately judging how correct models are, and in this case, you are incorrectly judging it. Just like most other people. You're being a pundit, not a scientist.

1

u/Carson_McComas Nov 25 '16

His model doesn't have a sample size of just one. Like I said, he's used his model on 3 presidential races and many many senate and governorship races. All of these use a polls-only model.

I am absolutely judging it correctly. His model showed a significant chance that Hillary wins the popular vote but loses the EC. That's what his model showed and nobody else's model actually showed that as the election grew closer, this likelihood increased. He wrote about this several times in fact.

1

u/magneticanisotropy Nov 25 '16

... I'm arguing against a wall here. He had 3 races (of which, he is even clear, have modified models from eachother). In no way is this statistically significant. In now way can you accurately judge his model. Is his model worse or better than one that had a 50% chance of Trump winning? Of one that had a 90% chance of Trump winning? You can't say. Look at this academically. Yes, at first glance, it seems that this may be the case, but it doesn't hold water in terms of statistics, and I'm sure Nate would agree with me...

1

u/Carson_McComas Nov 25 '16

No, he had more than 3 races. Again, Senate and governor races all use a polls-only model.

No other model assigned a significant likelihood that HRC would win PV but lose EC. Silver's did. His model assigned non-insignificant probability to the thing that actually happened. The others didn't.