538 was one of the least wrong predictions in the 2016 elections, to the point that Nate Silver was getting shit from all over the place for not giving Clinton better odds.

The inherent variance in elections is extremely minimal. The odds in elections are more representative of confidence in your sampling than a product of randomness. What I mean here is that if you polled each and every person who would vote, the result of that would tell you who is going to win with a practically negligible level of variance. By contrast, even the best model for something like a sports game is going to be probabilistic.

Imagine a thought experiment. We're able to do something 100 times, each on a consecutive day, but without any knowledge of what happened on previous days affecting the future, and without any future information or events affecting original 'intent'. If you played a sports game 100 times you'd generally get a wide distribution of results. But if you ran the election 100 times, you're generally going to get the same result.

The point here is that Trump winning did not indicate a 'low probability' event. It indicated a dramatic failing in sampling. Why was this? The 'art' in sampling is not only obtaining a representative sample, but actually determining what a representative sample even is. And the radicalization of our society is making this more and more difficult. The media was running bits literally comparing Trump to Hitler, individuals have been assaulted, robbed, and denied service simply because somebody became aware they supported Trump, and even business leaders have been harassed to no end for doing things that could be seen as supportive of Trump. Like you mention even pollsters that didn't cheerlead a completely crushing Clinton victory "got shit" -- that's all just completely absurd! This creates a chilling effect where individuals are going to be less inclined to respond, and/or respond honestly, when queried about their views. Then pollsters are left to rely on incorrect information which hampers not only sampling but even in the determination of a representative sample.

And the point of all of this now is that presumably most pollsters have tried to change their ways. But this will be the first election where that will be put to the test. So unbiased aggregation is more informative than ever. 538's "adjustments" were based on past polls before we had such social extremism, and by relying on them alone for their first real test - well, that's a huge leap of faith.

Polls would still have non-negligible variance in the scenario you suggest because people can change their minds and polls can't be conducted instantaneously. The director of the FBI publicly stating that one candidate is under investigation will probably change the outcome, but if it happens 12 hours before the election no poll on earth will be able to catch it.

You also seem to be under the impression that the polls were drastically off in 2016. They weren't: https://fivethirtyeight.com/features/the-polls-are-all-right National polls in 2016 were better than average, and while state polls were worse-than-average it still wasn't the worst error in the last 30 years.

The problem was never really with the polls (though there are problems there, most notably herding). The problem was with the narrative around the election which cited polls as evidence that Clinton was a sure bet even though the polls didn't really say that.

Of course, if you have evidence that the polls were more wrong than this I'd like to hear it.

That's pretty disappointing. Nate is clearly a smart guy who I ha.. a decent degree of respect for, but he's being quite disingenuous there. I predict a crushing win in a football game. I get a crushing loss. I respond, 'Well I think I was about as right as usual since I predicted just about the right number of shots on goal.' Presidential elections are decided exclusively by the electoral college, which is what everybody bases their predictions on. As everybody got this unbelievably wrong, he's instead looking at the popular vote and arguing that 'Well we were pretty much right there, so there was no problem.'

Everybody was predicting a crushing electoral victory for Clinton. Instead she suffered a crushing electoral defeat. Look at 538's own forecasts [1]. You'll note he put the odds of Clinton Winning the popular vote but losing the electoral college at just 10.5% when not only is that what happened, but it happened by a wide margin with Trump's electoral victory margin being even greater than was predicted for Clinton's.

As an aside, even the popular vote aggregates are somewhat disingenuous since they're heavily based on major polling screw ups. For instance Clinton's margin of victory in the popular vote nation wide is smaller than her margin of victory of victory in California alone. 538 was off by more than a million votes there, which they're now using to try to spin into defending the aggregate numbers (which are not even what the election was predicted on to begin with.) This is just really disingenuous stuff.

[1] - https://projects.fivethirtyeight.com/2016-election-forecast/

Your explanation doesn't really explain much. These polls have generally been accurate in the past, but they were very wrong here. Was that because the "art" of finding a representative sample got worse, it was never good to begin with, or something in society changed making this much more difficult?

Like I said, I think it was the change in society. We've become aggressively intolerant which is manifesting in various ways. Do you recall during the primaries when Trump supporters were literally being assaulted and hounded, just for being Trump supporters? Or how the media was running, non ironically, pieces literally comparing Trump to Hitler. The sort of stuff you might expect from internet trolls, not actual reporters.

Compare this to, for instance, how the media and society society responded to e.g. Romney, McCain, or Bush. Society has shifted sharply towards intolerance which is making obtaining accurate information very difficult when that information might leave one within the crosshairs of said intolerance. This creates a major chilling effect which leads people to misrepresent their views. Take the state of society today and then imagine somebody calls you and asks if you're voting and if so, for who. You're going to get some decent chunk of Trump supporters that will lie here. Voting is anonymous, but phone calls are not.

The burden of somehow trying to overcome this problem is on pollsters, and the only thing that's certain is that it's not going to be easy. Gallup was way ahead of the game here -- deciding for the first time ever to not make predictions about the 2016 election.