Hi there, argmin readers! Today’s post is a live blog of Class 15 of my graduate seminar “Forecasting: A Critical Retrospective.” In this class we discussed the purpose of election forecasting. The syllabus and list of past posts are here.
We’re a week out from election day, and we all know the Democrats are going to win a bunch of seats. The data team at The Economist has run 25,001 simulations of every House race and aggregated the results into this convenient histogram predicting a 95% chance that the Democrats win the House of Representatives.
They give only a meager 56% chance for the D’s to take the Senate. You can all place your bets now.
The numbers here are based on “national and race-level polls from FiftyPlusOne with fundamental data about past electoral results and the nominees in each contest.” Everything is neatly quantified into some code base to generate bell curves. Next Wednesday, they’ll release a model for 2028.
I’ve spent a lot of time on this blog and elsewhere flummoxed by the influence of this sort of forecasting, and I didn’t leave our class yesterday with a satisfactory explanation. Why do we need to know the outcome of an election before it happens?
We all know this deep down, but the answer seems to primarily be entertainment. Or at least, news posing as entertainment. In the US, polls in news reporting predate their use by campaign strategists by well over half a century. They have always been a fun way to inform readers of the dynamics of our endless presidential campaigns. Since elections are always happening, we comfort ourselves with forecasts that suggest these campaigns matter and are worth the vast amounts of money thrown at them. And since campaigns start the day after election day, polls provide a funnel of endless news content.
Moreover, polls give us an easy way to view politics as sports (a horse race), and sports get eyeballs more easily than policy briefs. Who has momentum? Whose gaffe has tanked their chances? You could even argue that turning elections into a sporting event inspires public engagement. In class, someone recounted an anecdote about only thinking Mamdani had a chance to be NYC mayor once reputable polls measured his growing support.
But there’s no real “score” in elections. It doesn’t matter if you poll well in June if you lose in November. Why do we need to pretend to measure the public’s opinion of how they’d act if an imaginary election were held today? Polls are a beautiful example of the quantification trap, where the numbers make it feel like reporters are being objective when reporting the “score” of the game. This makes them appear more objective than going around and talking to individual people at campaign events or providing other qualitative analyses. The media thinks the public wants a balanced calling of balls and strikes. If you’re just reporting other people’s opinions, you can pretend that you don’t have one.
For that quantification to be trustworthy, the public has to believe the horse-race polls are giving “accurate” scores. If people don’t trust the polls, they’ll stop reading them. A good polling outfit confers institutional prestige on the publication. So you’d better hire good pollsters and poll reporters. This is true in campaigns too: relying on polls keeps pollsters employed, and that gives campaigns one more useless way to spend down their unreasonably massive war chests.
Though forecasters want to be scored using ornately scientific proper scoring rules, they are scored on their most recent track record. They get their prestige through the same path as cephalopods choosing FIFA winners. The history of polling is littered with falls from grace. Literary Digest had decades of strong forecasts with their straw polls until 1936, when Gallup showed them up with modern random sampling techniques. Those sampling techniques whiffed, claiming Dewey would beat Truman. John Zogby was the king of polling in the 1990s, but the industry piled on him when he started missing wildly after switching from random sampling to “interactive polling.” Nate Silver was a media darling after Obama, but his forecasts started crapping out with the election of Trump.
And even if you wanted to properly score presidential election forecasting, it’s a fool’s errand. As Grimmer, Knox, and Westwood have pointed out, there just isn’t enough data to decide who is “best.” The evaluation of pollsters has to be based on calling winners and losers. They have to be judged with recency bias. Polls are infotainment, so they are better evaluated as media properties than as scientific instruments.
Nate Silver thought he had an out by claiming he’s just a humble handicapper. He’s forthright in telling you that he thinks his purpose is helping people bet on election outcomes. And since he said Trump had a chance, you could have made money if you read his reporting. This argument is some civically destructive nonsense, but fortunately everyone who isn’t a degenerate gambler is disgusted when Nate talks that way.
Maybe it’s just me, but I think we’re well past the heyday of the election forecaster. There are still tons of websites with poll aggregates, but these numbers seem to matter less than they used to. People are used to the polls missing bigly. Scammers are putting out fake polls for lols. There has been no successor to Nate Silver because we haven’t figured out what to do next. In class on Thursday, we’re going to talk about silicon sampling. Let’s hope that’s not it.
Beyond mere entertainment, polls sometimes function as (price) signals that shape the outcome. Common knowledge (even if logically ill-founded) that "candidate A is the electable one" sometimes influences strategic voting.
And yet it seems possible that Janet Mills was driven out of the Maine senate race by fake polls that caused volunteers and donors to drop her