If you look at midterm election polls right now, they are telling one consistent story across the board. Democrats look like they are on track to win and possibly win big. Right now, most surveys show that Democrats are likely to retake the House of Representatives, potentially retake it by significant numbers. And they're competing in states that Democrats have typically lost in the last few elections. States like Texas, states like Iowa, states even like Kansas. But despite these indicators, many people from both parties just don't believe it. Democrats are scarred by years, several elections of Donald Trump overperforming his poll numbers, a factor that many Republicans are counting on to repeat itself in this year's midterm elections. Quality polling has also gotten a lot harder and fake polls continue to sneak into the mainstream discourse and political narratives. So, at a moment when people are really starting to tune into this election and wondering what the hell is going on, what is going to happen, we wanted to talk to the person who understands the numbers better than pretty much anyone else. Nate Silver is America's preeminent election forecaster, one of the best known statisticians in American politics. He's the author and creator of Silver Bulletin. He's the co-host of Still Counting, the podcast, the new podcast, the new old podcast from Crooked Media. And of course, he created 538 and famously predicted the outcome of all 50 states in the 2012 election. We'll actually get into that in the episode. So this week, Ben and I sat down with Nate to figure out just what we should be thinking about the polling and why some forecasters, pollsters, maybe even members of the media may be hedging their bets when it comes to how Democrats are going to perform and potentially the margins of their victory. If they don't win, we'll look pretty stupid. But you know what? That's what we're willing to do here at this podcast. We also wanted to talk to Nate about prediction markets. It's a thing that he knows quite well, a subject he knows pretty well. He's been working with Polymarket for a while and when he believes that the prediction markets are actually better and ahead of the media and why they are causing this explosion in fake polling. It was a really interesting conversation and a great primer for the midterm election. So after the break, Nate Silver.
So Nate, right now the polls seem like they're telling a pretty clear picture of what's going on, the direction of things that we might be headed towards, you know, potential blue wave territory, major Democratic pickups both in the House and potentially in the Senate. Yet when you talk to a lot of normal people, you know, about this type of stuff, I think that there's still a lot of doubt. There's a lot of people who don't believe the polls, who think that they're going to, you know, miss Republican support. Again, I'm curious, are people right to doubt the polls?
Well, I don't think that's a normal person thing. I mean, I'm not sure if normal people are looking at Silver Bulletin and Real Clear Politics and sites like that. I think it's a... I mean, I can get psychological if you want, right? Let me start with the cynical interpretation, right? I think it kind of gets a little bit at people being terrified that they don't understand what's happening in the real America, right? And that fear kind of will maybe never really be resolved after 2016. It's just a permanent scar in everybody's mental map. I think, you know, people should always be concerned about the possibility of a systematic polling error, meaning an error where the polls miss in the same direction in every state or in most of the important states, right? What I would contend is that it's hard to predict whether that will be a D-leaning or an R-leaning error, and there are examples periodically of both. Obviously, the very salient Republican example is in 2016, 2020, to a lesser but still meaningful extent in 2024, the polls underestimated Trump and likewise down-ballot Republican candidates, right? They did not underestimate Republicans or overestimate them in 2018 and 2022. So my mind would go and say, okay, well, if you want to use small sample sizes, you should probably use the sample size pertaining to the election type that we're in right now, which is a midterm. There have also been elections when polls underrated Democrats. So in 2012, which was the year that I guess famously 538 called all 50 states, right? Well, actually Obama beat his polls by about three points. It's just kind of like it was convenient that there were a bunch of Obama plus one states that became Obama plus four. And then in New Jersey and Virginia in 2025, so the most recent general elections in New Jersey, Democrats beat their polls by like nine points, in Virginia by five points. And so the question is always like, are pollsters and as a kind of broader forecasting election analysis community overcompensating for 2016 or undercompensating? Because it certainly is true that it is easier to get college-educated voters on the phone. They tend to vote Democratic, right? You can try to correct for that by weighting your samples differently, by weighting to past vote share and things like that. It's also true that whichever party is more enthusiastic, their people are more likely to answer phone calls, but that's a mixed bag because in some sense, taking a poll is a proxy for voting, right? You know, if Republican respondents aren't responding, then are they going to vote? Because it's not Trump on the ballot. It's Roger Marshall and Ken Paxton and Darlene Graham and all these frankly mediocre, uninspiring Republican candidates who don't have any kind of proof of concept in terms of people walking over glass to vote for them like they do for Trump. That's kind of the complicated answer, I guess.
And I think different groups of Americans have different sets of doubts about the polls. Democrats, I think, have one theory, Republicans another, maybe independents another. Who do you think is most sort of unhinged in this regard as a poll believer?
Basically, the week before the election, I think everyone gets a little bit unhinged. I mean, I'd actually say the discourse has been like slightly more hinged this year, in part because like, I don't know, I think Republicans are perceiving that Trump is becoming potentially toxic, right? So you can say, well, ignore the polls, look for kind of signs, data-driven signs in the field, and like one sign is like Trump is not mentioned very often in Republican-produced ads, right? If Republicans thought that, well, Trump might be 37% approval in the Silver Bulletin average, but really he's 49, then you'd have Trump featured a lot more. But even in red states and districts, Republicans are trying to obscure their partisan orientation, talking about how Democrats are too woke and will let too many immigrants and let crime run amok and trans kids and all that stuff, right? But they're not really running on President Trump or his record.
Is there anything though that, I mean, when you sort of hear the don't trust the polls conversation, which has been, you know, obviously going on a long time, is there anything in particular that drives you crazy? Are there any real fallacies that you hear again and again?
Well, I get annoyed more when people talk about models as opposed to polls. So I think, and you're welcome to aggregate this out, right? I think 538 made maybe the best forecast we ever made in 2016, which is the year that we gave Trump a 30% chance, and 29% technically, when prediction markets had a much lower and the conventional wisdom has had much lower. And like Matthew Dow, my former ABC colleague, went on TV and was like, he has at best a 5% chance, right? It's ridiculous that people are making a mountain out of a molehill and trying to get clicks on, you know, and it's like, so, you know, I always felt like... And that election was annoying because we kind of went out of our way to point out that especially after the Comey stuff, that like Clinton's not all that far ahead here, these leads in the Midwest are pretty tenuous, and because polling errors are correlated, if she loses Michigan, she probably loses Pennsylvania. And so like, yeah, 2016, you know, I mean, I talked before about how people I think were traumatized by it, but like I, you know, I have my own unique form of that, I suppose.
But basically, your trauma is that you were right.
I think like a gambler. If you had looked at our forecast, you would have bet on Trump at six to one odds and made a shitload of money.
Yeah. Do you bet now on elections?
Not in politics. I bet on sports. I do bet on sports because, I mean, in politics I'd be worried that I would like influence the prediction markets, which we already do. But like, in sports, I do bet because I think it's a way to validate and verify our models, right? And sports are so liquid that like I don't think there's any potential for us to move the markets really. But yeah, no, I think, you know, look, I have various critiques about prediction markets. I do advise Polymarket on the side, but I think generally it's honorable to be willing to like actually invest something in a forecast because it's cheap to just kind of use rhetoric and then not be held accountable for it, right? Yeah.
Your reputation isn't enough.
In politics, it probably is. Yeah, but look, I do have pretty high, you know, I'm sure, I don't know how you calculate it, but like we're a subscription-based business and like, you know, after an election there's churn no matter what. But yeah, I think if we have a good night, then we'll have less churn than if we have a bad night. And so I, you know, I have more on the line here probably than most people do.
So one of the main ways that people engage with your content is through the Silver Bulletin model, Flipper. You have Flipper Premium. I guess you have multiple different versions depending on what your level of insanity is and how deep you want to go here. But one thing that I was particularly interested in is how the model this time was different than in previous years. Can you explain how you changed your model from this year based on some of the elections that you, I guess 2024, but also I would imagine you were thinking about some of the off-year cycles as well.
We haven't changed it that much. And I think actually it's a problem if people are always fighting the last war, right? So we have like an NFL model, right? And the NFL you have 32 teams times 17 games, so hundreds of games per year that are all kind of these independent events. And so yeah, every year you'd say it looks like we were somehow having too many touchdowns in the model, not enough field goals, or why are we so different from Vegas in this type of game, right? For elections precisely because polling errors are correlated. It's all one big election mostly, right? You only get like one data point every two years and one probably shouldn't change things all that much based on that. I mean, look, the thing that we have done over time is account for how politics and elections are changing as like partisanship grows ever greater, right? So a lot of the model is dynamic in the sense that like a lot of the weights for different variables are keyed off a partisanship parameter basically. So one example is like in modern elections you don't tend to have very many October surprises. The polls are actually quite steady through the entire stretch run. We did have a tick upward for Democrats as of Labor Day, but it's been kind of steady since. Right. And so like, you know, the model of oh everyone is a swing voter and they don't start paying attention till the kids start going back to school, well that's not really true anymore, right? People who vote midterms are, you know, either don't have a decision to make because they always vote for one party or they are deeply invested in politics the whole time. So like that's an example of a change, right? And then, you know, obviously on the one hand, polls have had a couple of rough years, although if you look at the broader history it's actually kind of always been like this. I think people were spoiled a little bit by some elections like 2008, but now things are more predictable based on presidential voting, right? So if you just say, so right now the generic ballot, which if your listeners are not political nerds, right, it's like national polls for the House basically, right? Those show Democrats ahead by nine points. They lost in 2024 by two points, right? So it's showing an 11-point swing. And what's interesting is that a lot of these states, Ohio, Iowa, Texas, Alaska, Florida for that matter, are states that like Trump won by between like 11 and 14 points, right? So they're right in that zone where if you have a swing nationally of 11 points and they all become competitive and then it kind of comes down to individual candidates. Does Talarico, he's a better candidate than Paxton, is he enough to overcome an R+11 state? Ordinarily no. But if you have an 11-point tailwind at your back then that gets much closer potentially. So, you know, so that's the saving grace is that because things are so partisan it creates some predictability in a world where things are otherwise rather unpredictable.
You talked a little about the prediction markets. I just wanted to go into that a bit more because you wrote in your most recent Silver Bulletin about the gaps between Flipper and the markets, and I think like this is for a lot of consumers and for a lot of the media kind of a new frontier, and a lot of media has commercial relationships with Kalshi, Polymarket. You advise Polymarket. We don't, but you know, could happen, who knows? And I'm curious kind of how we in the media, I suppose, first should evaluate that kind of information. Like how good is that information? Is that just basically somebody dipping a thermometer into Twitter? Is it somebody essentially taking Silver Bulletin and like adding a bit of noise and emotion? Or is it something, or is it some, as the markets people would have it, like the ultimate source of truth?
It's certainly not the ultimate source of truth. And you know, I've said this publicly, you know, I wish that prediction markets wouldn't use that branding. I think they're setting themselves up to be very like punchable when they do that. And like, particularly because like, look, the best use case of prediction markets I think is actually like on election nights, right? Or on nights when news happens because they hone in on the correct outcome most of the time like way faster than the decision desks do and things like that, right? And so the whole BS pretense of like, look, by was it 2016, it was clear by 3:00 a.m. that Trump was going to win, right? And like, but if you're watching CNN or ABC News or CBS then that might not have been as transparent as it could have been, right? So they're very good when they aggregate information from different people with different kind of layers of expertise, but it isn't, you know, prediction markets do not have and should not have like a mind of their own, right? They're aggregations, kind of weighted aggregations of people who are willing to bet money on the outcome for various reasons. Some of that might be because they're hobbyists with strong opinions, some of that might be because they are, you know, hedge funds in Dublin, and then there's kind of everything in between. You know, when the Silver Bulletin forecast came out this year, they did move the markets a little bit, which suggests that like either people believe we're wise or they believe that other people will calibrate to us in some way. But like, but what I don't want, I don't like when like prediction markets are a summary of opinion and information, and as a quantitative guy I like a summary that you can put into a percentage or probability. I don't know that they add that much information on their own, and so like it's redundant sometimes when you see some segments saying, well, Democrats are ahead by three points in Texas and Talarico is at 70% in prediction markets. That's kind of saying the same thing more or less, right? So, so yeah, I, you know, again, I'm interested in like the why questions, right? If you see a number at Flipper saying Talarico at 70% or whatever, I can explain how that number is derived and how it relates to the forecast in other states. And so it's even though people accuse like the models of being a black box, it's less of a black box. But you know, at the same time, you know, I do kind of agree with the premise that hey, if you think these prediction markets are so beatable, then maybe you're not in journalism, but maybe go and trade yourself, right? Certainly the experience of like sports betting is that edges are pretty small and there's only a small fraction of people that can beat Vegas.
One more question before we move on from that, which is, you know, when I first knew you, you were like a blogger on Daily Kos, contributing quite a bit to people's understanding, but you were not the Fed chairman, or you were not in possession of secret market-moving information which when it got out would shift people's perceptions and lose people money. And now you're in the situation where, I mean you, but also your copy editor and also the guy sitting behind you in the coffee shop could make real money betting front-running your forecasts. Has that changed how you operate? I mean, how do you prevent that from happening if you don't want it to happen?
Look, I'm not the one publishing these prediction markets. I guess I do consult for Polymarket. No, it hasn't changed.
But no, but what I mean is that like a Silver Bulletin, as you said, like a new Silver Bulletin send is going to move the prediction markets. So like if I could get that five minutes before, I would front-run it. And I'm just curious, like you're not the only one. I think a lot of newsrooms now actually, like there's this new thing that a story can, you can make money off of journalism, which what you do is journalism, and like have you thought about, like do you have to protect more closely your own analysis because it's market-moving?
I mean, I'd say at the margin it makes me like less inclined to do anything to alter the model once the model is released. Like we never have done much of that anyway. You kind of live with the choices you made. And the nice thing about a model is that whatever we'll refine it in the spring or the summer and then you just kind of press a button. I mean it's more complicated than that, right? But you collect the polls, you collect the latest fundraising information and the other inputs that we use, right? It's all objective data, right? And then press a button and 12 minutes later it spits out 40,000 simulations, right? And so like, you know, if it were like Nate's private stash of opinion, then that would worry me more. But this is a problem I think for the entire industry. Like something like, there are markets on who will be the first NFL draft pick, right? NFL insiders would make enormous amounts of profits by knowing that, right? I think there has to be some threshold of like, and I don't think election forecasts are in this category, right? I think there has to be some threshold of like a market where almost all the edge comes from insider info, like markets like what will the host say on this TV show tonight? Like what were words we mentioned on Saturday Night Live or whatever. Like I don't think those markets should exist, and I think that like people who are trading on them, you know, I mean I think in the long run they're not beatable unless you have inside info because that info will leak otherwise. So yeah, I'd like to see prediction markets pare down the number of offerings they make. But at the same time, I mean they're fascinating instruments, you know what I mean? Sometimes I try to figure out why did the market move this or that. Sometimes it's related to like weird Elon stuff on X or whatever, but it's this weird animal which is kind of like a hybrid between the conventional wisdom and a model. I'm not sure if it's adding, if there's any kind of like secret knowledge on top of that, right? But yeah, if you kind of like, it's a way of quantifying the vibes.
Yeah. So one of the new, and just, and also, you know, to your earlier point, I mean we've already seen, you know, the abuse of some of these types of, you know, bet on the first word that Trump is going to say in the speech literally with the guy operating the teleprompter, right? He was indicted, Trump's teleprompter, he was indicted. One other thing that we've seen a lot of discussion about this election cycle is, you know, with regards to the polling, is the seeming rise of fake polls, these viral polls. It seems like X is supercharging these, you know, these types of things, and you know, we want to get to some specific serious ones in a bit, but I'm curious, do you, have we seen a rise in fake polls? Is this a new thing? And why, if so, why is that the case?
So have we seen a rise? It's not as straightforward an answer as you would think. So I busted a fake pollster called Strategic Vision back in 2009. Then another one called Research 2000, which had done polling for Daily Kos as well as like a lot of daily newspapers, was proven to be fake. And like overall we have in our historical database of whatever 400, 500 pollsters, we have 11 pollsters that we have banned, in not all cases because they fake polls. There are sometimes other ethical things, right? But like, but let's say we've caught 10 fakes out of 500, right? Well, there are probably more than that. You know what I mean? You know, if you see kind of one ant in the kitchen, there are probably more, that's probably not the only ant in the kitchen, and certainly if you see two. But I do think there are a couple of things that make the environment more challenging now. One is that you do have this new motive created by wanting to potentially manipulate prediction markets. Two is that like it used to be that polling was done by telephone, and telephone polling is expensive. And so if someone purported to do 20 surveys every week by telephone interviewing 800 respondents each, like well that is a budget that would cost hundreds of thousands of dollars per week, right? And where are you getting that money? Now like the barrier to entry is lower and you can have like college kids use like Pollfish or some third-party service to cheaply survey respondents, and so you know, so a, that might make the quote-unquote real polls worse, but also it kind of, because it's so easy to do a real cheap poll, then it kind of makes the faking more plausible because it's hard to know the difference potentially. You know, it's not true though that like the industry is doing less vetting. I mean, probably there's more vetting now. We take the feed from The New York Times. I know those folks over there, and like a lot of polls that surface on Twitter do not make it in The New York Times feed because they are on the phone with people and emailing people and using their skill as actual journalists. But like, but you know, a clever cheater will lie to a reporter or an aggregator too. And like this is not like a national security clearance, right? There has to be some type of balance between, you know, I liken it to like airport security, right? You can't just kind of walk on through to gate B26, but at the same time like you can't have like a 10-hour line at Newark either, and commerce has to get underway, right?
Why do you take the feed from The New York Times rather than kind of, you know, vetting on your own?
Because it saves me a lot of work and they do a really good job and I've worked with the Times before, and they kind of use the same, they kind of inherited the schema that 538 I helped design. So it's almost like they are doing the work in the way that I would recommend. But yeah, look, I mean, if we had to hire...
That's pretty like, oh, I've outsourced the technical work to The New York Times. Ben, you know, I'm an independent content creator. I'm an independent content creator. I don't think it's...
So the fact of the matter is too that like by design, our models don't assign a lot of weight to unknown pollsters. They also detect what we call house effects. So if someone comes out and they say, we're going to just spam with lots of D-leaning polls that are four points more Democratic than the consensus, then we'll detect that and kind of pull three and a half of those four points back out basically. And so yeah, designing a model is partly defensive, right? You're trying not to have the model be undermined by outlier data or by bad data.
Do you think people in general, like the conversation, the gamblers are more suckers for these fake polls than they used to be? Do you think they're easier to pull off?
No, I think gamblers are generally much better at spotting fakes than the rest of the population.
I think journalists are good too. The journalistic skill set and the gambler skill set are maybe not as divorced as one might think, right? They're very detail-oriented businesses and kind of involve, you know, in gambling rather, you kind of know that like a bet that seems too good to be true usually is, right? I don't know if there's quite an adage in journalism like a story that seems too good to be true usually is.
Yeah. Too good to check is the...
Too good to check. Yeah. Exactly. So but there's some overlap. I once had a great, I had a tabloid editor early in my career, and I had a tip, great story, and I had the story, you know, I had a quote, and the editor, and I'm making more phone calls, and the editor says, hey, you got to stop that, don't go over-reporting that story.
That story is good just like it is.
