Okay, So A Poll Can’t Predict The Future

One of my 5 takeaways appears to have angered WPRI’s Ted Nesi, specifically the part about the usefulness of public polling.

Nesi rightfully calls me out for talking about polls as though they’re predictors of results. This point is well taken, I’m wrong, and I edited my post to include Nesi’s points. And, over the course of 9 subtweets, Nesi goes on to argue the following further points:

• Campaigns worked to change the facts on the ground that the polls reflected
• Polling showed momentum for Gina Raimondo and Seth Magaziner
• Polling showed a large amount of undecideds
• Internal campaign polls mirrored WPRI/The Providence Journal polls
• Don’t misinterpret data

Some of this is, “okay, fair enough.” Obviously the number undecideds is important. If there had been a low number of undecideds early on, we might have seen more negative campaigning as campaigns worked to attempt to dislodge voters from their chosen candidates.

I have nothing to say about internal polling, because as wonderful as it is that campaigns share their polling with Ted Nesi, it’s never a news story (for obvious ethical reasons). I can only go off of what’s getting reported.

But let’s look at the argument for momentum, because there were a couple of times that the narrative WPRI had was not in fact reflective of what their polling showed. The first is in May, when pollster Joe Fleming speculates that Pell’s drop from 14.7% to 11.5% is likely due to the issue of his stolen Prius:*

Fleming blamed Pell’s sinking poll numbers on the strange saga of his missing car, which garnered significant media attention in recent months. “It just had legs,” he said. “People kept talking about it. If you mention Clay Pell to voters right now, what’s the one thing they say? The car.”

Except, that’s not what the poll showed. The poll showed movement within the margin of error from the last poll. Fleming used the same number of voters, the same contact methods, and had the same MoE for each poll. There’s no way to tell if Pell was actually sinking, or whether it was just a sampling error. In the same manner, the 2.2% rise for both Taveras and Raimondo is also within the MoE and could also be attributable to sampling error. The same critique can be applied to this line from the May story:

The mayor has modestly increased his support among all subgroups of voters since February…

The subgroup MoE is even larger than the +/-4.38% for the regular 503 voter sample. There was no way to know whether it was a real increase or simply noise. But it fit a great narrative, and it got added.

The August poll is better, but again, it still has issues. Raimondo’s increase is still within the MoE from the May poll. While we can definitely argue momentum from the February poll, that’s not how the story was reported. It was momentum from the May poll. The poll did correctly display an increase in Pell support and a decrease in Taveras support. But the most interesting part of the poll wasn’t reported anywhere: Pell’s massive leap forward in support from Democratic-leaning unaffiliated voters.

“But wait,” you might be asking yourself, “wasn’t the conventional wisdom that turnout among independents would benefit Raimondo and that Pell represented the most Democrat-y of the Democrats?” Yes, and that’s in fact how Fleming reported it in February:

Fleming said the numbers show why the Raimondo campaign will want to attract more independents to vote in the Democratic primary, whereas “if I’m the Taveras campaign I want to see Democrats out voting in the primary instead of independents.”

But it wasn’t worth reevaluating that narrative, because it was already in place, and still mostly held up, even though Taveras had lost a lot of support among Democrats, Pell was up among both Democrats and Democratic-leaning unaffiliated voters, and Raimondo was up among Democrats and within the MoE for the unaffiliateds.

And yes, the May and August poll did show momentum for Magaziner.

So back to the rest of Nesi’s argument. Yes, campaigns worked to change the facts on the ground that the polls reflected. But campaigns were also doing what they were supposed to do: realize that polls are not always an accurate snapshot of any given time and work to make sure they’ll win.

And this gets me to the point I inartfully made in my last post. I think polling like this is a terrible way to evaluate the state of the campaign. The problem is that journalists, many of them being brought up to be objective observers, have very little experience on how to evaluate how an effective campaign works. Being able to evaluate the quality of the voter contacts a campaign is making, the caliber of its staff, and the ability of its get-out-the-vote operation are all far more helpful for painting a picture of the campaign in its moment than the polls are. But again, journalists don’t necessarily have the reference points to be good evaluators of campaign infrastructure.

Just take a look at the campaign managers for the five big names in the gubernatorial primaries. Raimondo had Eric Hyers, who has lots of experiencing managing winning campaigns in Providence, and now appears to have an ability to pull off unexpectedly large margins of victory. Pell was well-off but not as well-equipped with Devin Driscoll, who among other things led Obama for America’s Rhode Island chapter and worked with Rhode Islanders United for Marriage. Taveras had Danny Kedem, an outsider who was previously best known for abandoning Anthony Weiner’s mayoral campaign even as he was its manager. That should tell you a lot.

On the Republican side, you had Patrick Sweeney with Fung. Sweeney is an operative within the RI Republican Party, and though he’s only ever been on losing campaigns, he does have a lot of experience with Republican voters. Block had Jeff Britt, who has never run a successful campaign and has otherwise been a hired gun whose forte is opposition research (mudslinging). Looking at the campaign managers and doing some logical reasoning, you could make a reasonably informed prediction about where this race was going to go.

The other problem with polls is many voters will make my initial mistake, thinking of them as predictors and not fuzzy pictures of the present. Media has incentive to leave us with the former impression rather than the latter, because “Raimondo takes lead in race for Gov” is a great headline, but “Raimondo might be doing better, but we can’t be sure because of sampling error” just sounds like you’re faffing about.

As I’ve said before, leaving us with the wrong impression is harmful, because some voters attempt to vote strategically. If they feel that their guy is a loser because the polls say so, they may switch their vote.

The other way polls harm is that they get us focused on numbers, rather than issues. And yes, if it’s not these numbers that we’ll fixate on, it’ll be the fundraising numbers, that’s a given. And I understand there weren’t a lot of fireworks in this campaign until the end, and the candidates weren’t that far apart on the issues, but it might have been worth examining the dynamics of some of the legislative campaigns in the lull between Raimondo event and Taveras press release and Pell Prius saga. You know, the campaigns that collectively matter more than the race for governor?

So yes, let’s use the polls to inform reporting, analysis, and opinion. But maybe we need to stop making each release of a poll a two-outlet, day-long celebration of our wonderful public service to this state. Let’s temper that by realizing we all desire to craft narratives, and those narratives, even if proven correct, may be built on nothing more substantial than hunches and gut feelings.

---

 

*An earlier version of this post erroneously said Pell dropped from 11.7% to 9%. Thanks to Ted Nesi for pointing out the mistake.