Politics

/

ArcaMax

Why getting the numbers right isn’t enough for pollsters to be credible in today’s polarized climate

Clifford Young, Johns Hopkins University, The Conversation on

Published in Political News

President Donald Trump launched his second term with a series of executive orders, asserting his authority more decisively than in 2017. His moves, shaped directly by unfiltered public opinion, align – for now – with what many Americans want. Pollsters are tracking this public sentiment in real time.

A pollster – of which I am one – measures and analyzes public opinion, serving as an interpreter between those who govern and those who are governed. While the horse race poll during elections is the most visible aspect of our work, our role is much broader.

Pollsters wear multiple hats, ensuring accuracy while also advising decision-makers on how to communicate with the public and to anticipate shifts in sentiment. At its core, polling is both an analytical and interpretive discipline. Pollsters do more than measure public opinion — they amplify the public’s voice, ensuring that leaders understand the concerns of those they represent.

Because truth reveals itself on Election Day, a pollster’s credibility is always at stake. If the industry collectively misses the mark, public trust erodes, and confidence in the democratic system itself is called into question.

How did pollsters perform in 2024? The answer depends on perspective.

From an analytical standpoint, the broad story that pollsters told was correct. Americans were frustrated by inflation and the cost of living, unable to reconcile their financial struggles with the Biden administration’s assurances that the economy was strong. Polls also revealed deep disillusionment with the political system, with many believing it was rigged against them. Trump successfully positioned himself as the champion of this discontent.

Statistically, the industry performed well by international standards. A 2018 Nature Human Behavior study analyzing 30,000 polls from 351 elections in 45 countries since 1942 found the average polling error to be about 2 percentage points. In 2024, national and swing-state polls outperformed this historical benchmark.

Compared with the last 17 presidential elections, polling in 2024 was more accurate than in eight, roughly on par with five and worse than four. A postmortem will reveal areas for improvement, but from a technical standpoint, the numbers fell well within the 2-percentage-point standard mentioned above.

Yet, despite statistical accuracy, public perception tells a different story. The gap between what pollsters measure and how the public interprets their work continues to widen.

Many Americans across the political spectrum viewed pollsters as unreliable, if not outright deceptive, in 2024.

The political right claimed polls systematically underestimated Trump, while the left accused pollsters of falsely portraying the 2024 race as close.

Journalist and Trump biographer Michael Wolff even declared: “One of the lessons from this campaign, as it should have been from prior campaigns, is, kill all the pollsters.” His sentiment, while extreme, reflected a broader frustration.

A deeper issue is that pollsters are increasingly seen as part of an establishment that no longer represents the public. Pollsters are now lumped in with politicians and the media, being trusted by only 21% of Americans, according to an Ipsos poll, where I serve as head of polling. This climate of distrust means that even minor polling errors are interpreted as signs of bias.

Yes, pollsters underestimated Trump in 2016, 2020 and again in 2024. These errors have clear methodological explanations: Some Trump voters were hard to reach, others were reluctant to disclose their preferences, and flawed turnout models assumed lower Republican participation.

While such methodological challenges are common in any scientific field, polling faces an added burden – its results are immediately tested in high-stakes elections. But to many, getting it wrong three times in a row suggests not error, but intent.

Trust, once lost, is difficult to regain.

This credibility problem is compounded by the rise of probabilistic forecasting – an approach that, while mathematically sound, often creates misleading narratives.

For two decades, these poll-based probability models have dominated election coverage. Forecasters like Nate Silver have shaped public expectations about such metrics.

 

Probabilities describe what might happen – but they fail to explain why events unfold as they do. This lack of diagnostic power makes probability-based forecasts feel both vague and misleading. They provide an illusion of precision while obscuring critical data trends.

Consider Silver’s 2024 forecast, which gave Harris and Trump each a 50% chance of winning. The final result – Trump 49.8%, Harris 48.2% – fell within the expected range of outcomes. Yet to the public, a 50/50 probability implied total uncertainty, masking underlying factors that pointed to Trump’s advantage.

Other indicators consistently suggested Trump had the upper hand, such as weak Biden approval ratings, belief that the country was on the wrong track, and the strength of candidates on the main issue, inflation.

Polling is just one tool. The industry has other ways to tell a more nuanced story. But the overreliance on poll-based probabilities – by both analysts and the media – has narrowed the focus, limiting our ability to contextualize broader electoral dynamics.

Put differently, pollsters failed to set the correct expectations for 2024.

To rebuild public trust, perception matters as much as accuracy.

When polling errors consistently lean in one direction, many assume bias rather than statistical uncertainty. Addressing this requires both technical precision and clear storytelling.

Polls do more than predict winners. They reveal shifts in public sentiment, offering insight into how and why opinions change.

Yet accuracy alone no longer suffices. While the 2024 polls performed within historical norms, public expectations have raised the bar for what qualifies as accurate polling. In a polarized climate, even small perceived failures fuel distrust.

Meeting this challenge means refining polling methods – in particular, ensuring that pollsters are vigilant in capturing a representative sample of Americans.

But pollsters are more than election forecasters; they are interpreters of public sentiment. The overreliance on the horse race poll has narrowed the field’s impact. Polling must be framed within the broader context of political and social change, making sense of uncertainty rather than just quantifying future likelihoods.

Election surprises stem from incomplete narratives. Precision matters, but a pollster’s job is ultimately about understanding and communicating what drives public opinion.

Restoring trust will require embracing this broader role with clarity and conviction. The polling industry’s problem isn’t just about data – it’s about narrative failure.

If pollsters get the story right, the future shouldn’t surprise. This requires more than just methodological adjustments – it demands a fundamental shift in how pollsters communicate their findings to the public.

This article is republished from The Conversation, a nonprofit, independent news organization bringing you facts and trustworthy analysis to help you make sense of our complex world. It was written by: Clifford Young, Johns Hopkins University

Read more:
Polling in the age of Trump highlights flawed methods and filtered realities

Why Americans are so enamored with election polls

Polls have value, even when they are wrong

Clifford Young does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.


 

Comments

blog comments powered by Disqus

 

Related Channels

ACLU

ACLU

By The ACLU
Amy Goodman

Amy Goodman

By Amy Goodman
Armstrong Williams

Armstrong Williams

By Armstrong Williams
Austin Bay

Austin Bay

By Austin Bay
Ben Shapiro

Ben Shapiro

By Ben Shapiro
Betsy McCaughey

Betsy McCaughey

By Betsy McCaughey
Bill Press

Bill Press

By Bill Press
Bonnie Jean Feldkamp

Bonnie Jean Feldkamp

By Bonnie Jean Feldkamp
Cal Thomas

Cal Thomas

By Cal Thomas
Christine Flowers

Christine Flowers

By Christine Flowers
Clarence Page

Clarence Page

By Clarence Page
Danny Tyree

Danny Tyree

By Danny Tyree
David Harsanyi

David Harsanyi

By David Harsanyi
Debra Saunders

Debra Saunders

By Debra Saunders
Dennis Prager

Dennis Prager

By Dennis Prager
Dick Polman

Dick Polman

By Dick Polman
Erick Erickson

Erick Erickson

By Erick Erickson
Froma Harrop

Froma Harrop

By Froma Harrop
Jacob Sullum

Jacob Sullum

By Jacob Sullum
Jamie Stiehm

Jamie Stiehm

By Jamie Stiehm
Jeff Robbins

Jeff Robbins

By Jeff Robbins
Jessica Johnson

Jessica Johnson

By Jessica Johnson
Jim Hightower

Jim Hightower

By Jim Hightower
Joe Conason

Joe Conason

By Joe Conason
Joe Guzzardi

Joe Guzzardi

By Joe Guzzardi
John Micek

John Micek

By John Micek
John Stossel

John Stossel

By John Stossel
Josh Hammer

Josh Hammer

By Josh Hammer
Judge Andrew Napolitano

Judge Andrew Napolitano

By Judge Andrew P. Napolitano
Laura Hollis

Laura Hollis

By Laura Hollis
Marc Munroe Dion

Marc Munroe Dion

By Marc Munroe Dion
Michael Barone

Michael Barone

By Michael Barone
Michael Reagan

Michael Reagan

By Michael Reagan
Mona Charen

Mona Charen

By Mona Charen
Oliver North and David L. Goetsch

Oliver North and David L. Goetsch

By Oliver North and David L. Goetsch
R. Emmett Tyrrell

R. Emmett Tyrrell

By R. Emmett Tyrrell
Rachel Marsden

Rachel Marsden

By Rachel Marsden
Rich Lowry

Rich Lowry

By Rich Lowry
Robert B. Reich

Robert B. Reich

By Robert B. Reich
Ruben Navarrett Jr

Ruben Navarrett Jr

By Ruben Navarrett Jr.
Ruth Marcus

Ruth Marcus

By Ruth Marcus
S.E. Cupp

S.E. Cupp

By S.E. Cupp
Salena Zito

Salena Zito

By Salena Zito
Star Parker

Star Parker

By Star Parker
Stephen Moore

Stephen Moore

By Stephen Moore
Susan Estrich

Susan Estrich

By Susan Estrich
Ted Rall

Ted Rall

By Ted Rall
Terence P. Jeffrey

Terence P. Jeffrey

By Terence P. Jeffrey
Tim Graham

Tim Graham

By Tim Graham
Tom Purcell

Tom Purcell

By Tom Purcell
Veronique de Rugy

Veronique de Rugy

By Veronique de Rugy
Victor Joecks

Victor Joecks

By Victor Joecks
Wayne Allyn Root

Wayne Allyn Root

By Wayne Allyn Root

Comics

Tom Stiglich Gary Varvel Ed Gamble Jack Ohman Ed Wexler Bill Bramhall