I was reading through my daily litany of political blogs the other day when I ran across some new polling numbers in the Katherine Harris-Bill Nelson Senate race.
The results were interesting enough (more about them later), but as I looked behind the statistics, at the methods and pollster, it reminded me that I want to devote occasional column space this year to talking with you about the mechanics of politicking. It's important because polling is a valuable tool in politics, both in terms of running (and winning) campaigns and determining public opinion for those who govern. It is an honorable profession, and every pollster I worked with when I was a consultant from 1996-2004 had the highest level of integrity. Sometimes much higher than the candidates'.
The problem is not in the polling. The problem is in the way that political operatives use those numbers and the way that the media reports their efforts.
I was as guilty as the next guy. I didn't lie. But with polling, you don't have to lie to put your candidate in the best possible — if not entirely deserved — light.
The No. 1 hidden way to fudge polling results? Selecting the right people to be questioned for the poll — also known as choosing your sample.
Campaigns choose the people they're going to poll quite differently than do media outlets or independent polling organizations (Gallup, Harris, Zogby and the like).
Campaigns can control the outcome of the polling right from the start by the sample they select. Have a far right conservative or a dyed-in-the-wool liberal candidate who would benefit from only the hardest-core voters in their party casting ballots? Then ratchet down the sample to include only those voters who have gone to the polls in, say, three out of the last four primary elections. Those folks — politicos call them supervoters — will choose a very different candidate than, say, an occasional voter who only casts ballots in presidential years. Occasional voters are likely to be less partisan, more moderate, more swayed by advertising. Having them in the sample would produce a different result.
As for those media and independent polls, keep in mind that under Florida law, those pollsters do not have access to the actual voter registration lists — only campaigns and parties can get those. So those polls report their samples as coming from "likely voters." How do they determine who's likely to vote? Normally, they simply ask respondents on the telephone if they are registered to vote and are likely to vote. The problem is that most folks are likely to say they are likely to vote — even when they are not. Human nature is such that you want to sound like a good citizen.
I remember once working with my favorite pollster, Rob Schroth of Schroth & Associates, a Democratic firm. He told me once of inserting a question into the poll asking whether the respondent had voted in the last election. He got a percent response in the mid-60s. When he went back and cross-referenced the responses against the voter registration records, he found that number dropped by more than half.
Equally troubling is the size of a polling sample. And that brings us back to the Katherine Harris poll. After I ran across a mention of the poll on the liberal Florida News blog, I followed a link to Rasmussen Reports. Scott Rasmussen is a well-known national pollster who can legitimately claim some excellent prognostications. On Feb. 14, Rasmussen found that Harris still trailed Nelson in the U.S. Senate race, 49 percent-40 percent. More significantly, Rasmussen said the poll showed that Republicans were increasingly accepting Harris as their nominee, with 77 percent planning to vote for her compared with just 59 percent in January polling.
"Rank-and-file Republicans have made the adjustment …," Rasmussen wrote at that time.
As I do with every poll, my eyes immediately went to the poll's methodology, usually explained in a separate box or at the bottom of the story or news release. It said the poll was a telephone survey of 500 likely voters done on a single day in February. That standard line about methodology was the first red flag. In a state as large as Florida, any sample size less than 600-800 voters is not worth reading about.
"I am extremely suspect of polls with that low of an N [sample size] in a diverse state like ours," said Susan MacManus, an expert on polling and a political science professor at the University of South Florida. "What really drives me up the wall is when they proceed to break down responses by various demographic categories with extremely small cell sizes and no weighting whatsoever."
What she is talking about is that in a survey of 500 voters, carving out only the Republicans (as the Rasmussen poll does in examining Harris' acceptance rate in her own party) results in a much smaller sample and a higher margin of error. As a way of probing voter opinions, it is valid to use that smaller sample. As a way of predicting voter turnout — especially given Florida's racial and geographic diversity — it is not.
On top of that, a quick read through Rasmussen's website turned up the fact that this poll was automated through a cost-cutting process that some refer to as robo-polling. His technology doesn't use the normal live callers; instead, respondents are prompted by a recorded voice to answer questions by punching responses on their telephone keypads.
There is dispute in the polling industry about how accurate such automated calls are; Rasmussen touts his accuracy in the 2004 presidential campaign, in which he came within a half a percentage point in predicting Bush's and Kerry's totals.
The dispute over automation aside, the Katherine Harris poll shows just how complicated the science of polling can be, and how suspicious you — the consumer of political news — should be. Just like when you go into the grocery store and read a nutrition label, you should demand the same level of disclosure in our politics. Otherwise you can end up with a lot of empty calories.
Political Whore can be reached by e-mail at wayne.garcia@weeklyplanet.com or by telephone at 813-739-4805.
This article appears in Feb 22-28, 2006.