Gallup poll oversamples Republicans...

[davepurcell]

...just like the other polls that show Dubya with a large lead. There's a breakdown of the internals here (http://www.theleftcoaster.com/archives/002806.html).

The Gallup poll is based on the assumption that 40% of those turning out to vote will be Republicans, and only 33% will be Democrat. This despite the breakdown of the last three presidential elections (courtesy of John Zogby):

If we look at the three last Presidential elections, the spread was 34% Democrats, 34% Republicans and 33% Independents (in 1992 with Ross Perot in the race); 39% Democrats, 34% Republicans, and 27% Independents in 1996; and 39% Democrats, 35% Republicans and 26% Independents in 2000.

It's still a ballgame folks, and please don't let any erroneous polls discourage you from coming out on election day.

dp

[the happy prole]

I would guess that Clifton donates to the GOP for the same reasons most rich businessmen CEO's typically do. It's good for business, it's good for the rich. And I think personally they really do believe in conservative values and free market. That's why they became bidnessmen in the first place.

But let's break it down. First, Zogby's numbers are from the last three elections. Two were won by Dems; actually all three if you look at the popular vote. It stands to reason that when a Democrat wins, at that point in time more people will be identifying themselves as Dems.

So if Bush is indeed winning the polls, then probably there are now more people identifying themselves as Republicans than there were in 1992, 1996, and 2000. I would expect the party split should generally reflect the candidate vote, and this is consistent with that.

It makes no sense to assume that 2004 political breakdowns will be the same as 2000. Especially when the Zogsby data indicates that party splits do in fact, vary over time.

And of course, there is the issue of how do we even know Zogby is right and Gallup is wrong? Dude assumes that Zogby is correct and therefore any deviation means Gallup is messing around.

The author is either confused or deliberately misleading. The way that Gallup gave their explanation, they stated up front that the percentage differs. Of course it does. It's not that Gallup decides to call up 40% GOP. They just dial and ask. They ended up with 40% GOP.

What Gallup says is that they figure out a list of all numbers in the US (even unlisted), then randomly dial 1,000. Now they could be lying about that. But if they were, they wouldn't just send you a piece of paper that says "We poll 40% GOP." They'd be more clever than that.

Its possible that for some reason (more money, more time, better command of English, more patience, more phones, etc.), Gallup tends to get better response from Conservatives. So maybe they need to do a better job and fix that bias.

But I certainly don't see a clear-cut case for bias from what this guy has written, much less that Gallup deliberately mangles their polls because the CEO is conservative.

[davepurcell]

Good points, HP. I don't think the author needed to bring up the fact that Gallup's CEO is a major Rep donor. While I wouldn't put it past any of those f*ckers to do something shady after seeing what happened (and continues to happen) in Florida, I think it weakens his argument to throw it out there.

The author is either confused or deliberately misleading. The way that Gallup gave their explanation, they stated up front that the percentage differs. Of course it does. It's not that Gallup decides to call up 40% GOP. They just dial and ask. They ended up with 40% GOP.

I'm not sure that's the case. Gallup stated that they've been using those percentages throughout the election cycle and will continue to do so. If they were doing a pure random sample, then they wouldn't know in advance they were going to get 40% Rep, and it's almost impossible that they would get 40% (on avg) Rep every time.

That makes me think that they're purposely oversampling Reps for some methodological reason (e.g., they think that's how the election day turnout will break down). And even if they ended up with 40% Reps on a purely random sample, they could weight the sample to bring it more in line with how the turnout has been historically.

What Republicans seem to not be able to understand (not saying you're in this camp, HP) is that the lefty bloggers who've been criticizing these results aren't doing so in a desperate attempt to believe Kerry isn't behind. Dems want the real picture as much as anyone -- after all, how can you adjust your campaign if you don't know where you stand? These people have criticized the polls because, in many cases, they're unrepresentative and misleading. They're guilty of simply bad survey execution and it's fair to expect more of an organization with Gallup's reputation.

dp

[davepurcell]

What a surprise. The NYT poll is unrepresentative as well (http://www.emergingdemocraticmajorityweblog.com/donkeyrising/archives/000687.php).

[Marlowe]

Originally posted by davepurcell
What Republicans seem to not be able to understand (not saying you're in this camp, HP) is that the lefty bloggers who've been criticizing these results aren't doing so in a desperate attempt to believe Kerry isn't behind. Dems want the real picture as much as anyone --
C'mon, it's clear that this guy whose blog you linked, isn't simply interested in accurate results in polls.

The name of his blog is "the emerging democratic majority weblog". He himself has an ideological edge to his analysis wherein he purposts that Kerry and Democrats are 'really' higher in the polls than they're being represented. His "donkey rising" graphic is more telling than his analysis -- he has an ideological belief and he only gives evidence that supports his side of things.

But then, Dave, you're the same guy who called moveon.org a "centrist" organization. So, in your world he probably is unbiased.

[the happy prole]

I have never heard that any of the polls keep lists of who is registered with what party beforehand. I also haven't heard of any poll saying they weight their polls post-survey to reflect party distribution.

It also doesn't make sense to me that NYT and Gallup would both do the same thing. First off, both organizations have a lot more to lose than gain by rigging polls. Second, the NYT is supposedly liberal while Gallup is accused of being conservative. So why would they both favor Republicans?

Marlowe, davepurcell never claimed the guy wasn't biased. He called him a "lefty blogger" after all. I'm a lefty myself, and I'm voting for Kerry. But I too, would like to see accurate polls and I think there's a fair criticism being levelled here. I just don't think Gallup and NYT are deliberately taking party affiliation into account. It's just an error in methodology.

I think it's likely that post-RNC and Swift Boat, Republicans were more fired-up and therefore more likely to respond to a poll survey. Thus you end up with more Republican respondents. In other words, this may just be part of the well-documented "Post-convention bounce."

The new polls show that the race is much closer than the polls after RNC indicated. Is it Gallup and NYT's responsibility not to interview people so close to the convention, or should we just wise up and realize that post-convention polls aren't accurate?

Regardless, the election is close. Even if it weren't, you should be extremely wary of letting the poll results influence your decision to vote. I think we all agree on that.

[davepurcell]

Originally posted by the happy prole

It also doesn't make sense to me that NYT and Gallup would both do the same thing. First off, both organizations have a lot more to lose than gain by rigging polls. Second, the NYT is supposedly liberal while Gallup is accused of being conservative. So why would they both favor Republicans?

I don't think they're rigging the polls -- and that's why I disagree with the Left Coaster posting about Gallup's CEO. But their methodology strikes me as somewhere between bad and sloppy. I'd like to read more about it, though, because as much survey experience as I have, I'm not a poli sci guy. At the very least, I think the criticism coming from these guys is interesting and worth considering, and I think it's interesting that the pollsters have apparently been cooperative in answering their questions.

I think two things are possible: 1) their methodolgy is the same as it has always been and changing conditions (caller ID, cellphones, etc.) have made the model obsolete; or 2) this election really is so close and/or volatile, that the wildly varying poll results are a reflection of that. I think it's possibly a combo of both.

But I too, would like to see accurate polls and I think there's a fair criticism being levelled here. I just don't think Gallup and NYT are deliberately taking party affiliation into account. It's just an error in methodology.

Well said.

dp

[dcXhc]

Two points:

1- I'm not sure where the blogger gets the 4-point Rep advantage he cites. If you look at the complete results of the NY Times poll (http://www.nytimes.com/packages/khtml/2004/09/17/politics/20040917_POLLRESULTS.html) there is a 2-point Rep advantage when respondents are asked how they would describe themselves. There is also a hypothetical question regarding a House of Representatives election in which 41% of respondents state that they would favor a Democrat while 39% said they would favor the hypothetical Republican.

2- What is the basis for believing that this 2000 exit poll breakdown is still valid and accurate? In other words - both the NY Times polls and the Gallup poll are being questioned because they don't match an underlying assumption that stated party affiliation today is exactly the same as it was four years ago. Perhaps it is, or perhaps 9-11 and Iraq have changed things. I don't have an answer, but I think it is reasonable and prudent to examine this assumption.

[davepurcell]

Originally posted by dcXhc 2- What is the basis for believing that this 2000 exit poll breakdown is still valid and accurate? In other words - both the NY Times polls and the Gallup poll are being questioned because they don't match an underlying assumption that stated party affiliation today is exactly the same as it was four years ago. Perhaps it is, or perhaps 9-11 and Iraq have changed things. I don't have an answer, but I think it is reasonable and prudent to examine this assumption.

Good question. I'd guess it's because the NYT's responses on that question are out of line with other party ID responses turned up by other polls, but I dunno for sure.

Bottom line, this is all interesting because we may be seeing a sea change in how political polling works. Again, it could be just a volatile electorate, but it could also be a case of the old ways of business being outdated (Zogby has some interesting stuff on his site about adapting his methods). I guess we'll see...

dp

[davepurcell]

Nothing new here, but perhaps some additional confirmation of what we've been kicking around re: polls the last few days. This is from the Wall St Journal via Daily Kos (it's subscription-only, so I haven't seen the whole article):

WASHINGTON -- Widely divergent poll results in recent days underscore a paradox of the 2004 presidential race: Despite all the surveys, it may be the toughest election in memory for anyone to track.

Opinion polls themselves had been getting harder to conduct long before the matchup between President George W. Bush and his Democratic rival, Massachusetts Sen. John Kerry. The reasons range from growing reluctance to participate in surveys to increasing reliance on cellphones rather than the land lines pollsters have long used to ensure demographic and geographic balance in surveys.

But this year's bitter presidential contest has heaped on new challenges. They include an exceptionally close race and a polarized electorate that magnifies the consequence of different polling methods. In addition, unprecedented voter-mobilization drives by both parties make it especially tough for pollsters to say which voters probably will show up on Election Day.

"It makes it harder" to forecast the likely electorate, says Fred Steeper, a longtime pollster for Mr. Bush. In the six weeks to Election Day on Nov. 2, he adds, disparate polls may reflect sampling error and methodological differences more often than shifting opinion. "My advice to the consumer is ... the day-to-day reports of polling will exaggerate the changes in this race."

[AngelV]

Here's the full WSJ article (http://online.wsj.com/article/0,,SB109563333476021828,00.html?mod=home%5Fpage%5F one%5Fus):

http://online.wsj.com/public/resources/images/P1-AB983_POLL09192004205921.gif


Divergent Opinion Polls Reflect
New Challenges to Tracking Vote

By JOHN HARWOOD
Staff Reporter of THE WALL STREET JOURNAL
September 20, 2004; Page A1

WASHINGTON -- Widely divergent poll results in recent days underscore a paradox of the 2004 presidential race: Despite all the surveys, it may be the toughest election in memory for anyone to track.

Opinion polls themselves had been getting harder to conduct long before the matchup between President George W. Bush and his Democratic rival, Massachusetts Sen. John Kerry. The reasons range from growing reluctance to participate in surveys to increasing reliance on cellphones rather than the land lines pollsters have long used to ensure demographic and geographic balance in surveys.

But this year's bitter presidential contest has heaped on new challenges. They include an exceptionally close race and a polarized electorate that magnifies the consequence of different polling methods. In addition, unprecedented voter-mobilization drives by both parties make it especially tough for pollsters to say which voters probably will show up on Election Day.

"It makes it harder" to forecast the likely electorate, says Fred Steeper, a longtime pollster for Mr. Bush. In the six weeks to Election Day on Nov. 2, he adds, disparate polls may reflect sampling error and methodological differences more often than shifting opinion. "My advice to the consumer is ... the day-to-day reports of polling will exaggerate the changes in this race."

Media coverage of the campaign last week appeared to prove that point. On the same day last week, USA Today cited a new poll by the Gallup Organization in reporting that Mr. Bush "has surged to a 13-point lead" over Mr. Kerry, while other news organizations reported surveys by Pew Research Center and Harris Interactive showing the contest tightening to a dead heat.

Adding to the confusion is the way poll reports themselves become weapons in the campaign. The Bush campaign swiftly touts favorable surveys and seeks to discredit those showing Mr. Kerry drawing closer. The approach plays on the so-called bandwagon effects that energize supporters of a surging candidate and dispirit those of a lagging one.

Kerry advisers embrace dead-heat polls as a way to halt high-profile critiques of their campaign's inner workings and shift public dialogue to more fruitful ground such as violence in Iraq or domestic issues. Thus, even as Bush aide Matthew Dowd argued that Mr. Bush's lead was widening at week's end, Kerry spokesman Joe Lockhart told reporters, "The trends are going in our direction."

Underlying those conflicting arguments aren't just different political calculations but also differences in polling philosophy and techniques. Consider last week's Pew Research Center survey, which showed strikingly different research during two consecutive polling periods.

In the portion of the survey conducted Sept. 8-10, Mr. Bush led Mr. Kerry 52%-40% among registered voters. In a separate portion conducted Sept. 11-14, Messrs. Kerry and Bush were tied at 46%. But there was one other key difference, too: Among voters sampled in the first portion, self-described Republicans outnumbered Democrats by two percentage points; in the second, Democrats outnumbered Republicans by four percentage points.

Pew pollster Andrew Kohut described that difference as normal week-to-week drift -- because party allegiance is a fluctuating attitude -- that doesn't call his results into question. In fact, he says his surveys show the race is more volatile than other analysts have suggested. But the Bush campaign insists the partisan variation exaggerated the appearance of a trend toward Mr. Kerry.

Party allegiance "does not change in seven days" by that much, says Mr. Steeper, the Bush pollster. He says Mr. Kohut should have "weighted" his poll with a common assessment of partisanship for both samples; averaging the two would have shown the president with a steady lead of about six percentage points.

Bush advisers were more pleased by a CBS-New York Times survey late last week showing the president leading Mr. Kerry by nine percentage points, 50% to 41%, up from seven points the previous week.

Yet those CBS surveys were conducted the same way the Pew polls were -- without making any adjustment for the different number of Republicans and Democrats surveyed. And in the CBS polls, the number of Republicans surveyed rose sharply from the first week to the second.

Last week's CBS sample, in a mirror image of Pew's, contained four percentage points more Republicans than Democrats. Because this polarized contest has left roughly nine in 10 adherents of each party supporting its nominee, such variation in the number of Republicans and Democrats surveyed has an unusually large impact on polling outcomes.

In a close race, in fact, that can make the difference between an apparent dead heat and a solid lead for one candidate. If the CBS and Pew surveys are adjusted to reflect comparable numbers of Republicans and Democrats, their results would have been virtually identical.

Indeed that's precisely what liberal polling analyst Ruy Teixeira did on his Web log, called Emerging Democratic Majority. As the New York Times report of the poll carried the headline "Bush Opens Lead," Mr. Teixeira's blog declared, "CBS News/New York Times poll has it close to even."

(The Wall Street Journal and NBC News plan to release the latest of their surveys later this week. The Journal/NBC poll does adjust for variations in self-described party identification.)

Mr. Teixeira argues that the Democratic edge Mr. Kohut found is realistic, since exit polls from the 1996 and 2000 campaigns indicated that in both cases four percentage points more Democrats than Republicans showed up to vote. Slightly more self-described Democrats than Republicans voted in the 1984, 1988 and 1992 elections as well.

There's no assurance that will be the case this year, since both realignment of voter attitudes and party turnout drives can sharply affect that balance. Mr. Dowd says a roughly equal number of Democrats and Republicans will show up on Nov. 2.

Just who will turn out represents one of the biggest quandaries facing pollsters. About 105 million ballots were cast in 2000, and all sides agree more Americans will vote this time. Bush strategist Karl Rove predicts a total of around 110 million; Democrats estimate an even larger turnout, with some projections as high as 120 million.

Close to election time, pollsters like to report results among those considered most likely to vote on the theory that those results will align most closely with the final outcome. But weeks away from Election Day that's especially difficult to do, since many of the campaign's mobilization activities occur immediately before the election.

"I don't know how you factor that into your polling," Mr. Steeper says. Adds Democratic pollster Peter Hart, a veteran of presidential politics who helps conduct the Journal/NBC survey: "This is art. This isn't science. Nobody knows."

The Journal/NBC survey uses a single question to identify likely voters. It asks respondents to assess their interest in the election on a 10-point scale with 10 as the highest; those responding 9 or 10 are called likely voters.

The Gallup Poll, which provides surveys for CNN and USA Today, among others, assesses likelihood of voting in a different way that has raised the ire of the Kerry campaign. Gallup asks a series of questions first devised decades ago that assigns voting probability to each respondent; it then uses their answers and an overall estimate of voter turnout to identify the likely electorate.

Since mid-July, that method has yielded a likely electorate that is substantially more Republican-leaning than those of recent presidential contests. For instance, the likely-voter sample in last week's survey showing Mr. Bush ahead by 13 points contained seven percentage points more Republicans than Democrats. Given the current polarization by party, the survey would have showed a near-even race had the sample's partisan balance matched the 2000 exit polls or the registered-voter sample in the Pew poll.

[AngelV]

(continued)


As a result, Kerry pollster Mark Mellman has loudly accused the high-profile Gallup survey of using a likely-voter identification method that is "not very accurate," in part because the screening questions are outdated and because they can't properly measure voting intention so long before Election Day. The substantial variation between the likely-voter results and Gallup's registered-voter findings -- which showed an eight-percentage-point Bush lead -- is larger than what other likely-voter assessments usually record, Mr. Mellman says.

"We're open to any scientific evidence that would point to our modifying our likely-voter model," responds Frank Newport, editor-in-chief of the Gallup Poll. Mr. Newport says so far he hasn't seen any.

In 2000, Gallup's election-eve sample of likely voters showed Mr. Bush leading by two percentage points over Al Gore. Its registered-voter sample, showing Messrs. Bush and Gore neck and neck, was closer to the actual Election Day results. But Mr. Newport notes that in 1996 the likely-voter model more accurately forecast the size of Bill Clinton's victory over former Senate Republican Leader Bob Dole.

Mr. Bush narrowly lost the popular vote while winning enough electoral-vote battlegrounds to capture the presidency. And the same polling variations that can affect assessments of the race nationally are multiplied this time around by the intense focus on polls in battleground states. Democrats insist Mr. Kerry is more competitive with Mr. Bush in the race toward a 270-electoral vote majority than national polls would suggest. They argue Mr. Bush is piling up exceptionally large polling margins in states he's already expected to win, masking closer contests in battlegrounds such as Ohio and Florida.

Mr. Dowd argues, to the contrary, that Mr. Kerry's advantages in Democratic-friendly states such as California and New York offset Mr. Bush's edge in the South and Mountain West. As a result, he says, the president's advantage in battleground states matches his national lead.

The argument is especially hard to sift since different surveys of battleground states as a group show different results. There are multiple public polls of many individual battlegrounds, and the campaigns rarely publicize their private battleground-state surveys.

[SteelTown Boy]

whoa...reminds me that i gotta go and read up on that.

[markalot]

Polling is such bad news. A series of bogus polls can sway the election as people jump on a bandwagon that may or may not have existed.

[akip]

none of this is gonna matter if kerry can't figure out how to connect with the public.

[davepurcell]

Thanks for the article, Angel.

[Marlowe]

Heh -- I just happened to run across this link (http://www.emergingdemocraticmajorityweblog.com/donkeyrising/archives/000521.php) from, ahem, 'analyst' Ruy Teixeira regarding a June LA Times poll that over-sampled Democrats. The name rang a bell, and then I remembered he'd been discussed here.

How convenient that this guy finds it very plausible that an outsized sampling of Democrats could be accurate and acceptable in this poll, but then the story changes considerably once President Bush went ahead in the polls. When a poll showed Kerry well in the lead, the sample bias might very well be reflecting of a cavalcade of voters rushing to become Democrats! But then suddenly, and conventiently right when Presiden Bush jumps ahead, sampling bias became a pervasive and worrisome problem that had to be identified, reported and rooted out. Funny how quickly this polling stuff became such a problem, innit?

In other words, this guy is just another political hack trying to spin things his direction.

[davepurcell]

Fair question. I'd like to see if he's addressed that but I don't have time to wade through three months of posts on his site.

Teixeira is clearly a progressive but he's far from a "partisan hack." He's a respected political analyst with a PhD from one of the best sociology departments in the country. His last book drew praise from George Will, The Economist, and the Weekly Standard, not exactly bastions of progressive political thought.

Again, Dems have nothing to gain by deluding themselves about poll results. When polls use improper methodology, it's fair to question their results.

dp

[SteelTown Boy]

after reading,i decided that if a voter relies too much on polls(remember,they're biased either way) should not be voting.