Is Polling in the 2020 US Election Reliable?

[russ_watters]

I know some people still think the 2004 election was stolen, but have people been paying attention to the laughably silly polling going on in the current presidential election? It seems like the media has learned nothing since the 2000 election night debacle where they flipped back and forth between Bush and Gore.

A few days ago, Obama was reported to have a thirteen! percentage point lead over Clinton, and Clinton appears to have won by 4%, a swing of 17%. When they are that far off, it shows how utterly useless such polling is. Even the exit polls showed Obama with a small lead.

It makes for an interesting race when things are so chaotic, but I hope people keep the poll results in proper perspective.

http://blogs.usatoday.com/onpolitics/2008/01/the-dems-surpri.html

 

[Gokul43201]

A few days ago, Obama was reported to have a thirteen! percentage point lead over Clinton,

You had to pick the poll that gave Obama the biggest lead, didn't you? If you average over the 7 major polls that were conducted on the 5th and 6th, they gave Obama an 8% lead over Clinton (38% to Clinton's 30%).

Source: realclearpolitics.com

and Clinton appears to have won by 4%, a swing of 17%.

With 96% of precincts reporting, Clinton has a 2% lead over Obama (39% to Obama's 37%).

When they are that far off, it shows how utterly useless such polling is. Even the exit polls showed Obama with a small lead.

I see now that exit polls actually had been pretty accurate.

From CNN exit polls: Clinton 38.7%, Obama 36.6%, Edwards 16.7%
Latest count (96% done): Clinton 39%, Obama 37%, Edwards 17%

The exit polls were dead on!

http://www.cnn.com/ELECTION/2008/primaries/results/epolls/index.html#NHDEM

So the exit polls were extremely accurate, but the pre-election polling was off by about 10%, which is quite terrible. My guess for what caused the difference: more women turned up to vote for Clinton than expected, and more independents voted in the Republican primary, for McCain (pre-election polling had McCain with 32%, but he's won 37%).

 

[Ivan Seeking]

It seems that many people made up their minds today. This may have played a roll in the pre-election polling error.

Yes, the derivation showing that a Bush victory was a statistical impossibility was based on exit polls.

 

[Gokul43201]

It seems that many people made up their minds today. This may have played a roll in the pre-election polling error.

I doubt this made any real difference. From the exit poll data, it appears that about 39% of those that made up their minds today picked Clinton, and that's no different from those that had made up their minds earlier.

 

[russ_watters]

You had to pick the poll that gave Obama the biggest lead, didn't you?

Sorry, I'm a USA Today reader and USA Today/Gallup's poll was one of two that had Obama winning by 13%.

From CNN exit polls: Clinton 38.7%, Obama 36.6%, Edwards 16.7%
Latest count (96% done): Clinton 39%, Obama 37%, Edwards 17%

The exit polls were dead on!

http://www.cnn.com/ELECTION/2008/primaries/results/epolls/index.html#NHDEM

Exit polls reported during the day are not corrected for the demographic shifts (because they don't know what they are yet), but the final exit polls are. That's how the exit polls during the day can be inaccurate but the final exit poll is accurate. The exit polls reported during the day did not agree with the results. This was one of the accusations made about the 2004 election, that the exit polls were "fixed". In reality, that's standard procedure because the exit polls aren't really meant to predict the winner (yet they still use them that way), but only to discuss who voted for who and why. It's also the main reason why the 2000 election was called prematurely for Gore.

...but the pre-election polling was off by about 10%, which is quite terrible. My guess for what caused the difference: more women turned up to vote for Clinton than expected, and more independents voted in the Republican primary, for McCain (pre-election polling had McCain with 32%, but he's won 37%).

Here's some post-mortem:

He and other pollsters — including John Zogby, who had Obama up by 13 — hypothesize that large numbers of people changed their minds after most poll interviews were conducted over the weekend.

"I have polled many races, especially close ones, where 4% to 8% have said they finally decided on their vote the day of the election, and that can wreak havoc on those of us who are in the business of capturing pre-election movements and trends," Zogby wrote on his website.

Nearly one-fifth deciding on election day, he said, was "unprecedented."

http://www.usatoday.com/news/politics/election2008/2008-01-09-polls_N.htm?loc=interstitialskip

 

[russ_watters]

Yes, the derivation showing that a Bush victory was a statistical impossibility was based on exit polls.

The pollsters themselves disagree (and I don't just mean what they say, the VNS itself was disbanded for that failure) - the non-expert who wrote that analysis misused the data, but in any case the point of this thread isn't to dig up the past. Here's some info written prior to the New Hampshire primary:

Here are a few tips for making sense of the exit poll data that you do see tonight (Gokul, see #3):

1) An exit poll is just a survey. Like other surveys, it is subject to random sampling error and, as those who follow exit polls now understand, occasional problems with non-response bias. In New Hampshire (in 1992) and Arizona (in 1996)* primary election exit polls overstated support for Patrick Buchanan, probably because his more enthusiastic supporters were more willing to be interviewed (and for those tempted to hit he comment button, yes, I know that some believe those past errors suggest massive vote fraud -- I have written about that subject at great length).

2) The networks rarely "call" an election on exit poll results alone. The decision desk analysts require a very high degree of statistical confidence (at least 99.5%) before they will consider calling a winner (the ordinary "margin of error" on pre-election polls typically uses a 95% confidence level). They will also wait for actual results if the exit poll is very different from pre-election poll trends. So a single-digit margin on an exit poll is almost never sufficient to say that a particular candidate will win.

3) Watch out for "The Prior." At least two networks are likely to post exit poll tabulations shortly after the polls close that will update as the election night wears on (try these links for MSNBC and CNN). Those data are weighted to whatever estimate of the outcome the analysts have greatest confidence in at any moment. By the end of the night, the tabulations will be weighted to the official count. Typically, the exit poll tabulations are weighted to something called the "Composite Estimate," a combination of the exit poll data alone and a "Prior Estimate" that is based largely on pre-election poll results. So if you look to extrapolate from the initial tabulations posted on MSNBC or CNN (as we did here on Election Night 2006), just keep in mind that in the estimate of each candidate's standing in the initial reports will likely mix exit poll and the pre-election poll estimates (not unlike the kind we report here).

http://www.pollster.com/blogs/looking_for_new_hampshire_exit.php

I actually was in school in 1999 and took a class called "Public Opinion and Propaganda", taught by a fairly prominent public opinion polling expert (he didn't earn his living teaching). He warned us then that the pre-election polls at primary time were pretty much useless, even right up to the days before the election. The landscape is just too chaotic for them to be stable. And those polls taken in theh spring and summer, a year and a half before the election - they're good only for a laugh at how seriously the media takes them.

 

[falc39]

hey... did you guys know that the same diebold voting machines that were flawed was widely used? look at the stats:

http://ronrox.com/paulstats.php?party=DEMOCRATS"
http://ronrox.com/paulstats.php?party=REPUBLICANS"

there's a huge discrepancy between clinton and obama

it's not proof, but worth pondering

 

[Gokul43201]

Exit polls reported during the day are not corrected for the demographic shifts (because they don't know what they are yet), but the final exit polls are. That's how the exit polls during the day can be inaccurate but the final exit poll is accurate.

Okay, that explains it. I could've sworn that I'd previously seen the CNN exit poll say that Obama was up by 1% (I said as much in the other thread and had to delete it later when I saw the numbers again).

 

[russ_watters]

Yeah - the link in the OP is to an election night blog that says exactly that.

 

[ShawnD]

From falc39's link, I'm surprised how well Ron Paul did. He was polling within the margin of error for quite a while, but he beat Giuliani in NH.stat.

Is there a reason US primaries are all done on different dates for different states? Doesn't that sort of eliminate a significant chunk of democracy when you're allowed to see how the votes are going before you vote? It looks like <democrat> is going to win in that party, so I will vote for <republican> because of <reason>.

 

[Evo]

From falc39's link, I'm surprised how well Ron Paul did. He was polling within the margin of error for quite a while, but he beat Giuliani in NH.http://checkthevotes.com/primary_gop_New_Hampshire-summary.

The NH finals were Giuliani 8.5% & Paul at 7.7%

 

[ShawnD]

The NH finals were Giuliani 8.5% & Paul at 7.7%

Votes counted by hand:
Paul - 3,431
Giuliani - 3,038

 

[Evo]

Votes counted by hand:
Paul - 3,431
Giuliani - 3,038

That's not representative of the entire state and are meaningless, or are you be facetious?

Candidate Vote % Delegates
John McCain 88,466 37.1% 7
Mitt Romney 75,343 31.6 4
Mike Huckabee 26,768 11.2 1
Rudolph W. Giuliani 20,395 8.5 0
Ron Paul 18,303 7.7 0
Others 5,099 2.1 0
Fred D. Thompson 2,886 1.2 0
Duncan Hunter 1,220 0.5 0
Tom Tancredo 68 0.0 0

 

[ShawnD]

The whole reason for doing a hand count is that it's decided the machines themselves cannot be trusted. This is like when your doctor tells you something is wrong and you ask for a second opinion; you don't ask that same doctor to give you his opinion the second time as well.

 

[Gokul43201]

The whole reason for doing a hand count is that it's decided the machines themselves cannot be trusted.

Don't believe that's the case here. Some small fraction of the polling locations did not use the voting machines. Votes would be hand counted only at those specific places. The majority of the votes cast were indeed machine counted.

 

[Evo]

The whole reason for doing a hand count is that it's decided the machines themselves cannot be trusted. This is like when your doctor tells you something is wrong and you ask for a second opinion; you don't ask that same doctor to give you his opinion the second time as well.

Those aren't recounts, they are places where the tallying was done by hand, it's not a representation of the state wide voting.

There is going to be a recount, I doubt much will change. Either way Paul is dead as far as the Presidential race goes.

 

[russ_watters]

Is there a reason US primaries are all done on different dates for different states? Doesn't that sort of eliminate a significant chunk of democracy when you're allowed to see how the votes are going before you vote? It looks like <democrat> is going to win in that party, so I will vote for <republican> because of <reason>.

Yes, I agree that that's a flaw in the primary system. It's a big part of the reason for the issue in this thread: momentum affects the outcomes of future primaries.

However, you can't vote for either party in most primaries, just one. And few people will actually switch their registration to vote in the other party's primary (though it does happen occasionally).