Forums

Has anyone kept up with the allegation that many of the presidential polls are skewed by under sampling Republicans?

Obviously, there is some guess work in predicting the demographic turnout of likely voters, but one can make some reasonable estimates and boundary conditions based on previous elections. This link states that Rasmussen says that the US is made up of 35.4% Republicans, 34.0% Democrats, and 30.5% Independents (I suppose all other parties are counted as Independents). I would have expected Democrats and Republicans to be roughly equal, so this sounds plausible.

I’ve heard allegations that many pollsters are using the assumption that the next election’s turnout will be the same as 2008 turnout, though currently party enthusiasm suggest otherwise, and that basing polling results on this would produce results overly favorable to Obama. Dick Morris is one person who agrees with this analysis. I’ve heard some Democratic pundits defend the polling sampling, but I didn’t’ understand their argument, and I haven’t yet found a online explanation.

What difference does it make?

If we look at a recent (10/13/12) Washington Post poll, where Obama has a 3 point lead over Romney, we see the following weighting (at the bottom of the link):

Democrat: 35%
Republican: 26%
Independent: 33%

This gives the Democrats at +9 point advantage based on turnout. If the "correct" weighting should have be a +1 advantage for Republicans, should the corrected results show a +7 point lead for Romney? Surely, I'm over simplifying something?

I found a website http://www.unskewedpolls.com/ that claims to show unskewed results for various polls and the results are much more favorable for Romney than the published results.

Any thoughts on this subject?

There is good reason to skew a poll. Supporters of the apparent looser loose their enthusiasm and are therefore less likely to continue creating excitement, and more importantly less likely to vote if they think their cause is already lost. All of the opposites are true for the apparent winner.

So you want all polls skewed to your favorite.

I understand why one might be motivated to skew a poll. I'm eager to hear an explanation. It seems too blatantly obvious if someone wanted an underhanded way to skew the polls in a partisan manner. I'm wondering whether there is a logical explanation for why the polls aren't really skewed, even though a quick look at those demographic numbers may make it appear so.

I'm missing your point.

I want to hear an explanation for the other side. I'd like to hear someone articulate why they believe the polls aren't skewed, and why the weighting should be the way that it is.

Perhaps we should try to apply Hanlon's Razor to this and consider the possibility that the other size doesn't believe that the polls are skewed.

(10-23-2012 02:16 AM)rjschirmer Wrote: [ -> ]Perhaps we should try to apply Hanlon's Razor to this and consider the possibility that the other size doesn't believe that the polls are skewed.

I'm also open to the idea that maybe the polls aren't really skewed. I've only heard one side of the argument so far.

I made a scatter plot of Obama's lead vs apparent Democratic skew for various polls on 10/21/12. There is a definite trend. I got most of the skew data from http://www.unskewedpolls.com/, and I didn’t verify that accuracy of all of the skew data. I checked some of them, but Rasmussen and Gallop were among those I couldn’t find party affiliation weighting.

The trend line isn’t a curve fit. I simply drew a line with a slope of one, which would represent the trend that would occur of all of the Obama advantage was proportional to the party weighting. The trend does suggest that much of the difference in polling results can be explained by different weighting schemes for predicting likely voters. It’s not a perfect fit, but it does appear to fit within the margin of error.

I saw an interview with a lead polling editor at Gallop. Gallop has published polling results that is more favorable for Romney, and has received criticism from the Obama campaign and other Obama advocates. Near the end of the interview, the interviewer asked him about apparent skewing among polls, but the Gallop editor didn’t seem to address the question directly. I’m paraphrasing, but he said that “we don’t do any weighting based on political party affiliation.” He seemed to imply that their weighting is determined by other metrics, and that there may be legitimate factors which may affect likely turn out, and that their results would reflect those changes in turnout.

However, the Gallop editor’s explanation still seems problematic to me. How could Romney be winning by 6 to 7 points, when Democratic turnout still gives Obama a one point advantage? From one I’ve seen, there isn’t much crossover voting from the political parties. Therefore, a huge proportion of the independents would have been polling for Romney. If Independents were favoring Romney so strongly, I would suspect that the Republicans would also be much more energized, and this would be reflected in a much highly likelihood (and thus, weighting) for them to vote than Democrats.

Some other thoughts:

• For some of the polls which appear to be heavily skewed for Obama (6 to 9 points), either the independents are enormously favoring Romney, or a significant fraction Democrats are crossing over for Romney. Regardless, such enthusiasm for Romney among independents and Democrats would be inconsistent with such a low relative turnout among Republicans. This is also contrary to polling results which state that Republicans are more energized than Democrats for their candidate.
• Rush Limbaugh has a theory that most of these Obama-leaning polls are intentionally skewed in order to push support for Obama. He also believes that these polling agencies will attempt to salvage credibility by unskewing their polls shortly before the election so that they can claim to have been accurate. It will be interesting to watch to see whether the amount of apparent skewing declines as we near the election.
• I’m still eager to hear an explanation for the possibility that the polls aren’t really skewed. If you have a alternate theories, I’d love to hear from you.
  

EDIT (10/23/12 8:45 PM CDT): I corrected the trendline to show the correct slope of 1.

I've updated the graph with today's data. The trend is remarkable.

Most of the differences between the polls can be explained by a difference in the weighting of the political parties. All data points fit the trend except for CBS News, whose poll gives Obama a two point leads, though they report equal weighting between Republicans and Democrats.

All of the plotted data points are the published numbers that are rounded off to the nearest whole number. I may try to plot later numbers with the raw data.

This is only speculation, but if the data is unrealistically skewed and these graphs represent the skew accurately, then one might expect that the actual voter results on election day would be close to the intercept where the Apparent Democratic Skew would be zero. This would give Romney a roughly 6 point lead.

I have seen polls that show that Republicans currently have a higher enthusiasm, and thus, one might might expect to have a higher Republican turnout. If this is the case, then actual voter results on election day would to the left of the intercept, so the actual Romney lead could be higher than 6 points.

Finding useful articles about the poll skewing situation are hard to come by. The majority of the links which give an anti-skewing argument don’t address the justification for the apparent oversampling of Democrats. The Economist article “Skewed polls, Evil demon” states the following:

Quote:I don't want to spend any time at all explaining why the idea that pollsters overall are deliberately tilting their results in Mr Obama's favour is ridiculous. It's ridiculous, and the fact that a plurality of respondents in a large survey say they believe it to be true is just the latest addition to the colossal pile of evidence showing that we human beings spend our lives inhabiting an ornate palace of self-flattering delusions.

The article then makes three rambling and diffusely defined “points” which cover the bandwagon effect and generalized popular mistrust. The article seemed loaded with generalized smug condescension, but didn’t address the details of a sampling methodology.

There is also a Washington Post also has an article titled “Does the ‘skewed polls’ crowd have a point? Not really.” This article looks a little at public perception of media bias, but avoids all of the specifics of the over sampling issue.

The Atlantic has an article “Are Polls Skewed Too Heavily Against Republicans?” which is better than most, and actually discussed the over sampling issue:

Quote:Critics allege that pollsters are interviewing too many Democrats -- and too few Republicans or independents -- and artificially inflating the Democratic candidates' performance. Pollsters counter that the results they are finding reflect slight changes in public sentiment -- and, moreover, adjusting their polls to match arbitrary party-identification targets would be unscientific.

Unlike race, gender, or age, all demographic traits for which pollsters weight their samples, party identification is considered an attitude that pollsters say they should be measuring. When party identification numbers change, it's an indication of deeper political change that a poll can spot.

"If a pollster weights by party ID, they are substituting their own judgment as to what the electorate is going to look like. It's not scientific," said Doug Schwartz, the director of the Quinnipiac University Polling Institute, which doesn't weight its surveys by party identification.

Since they invoke the work “scientific,” I’ll state that a good scientist would want to use party affiliation data to look at boundary conditions and to make consistency checks. One consistency check that I’d do would be to compare party weighting to party enthusiasm, and to compare these numbers to previous elections. The Atlantic article even mentions these numbers. In 2008, when enthusiasm for Obama was very high, the exit polling show a Democratic turnout of 39%, while Republicans turned out at 32 percent, giving the Democrats a seven point advantage. For the 2004 election (Bush-Kerry), the turned was roughly equal. The 2010 mid-term elections were also said to have almost equal turnout. This year, the Republicans have been reported to have significantly higher enthusiasm. One might assume that this would mean we could expect a higher Republican turnout. A more conservative approach might assume equal numbers of Democrats or Republicans. However, the published polls tend to be assuming much higher Democratic turnout, with some polls assuming the same Democratic turn out as for the 2008 election.