Forums

I was doing some research and came across your thread - and figured I might give you some feedback disputing your allegation that pollsters are "oversampling" Democrats.

First, most polling outfits don't "weight" for party affiliation.

You stated (in your first post):

Quote:If we look at a recent (10/13/12) Washington Post poll, where Obama has a 3 point lead over Romney, we see the following weighting (at the bottom of the link):

Democrat: 35%
Republican: 26%
Independent: 33%

But that is inaccurate. What was at the bottom of that link was a question where they asked the respondents how they self-identified.

The poll didn't weight them that way, that's just what they were.

Pollsters do statistically adjust (weight) for certain factors: region, age, ethnicity, etc. - but most don't make adjustments for party identification.

Pew has a good rundown of this: Party Affiliation and Election Polls

And they state their case plainly:

Quote:To put it simply, party identification is one of the aspects of public opinion that our surveys are trying to measure, not something that we know ahead of time like the share of adults who are African American, female, or who live in the South. Particularly in an election cycle, the balance of party identification in surveys will ebb and flow with candidate fortunes, as it should, since the candidates themselves are the defining figureheads of those partisan labels. Thus there is no timely, independent measure of the partisan balance that polls could use for a baseline adjustment.

To make the argument that a particular poll is "skewed" because the party identification is "incorrect" (i.e. hasn't been weighted) introduces exactly the bias you (or someone making that argument) has argued against.

Of course this isn't true for all pollsters. Some do weight by party ID (Rasmussen, in particular). And that does lead to a problem. Suddenly you are comparing apples and oranges.

If you look at Rasmussen (which does weight for party ID), and compare them to a bunch of pollsters that don't weight for party ID, you're going to think that most polls are oversampling Democrats - when in actuality they are simply reporting what their respondents are telling them.

It's telling that your graph(s) look(s) eerily similar to this one:



(From this article: Why weighting polls for party identification is wishful thinking )

But as the author of that piece states:

Quote:The graph is supposed to show that the more there are survey respondents who identify as Democrats, the bigger the Obama lead. Your eye is probably drawn to the relationship between Rasmussen (at bottom left) and every other pollster. Join the dots, and you would seem to have found a fairly straightforward relationship: more Democratic-identified respondents giving a more pro-Obama poll result.

...

The problem is that the relationship is being driven by one datapoint. Take Rasmussen out of the chart, and you get this graph:

Which as the author states, tends to reinforce the fact that there is no real relationship.

And lastly, the final argument against supposed party ID skewing comes from this article:

No, The Polls Aren't Oversampling Democrats

The author sums it up:

Quote:When the discussion is framed as “how could there be more Democrats than 2008,” it’s easy to see how the “polls are wrong” argument gained currency. But since there are actually fewer Democrats in the polls than 2008, the better question is whether it’s possible for Republicans to have lost self-identified adherents over the last four years, as well. This discussion should be framed by the recognition that the polls are pretty accurate: When assessing whether to “buy” the polls, it’s not a question of whether they match your expectations. The possibility that Republicans are moving into the independent column is an appealing explanation: it contradicts the false assumption that the polls assume a 2008-esque number of Democrats; it reconciles Romney's strength with independent voters and the Democratic-edge in party-ID; and, it happens to be consistent with the polls.

Go check it out for yourself at: Party Identification - Adults

It pretty well backs up the final author's point.


I think it's interesting that so many people are, in effect, raging against the machine right now. But if you are a realist, then you have to agree that pollsters do what they do to make a profit. Most of them release these political polls on their own dime (i.e. they aren't commissioned, and thus there is no customer, per se). They are basically giant commercials for their primary business (polls commissioned for profit from businesses) or for their university (visibility). Thus they have a huge incentive to be accurate. And that alone should give you pause when considering whether a large percentage of pollsters would do something detrimental to that goal.

If someone is holding out hope that these polling firms are all so out of the loop, and thus biased against one candidate over another, then that probably says more about the biases of that individual than it does about the biases of any set of polling firms.

From one point of view, polling is a political tool to influence the decision making of a variety of audiences: first the money donors, second the voters. It is in the polling party's interest to see positive results.

TJkelli,

Thank you for your contribution. I appreciate your polite input.

I don’t think that it really matters whether they explicitly weighted by party at the outset, or they merely used another method to determine party turnout. The reported polling results still reflect assumed party turnout. Are the assumed turnout values reasonable? I think that a reasonable and a valid consistency check is to look at the polls’ assumed party turnout and compare it to other metrics and benchmarks. If the turnout numbers don’t look reasonable, then maybe there is a problem with the model that was used to determine turnout.

I’d say that before the debates, the poll predictions were more uncertain for a number of reasons. Right leaning votes tended to have more ambiguous feelings about Romney. The Guardian article was published on Sept 17, 2012, so the graph data was from polling before the debates. The data from my graphs were from last week, which was after the debates. The more recent polls (like the post-debate graphs I posted) appear to have a more robust trend, and aren’t dependent on a single data point. If I have time, I’ll try to generate an undated graph.


A premise to the view that the polls are not skewed (I’m using the word “skew” to include indirect skewing based on turn out models), is that Republican party affiliation is not popular, and that fewer people are self identifying themselves as Republicans. The remaining self-identified Republicans are still reliably voting Republican at a roughly 90% rate like Democrats are reliably voting Democrat at roughly 90%. The pool of independent voters now includes a population of people who previously self-identified as Republican, so it’s not surprising that the independents are leaning towards Romney. The Romney-leaning independents cancel out the shortfall of self-identified Republican voters, so the net totals reveal a closely matched race.

HOWEVER:
If I recall correctly, the current party turnouts for the Obama-leaning polls have turnout numbers that roughly match the turnout for the 2008 election. If party affiliation hasn’t changed since then, then this would be plausible. However, if I recall correctly, the 2010 elections had roughly equal numbers for Republican and Democrat turnout. If so, then the premise that the Republican Party has lost popularity and that the number of self-described Republicans has significantly dropped may be invalid. By “significantly” dropped, many of the polls show a drop from ~37% to ~30%, which would mean a drop of ~19% of party membership. That seems like a lot. Also, I understand that there is currently a mismatch of party enthusiasm, where the Republicans have higher enthusiasm, which would be expected to raise the relative turnout of Republicans.

You make a good point about the pollsters having an incentive to be accurate. However, I don’t think that the polling community is any more immune to political biases than the study of economics. According to the Limbaugh hypothesis, the polling firms may drop their skewing at the last minute so that their final polls better reflects reality and they can claim that their polls were accurate the whole time. I’m not sure that I’m that cynical, but I’ll be watching. Wasn't there a Gallop article last week which explicitly measured expected voter turnout by party, they reported a Republican turnout that matched Democrat turnout? (Sorry, I can't find the link now).

One nice thing about this issue is that we’ll hopefully know by Tuesday.

Here is a plot of Saturday's (11/3/12) data for Ohio. There's not much of a trend to plot, but almost all of them are assuming a Democratic turnout advantage of six to nine points. That seems rather big. I really wanted to include Rasmussen, but I couldn't find the party turnouts for their Ohio poll. As given, the polls state that Obama has a slight, but definite lead in Ohio. However, if the assumed turnouts are unrealistic for Democrats, then Romney could very well win this state.



Does anyone know what the party turnouts were in Ohio for 2004, 2008, and 2010? I'd love to know those numbers as boundary conditions. I thought that I heard a TV news speaker this morning mention that Ohio has more registered Democrats than Republicans, so maybe we can assume that Ohio should legitimately have Democratic weighting advantage that is higher than the national average, but that isn't telling me much.


I've also wondered about the pollster's explanations if the turn out models turn out to have be unrealistically favorable towards the Democrats. The Bradley Effect (Forbes, "Does the Bradley Effect Overrate Obama in the Polls?," 9/19/12) might be one explanation given. Here is a summary from Wikipedia

Quote:The Bradley effect, less commonly called the Wilder effect,[1][2] is a theory proposed to explain observed discrepancies between voter opinion polls and election outcomes in some United States government elections where a white candidate and a non-white candidate run against each other.[3][4][5] The theory proposes that some voters will tell pollsters they are undecided or likely to vote for a black candidate, while on election day they vote for the white candidate. It was named after Los Angeles Mayor Tom Bradley, an African-American who lost the 1982 California governor's race despite being ahead in voter polls going into the elections.

The Bradley effect might explain overall vote percentages, but I'm not sure if it would affect Democratic voters disproportionally, which would seem to be needed to explain the difference in party turnout.

I just uploaded an updated graph for the national polls. I only included polls that extended into November. This excluded the Gallup poll.

The CNN poll, which showed a tied race, is quite an outlier, with a Democratic advantage of 12 points. Drudge still list this as a "D+11" poll, though, I looked at the actual polling data, which shows that it is really a D+12 poll. Page 29 of the CNN report states:

Quote:Among those likely voters, 41% described themselves as Democrats, 29% described themselves as Independents, and 30% described themselves as Republicans.

A recent Gallup report states that there was actually a R+1 advantage for likely voters (sorry, I don't have a link). If I've interpreted that correctly, then one might extrapolate a Romney lead of +4 based on the recent polling results. Then, again, this whole hypothesis could be completely wrong.

Here’s an updated chart showing the final polls from this morning. It’s similar to yesterday’s chart.

• Monmouth/SurveyUSA/Braun was added (tied, D+4)
• IBD/TIPP was added (Obama +1, D+7)
• ABC/WashPost changed its Democratic weighting advantage by 2 from +4 to +6, and not surprisingly, its Obama lead increased by 2, from +1 to +3. This was driven by a reported increase in expected Democratic turnout from 33 to 35, while the Republican turnout stayed at 29.
• Gallup (with a Romney lead of +1) was excluded from the graph because I could not find their party turnout numbers.
• Again, the CNN poll with an expected Democratic advantage of +12 seems very odd. Everyone keeps reporting the CNN poll as a D+11, though the numbers indicate a D+12 advantage. Maybe there's a round off error somewhere, or maybe an error in an initial report keeps getting propagated.
  

Per a Washington Post blogger:

https://twitter.com/TheFix/status/266034291950112770

Quote:New party ID numbers in national exits: 38 D, 32 R, 30 Indie. D+6. So, the polls were right.

Obviously they are preliminary numbers, but they kind of prove the point. Most of the pollsters (sans Rasmussen) were right.

Apparently there was something to the way the sampling was done to reflect the anticipated turnout with Democrats turning out in greater numbers than before. Did the "independent" voters also sing towards Obama in greater than anticipated numbers? I haven't had a chance to really look at the numbers from last night, but I do have a lot of questions.

(11-07-2012 02:36 PM)tjkelli Wrote: [ -> ]Most of the pollsters (sans Rasmussen) were right.

It does appear that way. I've spent a little time reading explanations from pollsters and pundits who were wrong, but I haven't seen a good coherent explanation. Michael Barone mentioned he based his opinions on what he referred to as the "fundamentals," which has been a reliable conventional wisdom, but that these election results didn't quite follow them. I've heard lots of opinions, but they are mostly speculation. I'd like to think that we'll eventually have more real data to digest.

Tjkelli, you are welcome to stick around and participate in other topics. Just take a look at the FAQ and see if this is the type of interaction that you'd like to be a part of.

(11-08-2012 06:03 AM)rjschirmer Wrote: [ -> ]Apparently there was something to the way the sampling was done to reflect the anticipated turnout with Democrats turning out in greater numbers than before. Did the "independent" voters also sing towards Obama in greater than anticipated numbers?

I've heard conflicting reports and I haven't had a chance to see what is correct. I've heard that turnout for both was roughly the same as 2008 for both parties. I've also heard that Republican turnout was less than in 2008. I believe that independents tended to swing towards Romney, but I haven't confirmed it.