Intended for constructive conversations. Exhibits of polarizing tribalism will be deleted.
User avatar
Posts: 6661
Joined: Sun Jul 03, 2011 2:20 pm
Location: Stepford USA


Post by jennypenny »

Scott Keeter, who's considered one of the best in the field, has a new article out at Pew on the recent polling issues ... Understanding how 2020 election polls performed and what it might mean for other kinds of survey work

Note: I'm not trying to start a political debate. I'm sharing this because -- as Keeter suggests -- the issues mentioned have implications for other issues like climate change. It's also a fascinating field.

Posts: 211
Joined: Wed Jan 04, 2017 2:26 pm

Re: Polling

Post by Loner »

Funny thing. The same thing happened here in Québec in 2012. There was a student movement/protest at the time, and the polls were predicting an easy win for the left. The right ended up winning--big time. I forget the details, but the discrepancy between the polls and the actual results was stunning, something like a few pp. A bit like in the 2016 U.S. election. IIRC, the explanation was (very unsurprisingly, anyways, to those were not part of the urban-left bubble) a mix of "Partisan nonresponse" and "‘Shy [Conservative]’ voters", to use your article's terminology.

This kind of problem has been know to pollsters for ages. I don't expect any improvement if the general ambiance continues to make it difficult to be openly conservative (or against-the-well-thinking-urban-aspirational-class, or whatever) without being publicly hounded. For sure the same problem is affecting other types of polling and survey work, and I always discount them accordingly, depending on the context.

Posts: 1822
Joined: Sun Nov 13, 2016 2:31 am

Re: Polling

Post by ertyu »

The explanation I buy best is that announced poll results aren't to predict, they're to try to sway the outcome.

Posts: 228
Joined: Tue Oct 09, 2018 4:55 am
Location: Oxbridge, UK

Re: Polling

Post by UK-with-kids »

Same in the UK - polls have repeatedly underestimated the right wing vote. It's been called "shy Tories" for a long time (Tories being the original 19th century name for the Conservative party). And the explanations are the same - demonising right wing opinions and a lower turnout amongst left wing (usually younger) voters.

In the run up to the 2015 campaign even the unbiased BBC were repeatedly telling us that there was one thing we knew for certain about such a close election - no party would win an overall majority which meant there would be a 'hung parliament'. In the event the Conservatives defied expectations and won a small majority. The prime minister David Cameron famously explained in a victory speech that "Britain and Twitter - they're not the same thing."

Site Admin
Posts: 13335
Joined: Fri Jun 28, 2013 8:38 pm
Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 73

Re: Polling

Post by jacob »

Pertinent to point out that the "shyness" keeps shifting. Otherwise consistent underestimates of the same demographics would be easy to correct for mathematically. E.g. poll says 47%, actual result is 50%, adjustment becomes 50/47=1.064. However, if the demographics is not stationary (the same from time to time) then these adjustments will both decay in time AND there's room for surprises insofar the demographics move in unexpected directions.

In short, the problem is that polling is present data with a backward looking bias (some time-series fit) whereas one really wants forward looking predictions (an actual model for voter behavior), that is, where the demographics will be. This, however, is certainly subject to political feedback mechanisms. One reason Trump won in 2016 was due to going after a demographics the Democrats was considered solid blue (the "Blue Wall" of MN, WI, MI). Trump losing Georgia in 2020 was likely due to similar miscalculation.

Add: Roughly the same problem exists in trying to predict the market algorithmically. If you're a small fish (like a polling operation) you can only act passively in the sense that you can not influence the market. You can only ride it. If you're a big fish (like a political party) you can take positions that are large (and thus public) enough that the market is influenced. You can then ride the other market players as well. This means that if you're a little fish, you really need to understand the big fish as well or better than they understand themselves---since there are too few big fish for statistics to provide a decent model (the population size is too small), statistics is not the best tool for this. If you're a big fish you just need a lot of money or influence. Ponder this until it sinks in.

Posts: 401
Joined: Fri Mar 13, 2020 12:54 am

Re: Polling

Post by nomadscientist »

Given the huge change in personal behaviour and voting mechanisms, it's amazing polls were remotely accurate. All of the training data suddenly became invalid.

George the original one
Posts: 5372
Joined: Wed Jul 28, 2010 3:28 am
Location: Wettest corner of Orygun

Re: Polling

Post by George the original one »

I come to the conclusion that people lie terribly to polltakers, particularly in this age of alternate facts. This morning, for instance, a Microsoft poll asking about education had results saying 22% of people had graduate degrees while only 23% had bachelor degrees. 4 million respondents, so a very large sample!

Hristo Botev
Posts: 1326
Joined: Tue Jul 17, 2018 3:42 am

Re: Polling

Post by Hristo Botev »

DW makes a not insubstantial side income taking professionally-focused surveys, based on her credentials. Unless someone is paying me a good bit of money, I can't imagine taking the time to answer someone else's questions (for one, I'd be too tempted to nit-pick and argue with each survey prompt). And if it's a political survey, you couldn't pay me to take one. FWIW, consumer surveys are a big part of trademark infringement and false advertising cases, and you only have to be involved in a handful of those to realize that no matter how well the survey is crafted, there are a 100 very legit ways to tear it down.

Posts: 3380
Joined: Wed Apr 02, 2014 7:46 pm

Re: Polling

Post by IlliniDave »

I also am about fed up with polls and surveys and refuse to participate. The questions are leading and often push you into an inaccurate answer. No doubt a lot of people follow along with what they think most other people like, so there's at least a temptation to fit a poll to a narrative. Come to think of it, there is very little in the public sphere I trust these days. And you kids get off my darn lawn!

Gilberto de Piento
Posts: 1698
Joined: Tue Nov 12, 2013 10:23 pm

Re: Polling

Post by Gilberto de Piento »

IlliniDave wrote:
Tue Nov 17, 2020 6:25 pm
The questions are leading and often push you into an inaccurate answer.
"Push polling is a negative campaigning technique, typically conducted by telephone, used to influence voters by asking specific questions about an issue or a candidate."


Posts: 317
Joined: Sun Jul 01, 2018 11:45 am

Re: Polling

Post by bostonimproper »

They talked about this in a recent 538 podcast. I found their argument, that Trump consistently underperformed other down ballot Republicans, to be convincing that we're looking at demographic response problem rather than "shy Trump voters." Akin to the "shy Tory" you can also imagine the converse: over-enthusiastic response to polls by strongly D individuals, tilting them too favorably toward left partisanship.

Posts: 34
Joined: Sun Jun 24, 2018 4:27 am

Re: Polling

Post by plow_2 »

User avatar
Posts: 2221
Joined: Tue Jul 10, 2012 5:15 pm
Location: Orange County, CA

Re: Polling

Post by Sclass »

jacob wrote:
Mon Nov 16, 2020 11:41 am

In short, the problem is that polling is present data with a backward looking bias (some time-series fit) whereas one really wants forward looking predictions (an actual model for voter behavior), that is, where the demographics will be.

Add: Roughly the same problem exists in trying to predict the market algorithmically.....Ponder this until it sinks in.
A simpler way of saying this is polling sucks.

Pro Tip - don’t listen to the polls. Get your prediction from the sports betting pools.