Page 1 of 1
Posted: Mon Nov 16, 2020 9:36 am
Scott Keeter, who's considered one of the best in the field, has a new article out at Pew on the recent polling issues ... Understanding how 2020 election polls performed and what it might mean for other kinds of survey work
Note: I'm not trying to start a political debate. I'm sharing this because -- as Keeter suggests -- the issues mentioned have implications for other issues like climate change. It's also a fascinating field.
Posted: Mon Nov 16, 2020 10:24 am
Funny thing. The same thing happened here in Québec in 2012. There was a student movement/protest at the time, and the polls were predicting an easy win for the left. The right ended up winning--big time. I forget the details, but the discrepancy between the polls and the actual results was stunning, something like a few pp. A bit like in the 2016 U.S. election. IIRC, the explanation was (very unsurprisingly, anyways, to those were not part of the urban-left bubble) a mix of "Partisan nonresponse" and "‘Shy [Conservative]’ voters", to use your article's terminology.
This kind of problem has been know to pollsters for ages. I don't expect any improvement if the general ambiance continues to make it difficult to be openly conservative (or against-the-well-thinking-urban-aspirational-class, or whatever) without being publicly hounded. For sure the same problem is affecting other types of polling and survey work, and I always discount them accordingly, depending on the context.
Posted: Mon Nov 16, 2020 11:00 am
The explanation I buy best is that announced poll results aren't to predict, they're to try to sway the outcome.
Posted: Mon Nov 16, 2020 11:33 am
Same in the UK - polls have repeatedly underestimated the right wing vote. It's been called "shy Tories" for a long time (Tories being the original 19th century name for the Conservative party). And the explanations are the same - demonising right wing opinions and a lower turnout amongst left wing (usually younger) voters.
In the run up to the 2015 campaign even the unbiased BBC were repeatedly telling us that there was one thing we knew for certain about such a close election - no party would win an overall majority which meant there would be a 'hung parliament'. In the event the Conservatives defied expectations and won a small majority. The prime minister David Cameron famously explained in a victory speech that "Britain and Twitter - they're not the same thing."
Posted: Mon Nov 16, 2020 11:41 am
Pertinent to point out that the "shyness" keeps shifting. Otherwise consistent underestimates of the same demographics would be easy to correct for mathematically. E.g. poll says 47%, actual result is 50%, adjustment becomes 50/47=1.064. However, if the demographics is not stationary (the same from time to time) then these adjustments will both decay in time AND there's room for surprises insofar the demographics move in unexpected directions.
In short, the problem is that polling is present data with a backward looking bias (some time-series fit) whereas one really wants forward looking predictions (an actual model for voter behavior), that is, where the demographics will be. This, however, is certainly subject to political feedback mechanisms. One reason Trump won in 2016 was due to going after a demographics the Democrats was considered solid blue (the "Blue Wall" of MN, WI, MI). Trump losing Georgia in 2020 was likely due to similar miscalculation.
Add: Roughly the same problem exists in trying to predict the market algorithmically. If you're a small fish (like a polling operation) you can only act passively in the sense that you can not influence the market. You can only ride it. If you're a big fish (like a political party) you can take positions that are large (and thus public) enough that the market is influenced. You can then ride the other market players as well. This means that if you're a little fish, you really need to understand the big fish as well or better than they understand themselves---since there are too few big fish for statistics to provide a decent model (the population size is too small), statistics is not the best tool for this. If you're a big fish you just need a lot of money or influence. Ponder this until it sinks in.
Posted: Mon Nov 16, 2020 1:40 pm
Given the huge change in personal behaviour and voting mechanisms, it's amazing polls were remotely accurate. All of the training data suddenly became invalid.
Posted: Tue Nov 17, 2020 2:42 pm
I come to the conclusion that people lie terribly to polltakers, particularly in this age of alternate facts. This morning, for instance, a Microsoft poll asking about education had results saying 22% of people had graduate degrees while only 23% had bachelor degrees. 4 million respondents, so a very large sample!
Posted: Tue Nov 17, 2020 2:47 pm
DW makes a not insubstantial side income taking professionally-focused surveys, based on her credentials. Unless someone is paying me a good bit of money, I can't imagine taking the time to answer someone else's questions (for one, I'd be too tempted to nit-pick and argue with each survey prompt). And if it's a political survey, you couldn't pay me to take one. FWIW, consumer surveys are a big part of trademark infringement and false advertising cases, and you only have to be involved in a handful of those to realize that no matter how well the survey is crafted, there are a 100 very legit ways to tear it down.
Posted: Tue Nov 17, 2020 6:25 pm
I also am about fed up with polls and surveys and refuse to participate. The questions are leading and often push you into an inaccurate answer. No doubt a lot of people follow along with what they think most other people like, so there's at least a temptation to fit a poll to a narrative. Come to think of it, there is very little in the public sphere I trust these days. And you kids get off my darn lawn!
Posted: Tue Nov 17, 2020 10:24 pm
IlliniDave wrote: ↑
Tue Nov 17, 2020 6:25 pm
The questions are leading and often push you into an inaccurate answer.
"Push polling is a negative campaigning technique, typically conducted by telephone, used to influence voters by asking specific questions about an issue or a candidate."
Posted: Wed Nov 18, 2020 7:07 am
They talked about this in a recent 538 podcast. I found their argument, that Trump consistently underperformed other down ballot Republicans, to be convincing that we're looking at demographic response problem rather than "shy Trump voters." Akin to the "shy Tory" you can also imagine the converse: over-enthusiastic response to polls by strongly D individuals, tilting them too favorably toward left partisanship.
Posted: Fri Nov 20, 2020 4:55 am
Posted: Fri Nov 20, 2020 9:17 am
jacob wrote: ↑
Mon Nov 16, 2020 11:41 am
In short, the problem is that polling is present data with a backward looking bias (some time-series fit) whereas one really wants forward looking predictions (an actual model for voter behavior), that is, where the demographics will be.
Add: Roughly the same problem exists in trying to predict the market algorithmically.....Ponder this until it sinks in.
A simpler way of saying this is polling sucks.
Pro Tip - don’t listen to the polls. Get your prediction from the sports betting pools.