Poll Smoking
The Canadian Press' Joan Bryden penned a fascinating article over the weekend on the latest poll numbers.
The latest poll numbers that show the Tories on cusp of a majority? Or the ones before that showed the Tories "losing momentum"?
No, Bryden took a look at how valuable the cavalcade of horse race polls we're subject to actually are:
I'll admit that I'm as addicted as anyone out there. My reaction to e-mail poll updates from Nik Nanos is pavlovian. I look forward to Thursday, not because of Must-See-TV, but because of the latest Ekos numbers. But, hey, at least I recognize this isn't healthy...well, I'm at least 95% confident it isn't.
One of the reasons it may not be healthy is the quality of the data, something Bryden touches on in her article, and Pundits Guide explores in a bit more depth in her follow up post. But I think the real problem insn't the polls - rather, it's how they're being reported. Consider the following two leads...on the exact same fictitious poll (which, let's say, has the Tories down 2 and Liberals up 1):
The above illustrates the most common problem with the way polls are covered in the press - the need to make something out of nothing. Outside of elections or exceptional circumstances like the coalition crisis, people just don't pay a lot of attention to politics. After all, Canadians are generally more interested in Justin Bieber than Bev Oda - and who can blame them?
Because of this, the political landscape isn't going to shift over the course of a week. When I post my "Poll Soup" updates every month or two, there's rarely more than a 1% or 2% swing. Sure, the numbers move when important things happen (like Harper playing the piano, or taking a longer-than-usual Christmas vacation) but with 80 to 100 new polls out every year, you need 80 to 100 stories. And there just aren't 80 to 100 stories.
So what happens when there is no story in the overall numbers? People will often focus in on the age, gender, or regional splits. But hell, even huge sample Ekos polls have a 10% margin of error in the maritimes and 12% on the prairies. And if we're talking about 6 regions and seven or eight polls released every month, you're going to get a few regional shifts outside the MOE just by chance (that's what the "19 times out of 20" disclaimer is all about).
Which brings up the next problem - sensational polls get the most air. We see this especially during election campaigns, when you get one or two polls that shock everyone. The problem is, while we tend to focus on the most extreme, those are likely the least accurate.
Think about it. Say the Tories are up 35-30 and four new polls come out. Just because of the laws of sampling and differences in question wording, we're going to get some variety. So if the polls come back 35-31, 34-29, 35-30, and 38-28, which one do you think is going to get the most attention?
On top of that, with 80 to 100 polls a year, you're going to get 4 or 5 that fall outside of MOE because of that one time out of twenty. Yet those are the ones that will get people talking.
The end result of this is politics being looked at the same way a manic hockey fan follows his favourite team. They lose 3-2 in a shoot-out and he's on talk radio wanting the coach canned and the goalie traded. The next night they win 5-4 and he cancels his June vacation so as not to miss any playoff games. In both horse race polls and hockey, a trend over time does mean something - but, even then, it's all fairly meaningless until the playoffs (or the election) starts.
In the case of the hockey fan, it's all harmless. But when you get a situation where the poll numbers begin driving the narrative, that's a problem. It takes away from important stories and anything that happens is looked at through the lens of the horse race. The exact same speech or event will be interpreted in two completely different ways if it comes from a leader who's up 5 or down 5 in the latest poll.
This isn't an attempt to downplay the latest round of bad polling numbers - like I said above, if a trend emerges, it shouldn't be ignored. And we've had three polls out this week with the CPC in a double digit lead - that's something, even if it turns out to be a shortly lived and we're back to the usual 6 or 7 point gap in a few weeks.
I'm not advocating we ban or ignore polls. We're better off with well informed voters and polls provide information. Plus, hey, it's a lot of fun.
But it's likely worth taking a hard look at how polls are reported.
The latest poll numbers that show the Tories on cusp of a majority? Or the ones before that showed the Tories "losing momentum"?
No, Bryden took a look at how valuable the cavalcade of horse race polls we're subject to actually are:
Pollsters advise voters to be wary of polls ahead of possible spring vote
OTTAWA - Canada's notoriously competitive pollsters have some surprisingly uniform advice about the parade of confusing and conflicting numbers they're about to toss at voters ahead of a possible spring election: Take political horse race polls with a small boulder of salt.
"Pay attention if you want to but, frankly, they don't really mean anything," sums up Andre Turcotte, a pollster and communications professsor at Carleton University.
He has even more pointed advice for news organizations that breathlessly report minor fluctuations in polling numbers: "You should really consider what is the basis for your addiction and maybe enter a ten-step program."
I'll admit that I'm as addicted as anyone out there. My reaction to e-mail poll updates from Nik Nanos is pavlovian. I look forward to Thursday, not because of Must-See-TV, but because of the latest Ekos numbers. But, hey, at least I recognize this isn't healthy...well, I'm at least 95% confident it isn't.
One of the reasons it may not be healthy is the quality of the data, something Bryden touches on in her article, and Pundits Guide explores in a bit more depth in her follow up post. But I think the real problem insn't the polls - rather, it's how they're being reported. Consider the following two leads...on the exact same fictitious poll (which, let's say, has the Tories down 2 and Liberals up 1):
Conservatives Miss the Net on Arena Funding
An exclusive Calgary Grit poll shows the gap between the Conservatives and Liberals narrowing, as Stephen Harper tries to cope with the fallout from his decision to not fund the new Quebec City arena. Harper's Conservative are down 3 points in Quebec, with the NDP the largest beneficiaries. Still, the decision does appear to be a popular one in Harper's home province of Alberta, where the Conservatives have opened up a 40 point lead.
No Major Shifts in Public Opinion
An exclusive Calgary Grit poll shows the Canadian political landscape relatively unchanged over the past month, with only minor shifts within the margin of error. This is not expected to have any impact on the timing of the next election since, after all, political parties aren't going to base their decision on a single media poll.
The above illustrates the most common problem with the way polls are covered in the press - the need to make something out of nothing. Outside of elections or exceptional circumstances like the coalition crisis, people just don't pay a lot of attention to politics. After all, Canadians are generally more interested in Justin Bieber than Bev Oda - and who can blame them?
Because of this, the political landscape isn't going to shift over the course of a week. When I post my "Poll Soup" updates every month or two, there's rarely more than a 1% or 2% swing. Sure, the numbers move when important things happen (like Harper playing the piano, or taking a longer-than-usual Christmas vacation) but with 80 to 100 new polls out every year, you need 80 to 100 stories. And there just aren't 80 to 100 stories.
So what happens when there is no story in the overall numbers? People will often focus in on the age, gender, or regional splits. But hell, even huge sample Ekos polls have a 10% margin of error in the maritimes and 12% on the prairies. And if we're talking about 6 regions and seven or eight polls released every month, you're going to get a few regional shifts outside the MOE just by chance (that's what the "19 times out of 20" disclaimer is all about).
Which brings up the next problem - sensational polls get the most air. We see this especially during election campaigns, when you get one or two polls that shock everyone. The problem is, while we tend to focus on the most extreme, those are likely the least accurate.
Think about it. Say the Tories are up 35-30 and four new polls come out. Just because of the laws of sampling and differences in question wording, we're going to get some variety. So if the polls come back 35-31, 34-29, 35-30, and 38-28, which one do you think is going to get the most attention?
On top of that, with 80 to 100 polls a year, you're going to get 4 or 5 that fall outside of MOE because of that one time out of twenty. Yet those are the ones that will get people talking.
The end result of this is politics being looked at the same way a manic hockey fan follows his favourite team. They lose 3-2 in a shoot-out and he's on talk radio wanting the coach canned and the goalie traded. The next night they win 5-4 and he cancels his June vacation so as not to miss any playoff games. In both horse race polls and hockey, a trend over time does mean something - but, even then, it's all fairly meaningless until the playoffs (or the election) starts.
In the case of the hockey fan, it's all harmless. But when you get a situation where the poll numbers begin driving the narrative, that's a problem. It takes away from important stories and anything that happens is looked at through the lens of the horse race. The exact same speech or event will be interpreted in two completely different ways if it comes from a leader who's up 5 or down 5 in the latest poll.
This isn't an attempt to downplay the latest round of bad polling numbers - like I said above, if a trend emerges, it shouldn't be ignored. And we've had three polls out this week with the CPC in a double digit lead - that's something, even if it turns out to be a shortly lived and we're back to the usual 6 or 7 point gap in a few weeks.
I'm not advocating we ban or ignore polls. We're better off with well informed voters and polls provide information. Plus, hey, it's a lot of fun.
But it's likely worth taking a hard look at how polls are reported.
posted by calgarygrit at
6:00 p.m.
7 Comments:
One thing is probably certain. Any election in the next few months would change exactly nothing of substance. The likelihood of another CPC minority is high, followed by the (much) slimmer possibilities of a CPC majority or LPC minority.
So why are we having an election, other than to satisfy the media lust for one?
By
Möbius, at 8:01 p.m.
The end result of this is politics being looked at the same way a manic hockey fan follows his favourite team.
Bingo.
It boggles me how someone can vote for the same party every single election, agree with their party on every single issue, and feel the (often similar) positions of the other parties are without absolutely any value.
The only comparison is the way people follow professional sports.
By
Robert Vollman, at 8:13 p.m.
Agreed. I know people who proudly vote Liberal without a clue of party policies, relative to the opposition.
Same with conservatives, who don't actually care if the party is behaving conservatively.
At one time or another, I've voted (federally and provincially) CPC, PC, Liberal, Reform, NDP, and Green! I still haven't made up my mind.
By
Möbius, at 8:23 p.m.
These latest series of Polls don't matter.
The message from the left is not resonating. It is not credible.
The lesson being ignored for the Liberals is advertising does work when the message is credible.
The Conservatives advantage is clear: largest number of donors and subsidies are able to purchase a national media buy that hits prime time.
Their message is running against the silence from the coalition on television.
Closing the gap in donors and funds should have been the highest priority for the political parties.
Having the best ideas don't matter if you can't reach the audience or convince them.
By
CanadianSense, at 6:23 a.m.
Polls bore the heck out of me.... none of it means anything.
I admit I watched the Kerry vs Bush numbers a lot during that election, but otherwise - it's all fuss over nothing which means nothing and can't mean anything.
By
Jacques Beau Vert, at 11:43 a.m.
Canada is also have high ratings of chain smokers according to survey.
By
cowboy boot, at 10:21 a.m.
It can't work in reality, that is exactly what I suppose.
By
www.palencia-3d.com, at 11:24 a.m.
Post a Comment
<< Home