Saturday, January 07, 2006

Polls and Pundits

Andrew Coyne, determined to break the Michael Jordan comeback record, has been blogging for the past few weeks and even has his comments section back on.

Good thing, because Darrell Bricker from Ipsos Reid serves up this gem:


I've become a regular reader of your blog this election. Your commentary is great, plus you get a lot of smart chatter back from your readers.

Just to add my two cents, when you get a confluence of polls (with different methodologies, samples, and questions) all saying the same thing it's no longer a blip. It's a trend. This is real and it's big. That's because what has really turned against Martin this time are the fundamentals - leadership, party best on key policies, momentum, time for a change, trust, etc. For all the polling fans, these are independent variables - they CAUSE the dependent variable (vote) to change. Vote is a trailing indicator, not a leading indicator. The fundamentals started going downhill some time ago, vote is only now catching up.

Now, like everybody else, I remember what happened the last time. But, the big difference then was that the independent variables never got quite as bad for the Grits. This time they're horrible and they're still heading the wrong way. And, for all of the Conservative Nervous Nellies, you now have to learn to play with a lead!

A couple of last points. First, banning polls. What? You don't trust people with the truth? Ban them and we'll get parties leaking false polls, people who can afford to do private polls holding on to the results and using them for personal gain, and people who can't be prosecuted (eg Americans) doing them and leaking them on the Internet. That's better? Instead, we should have lots of polls so that the crappy ones become obvious. Information in a democracy is a good thing.

Second, there's no such thing as the 1 in 20 poll. It's a hypothetical concept. What a true rogue poll is is a poll in which everything was done correctly, and a result outside of the margin of error occurs. Show me a poll that was wrong, and I'll show you a mistake that a pollster made (question wording, sample design, interviewer bias, timing, etc).

Finally, I'm waiting for everybody to start crapping on our seat model again. But, instead of getting into the whys and hows at this stage, just search on my name at the Globe's website and you'll see an article I wrote for the last campaign on seat models. It'll answer all of your questions. And, I don't have much to say beyond that.

Anyway, keep up the good work! I look forward to being a regular reader.

I couldn't find that G & M article Darrell mentions so if anyone knows where it is, please link up to it in the comments section.

Also, Ekos has a very detailed breakdown of their latest poll numbers here. I've has some doubts about Ekos polls in the past and couldn't find their methodology on-line for this poll, but it's still interesting to see the breakdowns and bonus questions on this latest poll. The big sample size allows us to see some interesting regional breakdowns, especially in Quebec. (to be fair, Ekos has the Tories 4 or 5 points higher in Quebec than all the other polls released over the past two days)


  • What's most interesting is that the detailed political discussions aren't in the traditional media anymore, they're in blogs.

    There's no way Darrell Bricker would have been able to offer such a nuanced response to Andrew Coyne on middlebrow television, and newspapers are far too slow a medium.

    So, Bourque is right, what is interesting is how blogs are an important part of this election. Bloggers defined some of the issues, were the source of investigative research and continued bombshells, and the traditional gatekeepers, like the Globe and Mail's Simpsonian, are reduced to sputtering "what's going on here?".

    By Blogger Michael, at 3:09 PM  

  • Good insight from Darrell, and an excellent perspective from Michael.

    Note that the Rhinos don't have any candidates "running" in this election and, even stranger, haven't made a "press release" that introduces another of their rich set of humourous policy planks. So, to fill the gap, I invented a few that they might have considered for this election:

    Enjoy, Erik.

    By Anonymous Erik Sorenson, at 4:05 PM  

  • Unbelieveable. Check this out. Latest SES poll has Harper as better PM in Quebec.
    Harper beats Martin 27-26.

    Whoever said or thought 2006 was going to be a replay of 2004 is going to end up thoroughly lost.

    Take that Ralph Klen! Campaigns do matter.

    By Anonymous Anonymous, at 4:18 PM  

  • "Second, there's no such thing as the 1 in 20 poll. It's a hypothetical concept." Does anyone understand what Bricker means here? Sure, there's such a thing as the 1 in 20 outside-the-margin poll. At least, that's what the mathematics says. Either Bricker doesn't understand the math, or I don't get what he means.

    By Blogger Aeolus, at 10:33 PM  

  • Aeolus said: "Second, there's no such thing as the 1 in 20 poll. It's a hypothetical concept."
    Does anyone understand what Bricker means here?

    I'm not a statistician, and I don't play one on the internet, either.

    He's refering to the 1 in 20 (the one in the 19 times out of 20 margin of error) that in reality not possible without external contributories (as he mentions).

    Statistically it is possible, thus the margin of error. Reality says that you'd have to muck up big time to hit that 20th time. (emphasis on the "you", not the poll)

    Personally, I think they are valuable as bumf, indicators, and trends spotters.


    By Blogger lance, at 12:20 AM  

  • "Reality says that you'd have to muck up big time to hit that 20th time."

    No, reality doesn't say that. When it says "19 times out of 20" it means 19 times out of 20, assuming everything was done correctly.

    The 1 time in 20 poll is not some hypothetical poll. It is a statistical probability that 5% of the polls will be abberations.

    By Anonymous Joe the Iron Monkey Man who lives with cats and a woman, at 12:53 AM  

  • Bricker's statement "There's no such thing as the 1 in 20 poll. It's a hypothetical concept" is disingenuous. He claims that if you find a poll that is completely wonkers, he can find a problem with the methodology. This serves to further the value of his product in the eyes of potential customers.

    In reality, however, polling companies are already skirting the limits of polling theory, which says that if you have a large enough sample size, you can estimate the opinions of any size population with a reasonable degree of accuracy. This is because once you have a randomly selected polling sample of at least 800, the extremes of the polling sample will cancel themselves out. Those who misrepresent their opinions one way will cancel out those who do the same the other way. Those who are on the extreme range of the issue will be cancelled out by those on the other extreme range.

    The 1 in 20 poll is real, contrary to Bricker's self-serving analysis, and it simply refers to the 5% of the time when the combination of the poll's inherent limitations (the famous "plus or minus" qualification) plus the extremes make the predictive value of the poll worthless.

    What many people don't realize is that the number of people within the sample size doesn't have to change as the size of the population changes. The number of people required in the sample size to poll London, Ontario is the same as the number for China. And the key to the poll's accuracy is not the size of the sample group, but how well the sample group was chosen to ensure it reflected the range of people within the larger group.

    By Blogger Patrick, at 12:46 PM  

  • Really effective data, thank you for the article.

    By Anonymous My Wacky Friends, at 3:04 AM  

Post a Comment

Links to this post:

Create a Link

<< Home