Monday, July 05, 2010

Grab Your Pitchforks and Sampling Methodology Textbooks

Further to my post over the weekend, people are starting to take note of self-selection-gate. Yeah, I know it's just because Helena Guergis hasn't said anything lately but still, it's nice to see a mind numbingly boring issue that actually matters get some ink.

Below is a non-random sample of what people are saying:


"Abolishing this reliable source of useful information is little more than official vandalism."
-Montreal Gazette


"Before the federal government embarrasses itself further, it should turn back this loopy ruling."
-Edmonton Journal


"The government’s latest move to curtail the census is just another example of ideology trumping common sense."
-Toronto Star


"The outrageous decision by the federal government to eliminate the long form of the census questionnaire must be reversed immediately."
-Calgary Herald


"The findings from the census data influence everything from government spending priorities to political representation, both federally and provincially."
-Victoria Times Colonist


"Sending it out to more people doesn't solve the problem. The problem is that on a voluntary survey, people respond who feel like responding. The most vulnerable groups are the least likely to respond. So if you're interested in data about aboriginal people, if you're interested in data about recently arrived immigrants, if you're interested about the poor, the disadvantaged ... those are the kind of data that will be threatened."
-Ivan Fellegi, former Chief Statistician at Statistics Canada


"I'm just flabbergasted by the fact that they are taking the greatest source of information for the history of the country away from us."
-Gordon Watts, an amateur genealogist and co-chairman of the Canada Census Committee


"As a practising economist, the census is the single most important piece of information we get. It's absolutely crucial from a public policy point of view."
-Craig Alexander, chief economist at Toronto-Dominion Bank and president of the Canadian Association of Business Economists


"If response rates vary with the income and education levels, then you won't have a random sample of income and education levels. There is a rather large amount of evidence in the sampling design literature documenting the fact that people with lower levels of education and income have lower response rates, and so these groups will be systematically under-sampled."
-Stephen Gordon, professor of economics at l'Université Laval


"The long form is the only national source of information on aboriginal educational achievement. Without the census long form there will be no information about whether aboriginal education results are improving and no data with which objectively to assess policy alternatives."
-Social scientist Michael Mendelson


"As the organization that represents Canada's academic research community, we are deeply concerned about the disastrous consequences this will have for the scientific understanding of Canadian society, and for the ability to make informed decisions about social and economic policies."
-Canadian Association of University Teachers executive director James Turk


"Without robust Census data, it is difficult for local governments, health districts and other community service providers to respond effectively to shifting patterns of need or introduce changes – including cuts – that do the least harm or provide the greatest value for money. Indeed, it is the local level that is most hampered by this federal decision. The issue raised by cutting the Census long-form questionnaire is not just about having good information; it’s about having relevant tools for democracy."
-Armine Yalnizyan, Senior Economist Canadian Centre for Policy Alternatives


"Municipalities use the census like a GPS to navigate on-the-ground changes in our communities – to see where we need better bus service, to build affordable housing, or set up support programs for new Canadians. There’s a real concern that these changes are going to make it harder for us to meet the needs of Canadians – we need to know the federal government isn’t going to let that happen."
-Brock Carlton, CEO Federation of Canadian Municipalities


"We're not happy. Nobody on either board is happy." -Paul Jacobson, board member of Toronto and Canadian Associations of Business Economists


Hat tips...and more on this - Wherry, Selley, Tribe

10 Comments:

  • I am certainly no statistical genius, but wouldn't it be exceedingly easy to correct for the selection bias you speak of, if the bias is actually observed.

    Statistically correcting for the selection bias would correct for problems with general trends. However, the data for specific and underrepresented groups would be of lower accuracy, higher error etc. The only issue with the census data would thus be with specific information about small population groups.

    I do not know what tolerance is needed for data gathered on specific and small populations of people in canada, and I do not know if this tolerance will be achieved with a voluntary form; the former head of StatsCan seems to think an acceptable level of error will not be achieved, but I am naturally distrustful of anyone making public statements.

    Honestly, I think the effects from the voluntary form will not be as bad as we are led to believe. General trends can certainly be corrected through statistical methods. The only concern should be about the quality of specific data gathered on small and distinct populations within canada. I have not seen anyone anywhere give any information confirming that the data gathered in these circumstances will be seriously damaged, and until I do see information proving it I will temper my criticisms.

    By Blogger James McKenzie, at 10:34 PM  

  • Out of ALL of those quotes, only Michael Mendelson raises a valid point about what data might be required by policymakers which would not be available as a result of this decision.

    Still, I would contend that there are easier and cheaper ways of finding out how many Aboriginals achieve their Degrees. For one thing, Universities know this information (since they know how many students claim Status) and can provide it very easily to Government.

    For all the effort Dan has expended, there isn't one shred of justification to reverse this decision.

    By Anonymous Anonymous, at 3:25 AM  

  • Good post. Good to have all those quotes in one place. Star-phoenix has one too. Since you've gone this far, might as well update it with all negative coverage you can find, everywhere, making the point that East to West, South to North, Socialist to Capitalist, every half-sentient Cdn opposes this, this, I don't knwo, what word does one use, imbecility? But worse than that. I know: "lumpenfrackendementia".

    By Blogger Eugene Forsey Liberal, at 3:41 AM  

  • Thanks for the feedback Tony!

    By Blogger calgarygrit, at 9:30 AM  

  • I think there are two important points that come out of this that Anonymous and Harper choose to ignore.

    First, the data is important and useful for many different and diverse groups and interests, public and private/personal and business and academic. So the census, whatever the data collection methodology, is a very efficient way of satisfying diverse data needs and in a consistent way that sporadic, ad hoc data collection cannot do and certainly cannot do as efficiently or cost effectively.

    Second, the change will weaken that data collection and make it not as comparable to historical data and therefore not as useful and undermine the usefulness of the data collected because of the sampling problem Harper creates.

    If, as many defenders of Harper's boneheaded decision claim, the data is useless or there are better ways, then the logical decision would be to scrap the long form altogether. Instead, by making it voluntary, Harper has just made it less useful and less accurate. Kind of the worst of both options.

    By Blogger Ted Betts, at 9:58 AM  

  • Well said Ted.

    As the past head of StatsCan said, you either scrap the long form to save the money, or you keep the current system in place.

    I don't see any benefit at all of going to the optional method since it compromises the quality of the data.

    By Blogger calgarygrit, at 10:54 AM  

  • James:

    I think the correction you are considering would be what they do with, for example, voter intentions. They take a random sampling and then make sure the data set (i.e. those whose answers count) match against what the demographics are to balance out any higher response groups.

    That works fine and is good to ensure the demographics of the responders matches the demographics of the population.

    However, the problem with taking that approach to the census is that... the census is actually trying to measure that demographic and, specifically, any change from prior demographics. So I don't think you can correct it in the way you suggest. It's like saying you want to measure the length of a ruler to ensure accuracy by using the ruler itself.

    By Blogger Ted Betts, at 1:42 PM  

  • "Statistically correcting for the selection bias would correct for problems with general trends. However, the data for specific and underrepresented groups would be of lower accuracy, higher error etc. The only issue with the census data would thus be with specific information about small population groups."

    For groups that are represented by questions in the census, it would be possible to develop some kind of weighting scheme or a means of accounting for sampling error somewhat. It would not be ideal, but it could be done.

    However, what if census response/non-response is also driven by factors that are not captured by the census. For instance, what if Conservatives were less likely to complete the census than non-Conservatives. This would bias results in every group in a way that is very difficult to account for.

    Furthermore, usually the schemes that people use for weighting, etc. RELY on the census in the first place. This is because for a lot of data it is the closest thing we have to a population estimate.

    I think the appropriate response is to find ways to assuage the fears of those that believe their privacy is under threat without reducing the quality of the data. The fear that others will misuse the data (and it has happened) should also be a first order concern in terms of producing an accurate census.

    By Blogger hosertohoosier, at 2:13 PM  

  • I perhaps wasn't paying this quite as much attention as I maybe should have... interesting.

    By Anonymous Jacques Beau Verte, at 3:14 AM  

  • All my former college mates and friends at the university are fond of the best Papersmart research paper service suggested by educators worldwide. Learning new material as much as keeping in mind gained experience may require mental health and long lasting efforts of both the undergraduates and postgraduates. It comes as a rule that new information is learned through discussions.

    By Anonymous Vardi, at 7:31 PM  

Post a Comment

Links to this post:

Create a Link

<< Home