Ipsos Reid’s flexible truth
Posted on March 26th, 2012 –
Sarah just got off a phone call with Ipsos Reid about the home care she receives as part of her MS care (specifically the companies and individuals she deals with), and about halfway through it became apparent that not only was their poll not balanced, it was in fact extremely skewed and primed for an awful lot of abuse. The questions were so clearly biased that the only conclusion I could come to is that they (some level of government) are getting ready to reduce a services for the disabled and needed a justification for such an unpopular move.
A sample of one of the misleading questions is (somewhat paraphrased…it was a long poll): “Thinking about the past two months, how often would you describe the care provided by your therapist as helpful or adequate? Always, sometimes, never?”
Sarah asked for clarification since she’s only seen the physiotherapist for all of about 7 hours, and certainly not in the last two months. In fact, her agencies are so chronically understaffed and over-scheduled that she has only really seen most of them once in the past 6 months.
“This only applies to in the last two months”, replied the pollster. Well, being honest she had to say “never”. Of course, this “never” was only to apply to the last two months, but there was no way to clarify the answer, and the pollster’s impatience was evident on the other end of the phone (like I said, long poll). There was not even a way to specify that “never” didn’t actually mean “never” according to their own questionnaire. And the following queries continued to be within the two month period without this time frame being included in the questions, so that taken in isolation, the answers would literally be interpreted as never.
There were other questions that were equally, if not more misleading, essentially implying that the physiotherapist has been doing an awful job (“never” showing up, providing the “poorest level” of service, etc.), except that there were no options to clarify when, or even if the physiotherapist was here. “Only in the last two months”, was the constant reply, and Sarah wanted to answer as honestly as possible. I too can attest to the fact that the therapist was dedicated, caring, and a genuine helpful person, something that the Ipsos Reid poll twisted into literally the opposite thing. The options were always always or never, worst or best, etc. etc., with no way to qualify them.
Maybe if the remainder of the questionnaire was similarly misleading I might not have been so taken aback, but immediately afterward the section on the physiotherapist, a “Not Applicable” option suddenly became available in the list of possible answers. So the one service that is the least available and genuinely most useful is presented by the poll as a complete waste of time and money, and with an obvious and glaring bias when compared against the other home care services being reviewed. While Sarah could answer “not applicable” at some points, there were other questions where her answers were forced to be applicable even when they couldn’t possibly be.
To put this into context, imagine being called by a pollster and being asked, “How severe was the crime for which you were most recently arrest? Very, somewhat, not very?” What, you’ve never been arrested? Well that’s not an option (and hurry up and answer the question, we have other people to poll).
Now consider that such incredibly biased data were later used to justify new police powers, the building of mega jails, and the suspension of your civil rights. Or maybe it’s used to underpin the cutting healthcare services. Or maybe it’s being used to raise taxes, ban driving, or whatever. It’s not not a far leap.
I’m not sure what can be done about this except to share our polling stories online to expose such bias and to take every questionnaire result with a big hunk of salt. At the very least, when being presented with the results of a poll, we should demand to see the whole question, the whole set of answers, and most importantly be critical of what (and why!) things may have been left out.
... with pictures
... with sounds
... with videos
... with opinions
... that's shorter
... that's longer
March 27th, 2012 11:55 am
Hi Patrick,
I’m a journalist in Toronto, I’d like to talk to you about your blog. Can you send me an email, balkissoon at gmail? Thanks very much,
Denise