Confidence in Canadian Polling
November 3, 2017
In a considerable number of recent voting experiences, in Canada and in other jurisdictions, questions have been raised about the accuracy of polls, and the professionalism of pollsters.
In our latest nationwide study, we decided to take a look at confidence in polling has held up, asking questions about the accuracy of polls, the professional ethics of pollsters and the way in which the media reports polls.
The results paint a picture that is neither all that flattering nor intensely critical.
Only 36% say the professionalism and ethics of pollsters are excellent or good. But only 11% say poor or very poor. The rest (41%) give pollsters an “acceptable” grade.
Slightly weaker numbers are found for the “accuracy of polls”: 30% say excellent or good, while 19% say accuracy is poor.
On the way media cover polls, responses are in the same ballpark: 30% say excellent or good, 18% poor, and the plurality say “acceptable”.
Looking at some of the subgroups of the population reveals:
- Skepticism about the professionalism of pollsters is higher than average among Albertans. 17% of Albertans give pollsters a poor grade. Quebecers are at the opposite end of the spectrum, with 52% giving an excellent or good rating.
- When it comes to the accuracy of polls, differences are more muted, although Conservative voters are more likely than others to give a poor grade (28%).
- Finally, on the way the media cover polls, Albertans are particularly disappointed, with only 18% offering an excellent or good rating, and 29% offering a poor or very poor opinion. Conservative voters are also 9 points more likely than average to give the media a poor rating for the way they cover polls.
UPSHOT
According to Bruce Anderson: “The results may offer comfort to pollsters in that the public has not completely soured on the role of polling in politics – but this is hardly a ringing endorsement of either the accuracy or the professional ethics of those of us who work in this field. Instead, I see these findings as a signal that confidence in our work needs to be earned constantly, and there is no deep well of accumulated credibility to draw upon.
News organizations should also read into these numbers that questions have been growing about the way they use polling information in their coverage of news and politics. Given the results that we see among Albertans, it’s reasonable to assume that the controversy surrounding the Calgary mayoralty campaign is part of how people are responding in that province.“
According to David Coletto: “Too often, we pollsters believe our work is infallible and able to easily predict the future. And for the past few years, apparent polling misses in different elections have given not only our industry plenty of reason to reflect on how we approach our trade but also, as this survey suggests, many Canadians as well.
Our reputation is constantly being tested by the work we put out and how we serve our clients. It is also a function of the leadership we take in managing expectations about what is possible with a survey today and working with your media partners and those who cover polls to set the boundaries to what is reasonable to expect from them. The reputation of polling in Canada is not in a crisis or under siege, but we can’t assume the broader public isn’t watching, reading, or listening to our work with a critical mind.”
METHODOLOGY
Our survey was conducted online with 1,500 Canadians aged 18 and over between October 20th to October 23rd, 2017. A random sample of panelists was invited to complete the survey from a large representative panel of over 500,000 Canadians.
The Marketing Research and Intelligence Association policy limits statements about margins of sampling error for most online surveys. The margin of error for a comparable probability-based random sample of 1,500 is +/- 2.5%, 19 times out of 20.
The data were weighted according to census data to ensure that the sample matched Canada’s population according to age, gender, educational attainment, and region. Totals may not add up to 100 due to rounding.
ABACUS DATA INC.
We offer global research capacity with a strong focus on customer service, attention to detail and value-added insight. Our team combines the experience of our Chairman Bruce Anderson, one of Canada’s leading research executives for two decades, with the energy, creativity and research expertise of CEO David Coletto, Ph.D.
