The polls were not exactly the miss they have been made out to be. What’s more, some polls provided good examples of how to do it right.
Published: Monday, 06/30/2014 12:00 am EDT
The Hill Times
OTTAWA—No, the provincial election results in Ontario were not to polling what the most recent provincial elections in Alberta and British Columbia were, despite the claims of some. Most polls, whether they were recording the intentions of likely or all eligible voters, had the Liberals winning, the Progressive Conservatives in second, and the New Democrats in third. Some even suggested a Liberal majority was in the works.
Certainly, the polls were not as accurate as they were in the most recent provincial elections in Quebec and Nova Scotia. The likely voter models that were employed by most pollsters turned in worse performances than the unadjusted numbers. But that was a failure of the turnout models, not the polling itself. None of the polls conducted in the final days of the campaign, recording the voting intentions of all decided Ontario voters, had the PCs ahead.
Some pollsters did worse than others. But two that stood head and shoulders above the rest provided important examples of how polling should be done going forward: EKOS Research, headed by Frank Graves, and David Coletto’s AbacusData.
The final EKOS poll suggested that a Liberal majority was a distinct possibility. The poll gave the Liberals 37 per cent support against 31 per cent for the PCs, not far from the eventual result of 39 per cent for the Liberals and 31 per cent for the PCs. However, the NDP tally of 19 per cent was significantly below the actual 24 per cent the party received, and EKOS’s likely voter model—while giving the Liberals a majority—was off by a larger degree for the PCs and Liberals and underestimated the NDP even more.
What EKOS did well was that it implemented a daily tracking in the final days of the campaigning, allowing the company to see how opinions were moving as the vote approached. EKOS was thus able to record a brief uptick for the Tories in the final weekend—one that was apparently registered by the internal polls of the Liberals as well—before the vote returned to the low-30s that EKOS had earlier been reporting.
Another laudable thing that EKOS did was that the company did not hide or make glib excuses when the results came out, as some other companies did. Instead, Graves wrote a detailed post-mortem investigating why his likely voter model did not work and why his final polls underestimated the New Democrats.
The reason? By any objective measure, NDP supporters fit the profile of people who would be less likely to vote, and yet they did anyway. Voter turnout models, in the end, are based on assumptions of what the voting population looks like, but in every election that population can be motivated by different factors and change in significant ways. Another cause? New Democrats were more likely to use cell phones, and while EKOS did poll some cell phones the proportion of cell phone-only respondents was lower than among the general population. Generally, these people do not vote. But on June 12, they did.
So, in addition to putting in a strong performance in terms of the eventual Liberal and PC results, EKOS went the extra mile after the election to explain why it missed where it did. That is the kind of upfront transparency that other pollsters should emulate.
Abacus is also worthy of highlighting primarily because of the highly detailed polls they conducted throughout the campaign. Their final estimates were also good (35 per cent for the Liberals, 32 per cent for the PCs, and 26 per cent for the NDP), but suggested a Liberal minority rather than a majority. The likely voter model, as with others, did less well.
But those were just the horserace numbers. What Abacus did well was that its polls investigated a myriad of other factors. From its final poll of the campaign, we can learn that a plurality of voters expected the Liberals to win; that more Ontarians reported being contacted by the Liberal Party than by the PCs or NDP; that a majority of Ontarians had a negative impression of Tim Hudak; that Hudak was a drag on his party’s support; and that the Liberals led on jobs and the economy, the top issue of the campaign by a wide margin. From earlier polls, we were able to glean that the Liberals led among swing voters, whether from the right or left.
Because of all of these extra bits of information—and this is only a sampling of the reams of data Abacus released with each report—it was possible to understand the dynamics of the campaign in a way that horserace figures can only hint at. This information suggested that despite the close race Abacus recorded among decided and likely voters, the fundamentals were pointing to an advantage for Kathleen Wynne.
Quality, depth, frequency, and transparency. The results were mixed for the pollsters in Ontario, but the ones that strove to achieve the highest degree of these four objectives showed how polling can be done right.
Éric Grenier writes about politics and polls at ThreeHundredEight.com.