Polling for the Labour party over the last five years, the great mystery was why the newspaper polls consistently showed a much more favourable position for Labour than we were finding in our internal data. While the lead in the public polls suggested Labour had got past the issues that sunk the party in 2010 – its perceived record on the economy and immigration – we knew we had much more work to do and were still dogged by a loss of trust.
From January 2011 to Spring 2013, Labour’s average vote share in the public polls rarely dropped out of the low 40s. We consistently had it around 7 points lower. While the public polls had Labour ahead until early spring of this year, in the party’s internal polls cross-over came right after conference season in 2014. A four point Labour lead in early Sept, turned into a tie in October, followed by small Tory leads; prompting the party to put reassurance on fiscal policy and immigration at the heart of the campaign launch before Christmas. This plan worked through the opening weeks of the short campaign, with Labour pulling ahead in the English marginals following Ed Miliband’s strong debate performances and the non-doms row.
Our final poll, in late April, told a different story. As focus groups showed the SNP attacks landing, we had Labour behind in the marginal seats among likely voters. A public poll in a similar set of seats at the same time showed a 3 point Labour lead.
The campaign strongly toughened our stance on the SNP before the final Question Time, but it was not enough. The Tories successfully used the fear of Scottish influence as a way of catalyzing pre-existing doubts about Labour in a way that had not been possible earlier in the campaign.
One area where we were wrong was the belief that the Tories were facing even deeper structural problems than Labour. Their surge against the Lib Dems in the west country was invisible to us. Labour’s collapse in Scotland gave the Tories an unexpected weapon with which to squeeze Ukip. Labour’s relative inability to reach into the Tory vote proved even more costly than expected.
As several major polling companies have acknowledged, the 2015 election was a collective failure for the British polling industry. It isn’t yet possible to be sure of the explanation and the industry will work hard to make sure it does better in the future. My sense is that the answer lies more in the questionnaire design than who gets interviewed. Whenever we tried weighting our polls to match what other pollsters do, or tried a telephone rather than online methodology, our Labour share remained stubbornly low.
The main difference between our polls and the newspaper polls is that we don’t ask the voting intention first. As Politicalbetting.com’s Mike Smithson found out when he accidentally participated in our only telephone poll of the last 4 years, we first ask respondents to think about the country, the economy, their top issues, the parties and the leaders. We think it gets them closer to their ballot box mindset.
This technique delivers a much lower don’t know number – generally half the level found in the public polls. We treat this ‘don’t know’ group differently to most of the public polls, asking them questions about who they are likely to vote for rather than assuming they are likely to vote for whoever they voted for last time. Of course, that requires many more questions and so is more expensive to implement especially for a phone pollster where every minute costs money. If we had run a final poll close to election day, would we have got the Tory margin right? It’s hard to know. But if this explanation is broadly true, it means the drift to online polling remains valid.
The more optimistic public polling helped Labour at various points, giving us momentum in the face of a hostile media. Ultimately, however, it proved costly, allowing the campaign to become a referendum on the popularity of a putative Labour/SNP government. If Labour had been seen to be a couple of points behind, false Tory claims would have had less bite and scrutiny on the parties would have been more balanced, asking as much about the future of the NHS and living standards as they did about the role of Nicola Sturgeon. A key challenge for the next Labour leader is to make sure the party’s electoral position is robust to such vagaries.
The British polling industry is full of people with great integrity and deep understanding of survey methodology. It is the most open industry in the world when it comes to data. It has rightly recognized that it got this election wrong, and that error had a real impact on the reporting of the campaign. The last polling inquest in 1992 provided some major innovations in the way polls are conducted; this time I hope that is matched with a change in the way the media report on them.
James Morris is a partner at GQR and former pollster to Ed Miliband and the Labour party, who tweets as @jamesdmorris.
Content from our partners
Related
