The pollster and commentator Peter Kellner feels I have been unfair in my characterisation of the People’s Vote campaign’s polls as been deliberately designed to achieve the most favourable pro-Remain response. Is he right?
Kellner has three objections: the first is that in my comparison of two separate YouGov questions, one by the Economic and Social Research Council’s (ESRC) long-running study of party members, and one by the People’s Vote campaign, I have omitted a third question by the ESRC that produces a result that is closer to that of the People’s Vote. The second is that none of the three questions can be fairly said to be “loaded” or designed to produce a certain response, and the third is that the three questions are too different from one another to be a useful point of comparison.
I’ll take those in order. Today’s widely reported ESRC survey asks Labour party members the following question “Do you think Labour should or should not fully support holding a new referendum on Brexit?” to which 72 per cent of respondents said yes. I contrasted that with the People’s Vote question back in September 2018, which asked Labour activists “When the negotiations with the European Union about Brexit are complete, would you support or oppose a public vote on the outcome ?”, to which 86 per cent agreed.
I have omitted another question by the ESRC, which asked Labour members to “imagine that Parliament eventually votes to reject Mrs May’s proposed Brexit deal… Would you support or oppose holding a referendum on the deal if Parliament voted to reject the proposed Brexit deal?” to which 79 per cent agreed.
I did this for two reasons. The first was that none of the coverage of the ERSC research cited the second figure (nor, for that matter, did the response of Best for Britain, another pro-European organisation). I didn’t consider it to be all that useful to add a third question into the mix, not least because in both cases, the difference between the People’s Vote question is outside the margin of error (that’s plus or minus three per cent). Whichever question you use, there is a difference between the response to the ESRC’s survey and that conducted on behalf of the People’s Vote campaign.
More importantly, I regard the ERSC’s question about whether to “fully support holding a new referendum on Brexit?” as the essential question. When you boil it down, the central aim of the People’s Vote campaign is to have, effectively, a re-run of the last referendum in order to get a different result. That’s fine, and in my personal view highly desirable that Brexit is reversed or stopped by any democratic means available.
But it is, by necessity, a narrower question than the one posed widely shared by the People’s Vote campaign in September, which simply asked whether people would support “a public vote on the outcome”. That could mean a far greater number of things: it could mean a vote on leaving with Theresa May’s deal or leaving with no deal at all: a vote that is obviously unacceptable to anyone wishing to remain in the European Union. It could mean a three-way vote between leaving with a deal, leaving without a deal, or staying in the European Union, which is unacceptable to anyone who believes that it is irresponsible to put the disruption and chaos of a no deal exit to a public vote. Or it could mean a referendum in which the choice is leaving on Theresa May’s terms or not leaving at all, a referendum which is unacceptable to many Leavers, who believe that would be a vote on two ways in which you are not really leaving the European Union at all.
Self-evidently, if your opinion poll question allows all of those possible outcomes to be contained within, you will get more support than you will if you choose a narrower question. The important question from a pro-European standpoint, however, is how to people feel about having another option to decide to stay in the European Union.
As to whether the questions are too different to be meaningful points of comparison, in my view, that one poll asks people what they think the country should do and one poll asks what they think the leader of the party they are literally a member of should do is a distinction without a difference, but I am prepared to accept that other people see it otherwise.
There is an open debate about whether the People’s Vote campaign is following best practice by asking its second vote questions at the end of a long series of questions about what Brexit means for both matters of national policy – the ease of ending austerity, for instance – and their families. Some pollsters believe that this influences responses unduly. Ed Miliband’s former pollster James Morris found that the polls he conducted in which questions of voting intention following extensive questions about politics produced what turned out to be more accurate results. It’s too early to say conclusively whether or not this holds true.
But that’s besides the point: it is, in my view, wholly legitimate for the People’s Vote campaign to use the most favourable questionnaire wording and structure it can to increase its prospects for securing the referendum re-run it craves. But it is not accurate to do as Kellner does and claim that there is “no evidence” that the campaign’s choice of questions has an impact on the answers it receives.
Content from our partners
Related
