Followers of our Poll of Polls will be aware that it has been suggesting that the referendum race has narrowed somewhat during the course of the autumn. At the end of September it stood briefly at Remain 54% Leave 46%, for most of October it read, Remain 53% Leave 47%, but since the middle of November it has suggested that the Remain camp are on just 51% and Leave on 49%.
However a glance at the list of individual polls that have been included in our Poll of Polls calculation reveals that while most have put Remain narrowly ahead (though no more than that) two have suggested that Remain are in fact quite comfortably ahead. The first was a poll conducted by ComRes towards the end of September, the other one undertaken by Ipsos MORI in the middle of October. ComRes estimated that (once Don’t Knows were excluded) Remain were on 60%, Leave on 40%, while Ipsos MORI reckoned Remain were on 59%, Leave on 41%.
‘So what?’, you might say. After all there have also been five polls that have given Leave a small lead. Given that all polls are subject to a degree of random fluctuation, are not both sets of departures from the general picture no more than the kind of variation we might well anticipate to find purely by chance?
However, these two ComRes and Ipsos MORI polls also share another characteristic that distinguishes them from every other poll of referendum voting intentions that has been published so far – they were conducted over the phone rather than via the internet. Their findings might be an indication that the picture of the referendum race that a poll paints depends on how it is conducted. If so, we would not know which picture was right and which wrong, but it would mean we could not be sure how close the referendum race is after all.
Still, you might not unreasonably feel that two polls (one of which in fact administered the question that will appear on the ballot paper to only half its sample) constitute too thin a reed on which to rest such an argument. However, if we look at the polling that was done during the summer in which respondents were asked how they would vote in response to the question that the government was originally minded to include on the ballot paper (viz. Should the United Kingdom remain a member of the European Union?) two phone polls again stand out. The first of these, conducted by ComRes, suggested that 65% would vote Yes (after Don’t Knows were left to one side), while the second, undertaken by Ipsos MORI, put the figure as high as 75%. No other poll (all of them conducted online) reported a Yes vote of more than 59%.
In short, although only a few telephone polls of referendum voting intentions have been conducted, the findings of the handful that have been undertaken have consistently been more favourable to the Remain side than those conducted online. That must leave an air of uncertainty about how close the race is. Indeed, one academic analysis that attempted to take systematically into account the differences in the estimates of referendum vote intentions produced by the different companies has suggested that Remain may be as much as ten points ahead of Leave.
Which brings us back to the Poll of Polls and the apparent narrowing of the referendum race. As we warned on launching the feature, if there are systematic differences in the results of polls conducted by different companies (and/or polls undertaken using different methods) the figure that it reports at any one point in time may well depend on which particular polls are included in the calculation – that indeed is why which polls are included is displayed as you move along the Poll of Polls timeline. Because no telephone poll has been conducted since the middle of October, all the more recent figures for the Poll of Polls are based entirely on polls conducted via the internet- and perhaps it is this that more or less accounts for the apparent narrowing of the race.
One way of checking whether the race has indeed got closer is to see whether there is any discernible trend in the results reported by the same company (deploying the same method) over time. The company that has conducted by far the most polls of referendum vote intention is ICM, who have been publishing an estimate on a weekly basis (the most recent of which was released yesterday). And as our first figure below shows these polls do not suggest there has been any discernible trend over time. Between them the 14 readings have on average put Remain on 53.5%, Leave on 46.5%, and the more recent readings have been as closely anchored to that pair of numbers as were much earlier ones.
The other company that has conducted more than just the occasional poll is YouGov. Its readings, as our second figure shows, have tended to suggest that the race is rather narrower than ICM have suggested. Even so there is again little consistent evidence that the picture has changed over time. YouGov actually put Leave slightly ahead in the middle of September, while the company’s most recent reading has the two sides on 50:50 each.
Note that in this figure we have excluded the two polls that YouGov conducted for (and were weighted by) an academic consortium at Kent and the LSE, but included the monthly polls undertaken as part of YouGov’s Eurotrack series where the question asked is slightly (but only slightly) different from that which will appear on the ballot paper.
Anyone who relied on the polls during the general election is doubtless already inclined to view the figures they are producing for the EU referendum with caution. In truth we would all be wise to do so. At the moment we cannot be sure just how close the race really is, while it appears that we may all too easily form the impression that the race has narrowed when perhaps it has not. But then polls are always better taken than inhaled.
Written by John Curtice, Senior Research Fellow at NatCen, Professor of Politics at Strathclyde University, and Chief Commentator on the What UK Thinks: EU website. This article originally appeared on WhatUKThinks.
