Making social science accessible

27 Nov 2015

Politics and Society


What arguments will have the strongest effects on Britain’s EU referendum? Will citizens be more strongly influenced by arguments that focus on economics, or those that stress political or cultural points?

To answer these questions we undertook an online experiment, testing the effects of positive and negative arguments that focus on economics, political influence or culture.

Between 24 September and 1 October we surveyed a representative sample of 5,333 British adults using YouGov’s online panel. Respondents were randomly assigned to either a control group, who received no ‘treatment’, or to one of six ‘treatment groups’ where they were asked to read a series of positive (pro-EU) and/or negative (anti-EU) arguments. We then asked each person how he or she would vote in a Remain/Leave referendum.

The arguments that we used are adaptations of those used by Remain and Leave campaigners. For example, the ‘positive cultural’ argument claims that Britain shares important values with its European neighbours and that by remaining in the EU Britain can protect its culture, traditions and values.

The ‘negative cultural’ argument, in contrast, contends that the free movement of migrant workers threatens Britain’s long established values and ways of life and that only by leaving the EU can Britons protect their culture, values and traditions.

The economic arguments focused on the amount of trade the UK does with the EU (on the pro-EU side) vs. the economic weaknesses of the Eurozone and the costs of EU membership (on the anti-EU side), while the political arguments focused on Britain’s influence in setting EU rules (on the pro side) vs. Britain’s influence in the world outside the EU (on the anti side).

The full texts of the arguments – or ‘vignettes’- can be read at the bottom of this blog.

We wanted to simulate a real world campaign so we presented all respondents with three arguments: a cultural argument, either negative or positive; a political argument, either negative or positive; and an economic argument, either negative or positive.

Each person was presented with either three pro-EU arguments, or three anti-EU arguments, or two pro-EU and one anti-EU, or one pro-EU and two-anti-EU. This enables us to estimate the effect of any one particular argument on a person’s likelihood of voting to Remain or Leave the EU, controlling for all the other arguments they receive.

Let us begin with the referendum vote intention. In the control group, which received no treatment, respondents were given the following question: ‘There is expected to be a referendum on Britain’s EU membership. How will you vote? Should the UK remain a member of the EU or leave the EU?’ Leave is ahead on 42%, with Remain on 38% and Don’t Know on 20%. This is broadly consistent with other recent polls that suggest the race is very close.

But what about the effects of the arguments? As shown in the chart below, at this point in the campaign it appears that both positive and negative arguments sway voters. We find little evidence to support the claim that economic arguments are more powerful, or that voters are swayed more by cultural arguments about immigration. On the contrary, while arguments that stress the positive cultural reasons or positive economic reasons for remaining in the EU do have statistically significant effects, nudging Remain into the lead, as the chart shows the race still remains very close – in fact too close to call. No other arguments, either positive or negative, had statistically significant effects on the Remain or Leave vote at the aggregate level.

Figure 1. EU Vote Intentions by Treatment Group – All Voters


But what happens when we drill down further and explore the effects of these arguments on Conservative and Labour party voters (due to small numbers of respondents within each of the treatment groups we are unable to look at other party voters)? Take a look at the chart below.

In the control group we can first see how different Conservative and Labour voters really are. Conservative voters split 49-29 in favour of Leaving the EU whilst Labour voters split 53-27 in favour of Remaining in the EU. These figures can then be compared with those for the various treatment groups to see what impact the arguments had on Conservative and Labour voters. Across both groups we find that, overall, it is the negative arguments –which stress the economic costs of EU membership, the threat from free movement and the need for Britain to forge a political future outside of the EU- that tend to have the strongest effects.

Among Conservatives it is the negative cultural, economic and political arguments that produce a statistically significant increase in support for Leaving the EU (when compared with the control group). Support for Leave among Conservatives is a significant 4 points higher among those who are given an argument about the ‘threat’ from free movement, 7 points higher among those presented with an argument about regaining national sovereignty and 8 points higher among those given an argument about the economic costs of EU membership. This suggests anti-EU campaigners may want to target negative messages toward Conservatives, whereas the Remain camp needs to think about how to respond to the finding that ‘love-bombing’ Conservatives with positive messages about the EU appears to have no significant impact.

Figure 2. EU Vote Intentions by Treatment Group – Conservative Supporters


Among current Labour voters, too, we find that negative arguments produce significant effects, increasing their support for Leaving the EU by between 3 and 7 points when compared with the control group. But interestingly for Labour voters it is the negative cultural arguments that appear to generate the largest change, increasing their support for pulling Britain out of the EU by seven points and reducing their support for Remaining in the EU to only 50%. In Labour areas therefore anti-EU campaigners might want to devote energy to tapping into this specific anxiety about the free movement of migrant workers, which as we know from other research is one major reason why Ukip has been able to sink roots in Labour areas. Pro-EU campaigners, meanwhile, need to think seriously about how to neutralise these negative cultural arguments. Simply pumping a set of positive messages about the EU into Labour-held areas seems unlikely to have an effect.

Figure 3 EU Vote Intentions by Treatment Group – Labour Supporters


Overall, therefore, the arguments that will be employed during Britain’s referendum look set to make a difference to the overall vote. While some anti-EU campaigners might want to resist the idea of ‘going negative’, our results suggest it may be in their interest to do so, although some arguments appear to have stronger effects among some groups of voters than others. Clearly, this is only the start of our analysis. In future blogs we will explore the effect of these arguments among different age groups as well as social classes, as well as undertake some more sophisticated analysis for a research paper.

The arguments or ‘vignettes’ that we used:

Positive economic. “Britain’s membership of the European Union (EU) attracts significant inward foreign investment into the British economy. The EU is Britain’s major trading partner, which in 2014 accounted for 45% of exports and 53% of imports of goods and services. It is estimated that over three million jobs in Britain are linked, directly or indirectly, to its exports to the European Union. By remaining in the EU, these economic benefits would be safeguarded.”

Negative economic. “The Eurozone economy is experiencing significant economic problems, including high debt and youth unemployment. Britain’s membership of the EU is also costly. In 2014 alone, Britain’s net contribution to the EU budget was an estimated £9.8 billion, up from £3.3 billion in 2008. For Britain, remaining in the EU would risk its economic recovery and endanger the jobs of British wage earners.”

Positive cultural. “Britain shares important values with our European neighbours, such as freedom of speech, gender equality, the rule of law, and respect for liberal democracy. Amid a world that seems less stable, and where there are competing ideologies, European countries can better promote and protect their values by acting together, as members of the European Union (EU). For Britain, remaining in the EU would help defend its national culture, traditions and values.”

Negative cultural. “One of the founding pillars of the European Union (EU) is ‘free movement’, which allows citizens of EU member states to travel and work freely in other EU member states. But this free movement of migrant workers into Britain poses a threat to the country’s long established values and ways of life. For Britain, leaving the EU would help protect its national culture, values and traditions.”

Positive political. “If the European Union (EU) is to work in Britain’s interests, then Britain needs to be involved in the decision-making process. France and Germany would have no incentive to listen to Britain if it is not working closely with them as a member of the EU. If Britain were to leave the EU, to continue to trade with EU countries it will need to apply EU rules on trade, investment, product standards and services, but it will have no say when these rules are made. For Britain, remaining in the EU would ensure that it enjoys these political benefits.”

Negative political. “By leaving the European Union (EU), Britain would be able to set its own course. Britain does not need to be a member of the EU to play an important role in the world. Britain has a ‘portfolio of power’ in its own right, which includes membership of the G20 and G8 nations, a permanent seat on the UN Security Council, leadership of the Commonwealth of 54 nations, and a close relationship with the United States. London is the financial capital of the world. For Britain, leaving the EU would allow the country to regain its national sovereignty while continuing to be a major power on the world stage.”

By Matthew Goodwin, Professor of Politics at the University of Kent, Simon Hix, Harold Laski Professor of Political Science at the London School of Economics and Mark Pickup, Associate Professor of Political Science at Simon Fraser University. Matthew and Simon are senior fellows of the ESRC’s ‘UK in a Changing Europe’ programme.



Has Reform UK really come of age?

Core vs swing voters: unpacking Labour’s electoral dilemma

The restoration of power-sharing in Northern Ireland: back to the old routine?

What does the launch of Popular Conservatism mean for the Conservative Party?

Will Labour’s stance on Gaza harm its electoral prospects?

Recent Articles

Subscribe to our newsletter

* indicates required