Jonathan Portes outlines and evaluates the debate around the economic impact of Brexit in light of a new UK in a Changing Europe working paper from Graham Gudgin and Saite Lu disputing John Springford of the Centre for European Reform’s model estimating how much Brexit has cost the UK economy.
How much has Brexit cost the UK economy? The best publicised estimate is that produced by John Springford of the Centre for European Reform, who concludes that UK GDP is currently about 5% lower than it would have been had we not voted to leave. In a recent UK in a Changing Europe panel discussion, we brought together Springford and two Brexit-supporting economists, Julian Jessop and Graham Gudgin, to debate his findings.
Since then, Springford has published a more detailed explanation and defence of his estimates; and, in response,in a new working paper, Gudgin and his co-author, Saite Lu, argue that: ‘[Springford’s] CER doppelganger index does not provide a credible measure of the impact of Brexit.’ What should non-economists make of these competing arguments?
There are three key points which need to be adjudicated here. The first concerns the appropriateness and validity of the Springford ‘Synthetic Control Method’ (SCM) approach. This constructs a counterfactual ‘Britain without Brexit’ using a weighted combination of other countries, whose path of GDP growth and other variables tracked the UK in the period running up to Brexit, with the countries and the weights determined by an algorithm to ensure no bias creeps in.
Gudgin and Lu accept that this is now the standard approach, but nevertheless don’t think it should be used here. Their fundamental objection is that we are ‘asked to believe’ this weighted combination, while it tracks the performance of the UK economy before Brexit, includes many countries which individually don’t look much look like the UK.
‘In fact, in the great majority of the 22 selected countries quarterly growth in GDP is not statistically significantly correlated with the UK….We are asked to believe that a composite index composed of countries which did not individually have a similar growth pattern to the UK prior to June 2016 should nevertheless in aggregate provide a meaningful counterfactual for the UK [for the subsequent period]’
Well, yes. Just as we are ‘asked to believe’ that an index composed of England, Wales, Northern Ireland and Scotland should track UK GDP, even at times when the constituent countries grow at very different rates to the UK as a whole. Or that one composed of the 50 US states should track US GDP, even though no individual state will grow in a similar way to the US as a whole, and individual state growth may not even be strongly correlated with US growth overall. This is how composite indices work. Gudgin and Lu’s objection here is simply wrong, as a matter of arithmetic rather than economics.
Nevertheless, let’s move on to the second point, which is Gudgin and Lu’s alternative approach. This addresses Springford’s challenge at the debate, summed up in a common economists’ expression “it takes a model to beat a model”. Gudgin and Lu’s own model is based on his view that any counterfactual should simply be drawn from countries whose growth rate was indeed strongly correlated with UK growth in the pre-Brexit period. Using a simple regression analysis, known as an Error Correction Mechanism (ECM), they construct a counterfactual that uses only three countries, the US, Germany and Switzerland, all of whose growth rates he finds to be significantly correlated with that of the UK. They argue that the “equation …provides an excellent fit for the UK quarterly GDP index between the first quarter of 2009 and the second quarter of 2016”.
The results are shown here:
GDP volume index with 2009Q1=100, UK vs ECM Forecast
Figure 4 from Gudgin & Lu ‘The CER doppelganger index does not provide a credible measure of the impact of Brexit’.
Source of data: https://stats.oecd.org/Index.aspx?DataSetCode=QNA# The forecast uses the parameters from the equation above.
The bottom line here is that, as Gudgin and Lu say, ‘the measured Brexit impact by 2023q1 is almost identical to that obtained using the CER-weighted doppelganger.’ Now, Gudgin and Lu’s methodology, which might perhaps have been standard a decade or so ago, has largely been superseded in contemporary economic analysis by that adopted by Springford. Nevertheless, it is a very useful check. Sometimes it takes a model to beat a model, but here Gudgin’s own model has – very strikingly – reinforced rather than contradicted the results of Springford’s model.
Finally, Gudgin and Lu say that both their own model and Springford’s should be ignored, because the results are largely driven by US outperformance, in turn driven by fiscal policy. They then note that even excluding the US does not change the Springford result much, and make some further arguments about the inclusion of Australia, New Zealand and Greece. Of course, there are large and important differences between any of these countries and the UK, but that is not in itself a defect in the SCM (or the ECM), as noted above.
Nevertheless, underlying this is a legitimate point, by far Gudgin’s strongest. Both Springford’s SCM and Gudgin’s ECM can, by definition, only measure the ways in which the UK’s growth changed after 2016 relative to other countries. They cannot measure *why* it changed. While some or even most of that change may be attributable to Brexit, other factors – for example, as Gudgin argues, changes in US fiscal policy, or the acceleration of economic recovery in Greece – will also affect relative growth rates, as indeed will changes in the UK economic environment unrelated to Brexit. For example, the recent slowdown in UK labour supply growth is not primarily a Brexit phenomenon. The various tests Springford applies – dropping particular countries or constructing confidence intervals – can only partially account for this when it comes to other countries, and when it comes to the UK not at all.
So where does this leave us? Gudgin and Lu’s paper, far from being a convincing critique, actually provides a further validation of Springford’s methodology as the best available, and analytically robust, way to model the impacts quantitatively. But this does not mean the CER estimate of approximately 5% (or Gudgin’s slightly larger one) is accurate – it should be supplemented with a qualitative description of other things that might have changed since 2016 to affect the UK’s relative growth trajectory.
Personally, I remain of the view that a substantial part but by no means all of the ‘underperformance’ – which, as Gudgin and others point out, is at least in part shared by some other large European countries – is unlikely to be the result of Brexit, and that the Springford and Gudgin estimates are therefore too large.
As I said in the panel, my central view remains that the best estimate of the negative impact on Brexit on UK GDP is 2-3% of GDP, but this is a subjective assessment, not a quantitative model, and there remains plenty of scope for debate. I will discuss this more general question in a subsequent blog.
By Jonathan Portes, Senior Fellow, UK in a Changing Europe.
