The flagship debate programme, BBC Question Time, has often been accused of bias in its panel selection, audience selection and even location selection.
This has become all the more common since the EU referendum, with the new BBC chairman, Richard Sharp, claiming that both Brexit coverage in general and ‘Brexit representation’ on BBC Question Time has been ‘unbalanced’.
Most of these accusations are based on anecdotal evidence, but some studies claim to have uncovered systematic unfairness in Brexit representation.
Three such tallies, from the Institute of Economic Affairs (2016-17), the blogger Joel Rodrigues (2017-19) and the anonymous Question Time Watch (2016-2021), all report a ‘bias against Brexiters’, with roughly twice as many Remainers as Leavers having featured on the programme, compared to the 52-48 victory for Leave.
There are two key questions, though, which these studies fail to address comprehensively, if at all. First, what is the appropriate baseline against which to measure bias? Second, how do you measure Brexit opinions after the referendum?
Although a straightforward comparison of how well panellists’ views mirror that of public opinion may offer some useful insights, this sets a much stricter test than the programme itself sets out to achieve. Its website states the far more limited aim of producing a panel with a ‘broad range of views, knowledge and experience’.
A second issue is that the BBC Question Time panel selection is essentially a sampling process, but not of the general population. Over 60% of slots are reserved for political party representatives.
The BBC typically reserve one slot for the Government and one for the main opposition, plus ‘representatives from other political parties across the series, taking as our guide the level of electoral support at national level which each party enjoys’.
With the majority of each panel being drawn from the political class, the overall Brexit views across each episode will inevitably be weighted more toward the views of Parliament rather than that of the general public.
Moreover, the need to reflect party-political balance, as well as the issue-specific nature of each weekly episode, are competing criteria that guests must be selected by.
In light of these points, perhaps the more relevant questions are: how well do the Brexit views of political guests match up with those of Parliament?
And do the BBC use non-partisan guests to oversample pro-Brexit voices to compensate, or attempt equal representation?
Our project coded guests’ Brexit views, profession, and party affiliation for every episode between the 2015 and 2019 general elections.
This spans the first announcement that an EU referendum would take place, and the election of a Conservative majority that effectively ended the debate.
If we look first at the period between 2015 and 2017, when the most relevant measure of each guest’s Brexit stance is their vote in the referendum, we can see from Figure 1 that among politicians, the ratio of Remainers to Leavers was about 3:1.
While this vastly differs from the outcome of the referendum, it almost perfectly represents the distribution of viewpoints in the 2015 Parliament, where 74% of MPs supported Remain.
Non-politicians, on the other hand, were almost evenly split, with a one per cent margin in favour of Remain (however, no information was available for around five per cent of the non-partisan guests).
This points to the BBC reflecting the balance of opinion in Parliament through their political guests, and wider public opinion through their non-partisan panellists.
With around two thirds of guests being politicians, this inevitably skews the overall representation of Brexit views away from the referendum result.
Moving on to the issue of how to measure Brexit viewpoints over time, previous studies either stick with 2016 Referendum vote choice (Question Time Watch), or partially recode politicians from Remain to Leave if they have publicly changed their stance (Rodriguez, IEA).
The problem, though, is that this simple Leave/Remain divide does not reflect the political fragmentation that occurred during the Brexit negotiations of 2017-19.
The dividing lines that emerged via the party manifestos at the 2017 UK general election largely endured until the UK’s eventual exit from the EU in 2020.
We therefore coded panellists based on their parliamentary voting record (if they were politicians), as well as relevant media articles, interviews, public statements, debates, and tweets across four main categories:
- In favour of second referendum and/or revoking article 50
- Seeks deal, but firmly against a second vote, or no-deal outcome
- Seeks deal, open to no-deal as a potential option
- Favours ‘hardest’ form of Brexit and/or no-deal
As can be seen in Figure 2, 47% of politicians on BBC Question Time after 2017 preferred a second referendum, while a majority advocated for delivering Brexit in one form or another.
Again, the political guests on BBC Question Time were largely representative of Parliament, where support for a second referendum in the first two indicative votes held in the spring of 2019 was 41% and 43% respectively.
In the same round of votes, support for setting a deadline for leaving with no deal was at 25%, which again matches up quite well with the sample of political guests, as 12% were outright in favour of no-deal Brexit, while we would expect a sizeable portion of the 18 per cent who would not rule out no-deal as a negotiation tool to also support the motion.
As for the non-partisan guests, YouGov polling from October 2019 indicates that around 38% of the public were in favour of remaining in the EU, 17% supported leaving the EU with no deal, and 30% supported leaving the EU with the Government’s currently negotiated deal.
Comparing this to the non-politician group in Figure 2, we can see that 36% of guests supported a second vote, 14% favoured a no deal outcome, and 37% favoured some form of Brexit deal.
As with the previous measure, this indicates that non-partisan guests were largely representative of wider public opinion, but when it comes to the overall picture, the sheer number of politicians swung the overall Brexit views more toward Parliament than the public.
In response to the accusations of bias against Question Time regarding Brexit, the answer relies entirely on which benchmark the programme is being judged against.
Once we differentiate between the partisan and non-partisan guests, it appears that the programme was relatively successful in representing the views of Parliament through its political guests, and wider public attitudes via non-partisan panellists.
This goes for both the Leave/Remain divide surrounding the time of the referendum itself, as well as the more nuanced positions that emerged in the subsequent negotiating period.
Whether panel selection should reflect public opinion at the time of the referendum is a relevant issue for discussion.
However, due to both the show’s balance of politician and non-partisan guests, as well as their own relatively limited criteria for representation, this may not be the most appropriate means by which to measure bias.
By Heinz Brandenburg, University of Strathclyde and Brian Boyle, University of Strathclyde.
