This was the eye-catching claim made in a recent story in The Times, which asserted that ‘[a] third of Londoners are completely unvaccinated’. With case rates in the capital shooting through the roof and hospital admissions threatening to follow suit as the Omicron variant arrives in town with a vengeance, this sounds like a pretty alarming state of affairs. Indeed, with the emerging evidence suggesting that even two doses of vaccine provide only limited protection and a booster vaccination is required to restore high levels of immunity against Omicron, these seem like worrying times to be in London.
But can this claim really be true? Superficially it appears to be – the figures are taken from the government’s generally excellent Covid-19 dashboard. In fact, you could argue on the basis of the dashboard data that a third is an understatement as this figure reflects the vaccination rate in people ages 12+ only. There are around 1.4 million under 12s in London, which would bring the proportion of people in London who have not had a single vaccination up to 41%. Yet, all is not as it seems…
When it comes to calculating a vaccination rate, we need to know both the number of people we have vaccinated and the total number of people in the population. Counting the number of vaccinated people is easy – we already know how many vaccine doses we’ve given out and how many of those were 1st, 2nd or 3rd / booster doses. But knowing how many people there are in London is, it turns out, surprisingly difficult.
The vaccination rollout has been coordinated using health care records – something called the National Immunisation Management System, or NIMS, to be precise. This is the same system that the NHS uses to send invitations for seasonal flu vaccinations. NIMS gets its data from primary care records – your local GP surgery. This seems like a pretty good way to count people – almost everyone is registered with a doctor after all. However, it turns out there are some serious issues with this data.
Often when people move to a new area and register with a new GP, they don’t get removed from the records of their old one, and so appear twice in the data. Similarly, when people pass away, or leave the country entirely, they aren’t always removed from the system. With the combined effects of Brexit and then Covid-19, we’ve seen a lot of emigration from London in particular, and much higher than usual mortality rates. The upshot of this is that NIMS almost certainly overstates the number of people in the capital.
To get a sense of just how big this overstatement might be, we can look to alternative estimates of our population. The latest of these comes from the Office for National Statistics (ONS) and represents their best estimates of the population in mid-2020. These numbers are themselves imperfect, being based on 2011 census data and explicitly not accounting for the mortality impacts of the pandemic, but the difference is remarkable.
NIMS says that there are 9.2 million people aged 12+ in London while the ONS estimate is 7.6 million. That’s a staggering 1.6 million people in London who may or may not actually exist. Since we can’t vaccinate people who don’t exist (for fairly obvious reasons), our choice of population estimate has a huge impact on our estimate of the number of unvaccinated Londoners. Moving from NIMS to ONS increases our estimate of 1st dose coverage in London from 68% to 82%.
A second notable thing about this graph is that the difference between the two estimates is so much bigger for London than other parts of the country. This is in large part because the differences between NIMS and ONS estimates are highest in younger adults, and London has a much younger population than the rest of England. Higher rates of rental, rather than home ownership in London than the rest of England are also likely to contribute to Londoners moving more frequently, increasing the chances they appear in NIMS more than once.
So we have these two, very different figures, but which is right? The answer is that they are both definitely wrong to some extent. NIMS will overstate the number of young people, while ONS doesn’t account for the high death toll in older age groups during the pandemic, so is likely to overstate the number of people in those age groups who are alive today. And both sources are unlikely to fully capture the extent to which former London residents have left the capital or even the country in the past two years. But, there is clear agreement from experts like David Spiegelhalter that ONS figures are more appropriate when looking at vaccine coverage. Indeed, NHS England use ONS estimates for their national and regional vaccination coverage data and the government dashboard uses it for their national figures, but NIMS for their regional numbers.
The huge sensitivity of our overall estimates of vaccine coverage to this choice is a big challenge, particularly for London, and it’s something that doesn’t just affect our estimates of how many unvaccinated people, but also 2nd doses and booster coverage.
There are valid arguments for the use of NIMS (like here) but when our best estimate is that less than one in five Londoners aged 12+ have yet to have any vaccinations, publishing figures from trusted sources that say the true number is one in three is only likely to lead to confusion. This problem is all the more acute because precisely this same issue was what led to recent controversy about Covid-19 case rates in vaccinated vs. unvaccinated groups that led to the UK statistics regulator publicly criticising UKHSA.
In the longer term, hopefully we can work towards reducing the gaps between different population estimates (something the 2021 census will certainly help with), but until then those of us working to communicate Covid-19 data to the public need to do a better job of highlighting these uncertainties and their implications.
By Colin Angus, Senior Research Fellow in the School of Health and Related Research at the University of Sheffield.
