Using administrative data to measure the labour market

Official labour market statistics are based largely on surveys that can be expensive, biased by non-response, subject to recall error and too small to support detailed analysis.

Administrative and other new forms of data can address some of the shortcomings of survey data, providing an opportunity to observe labour market behaviour at scale on a consistent basis over a long period of time. This project draws on different forms of data to enhance labour market statistics and to understand aspects of the labour market impacts of COVID-19.

In the light of new working patterns (such as self-employment), ESCoE researchers also used administrative data from tax records to capture how economic activity is changing, what shifts in workforces mean for income growth and how well this activity is captured in current surveys. Researchers also investigated the use of electoral roll data to improve migration statistics.

To establish the potential for administrative data, we used HMRC’s PAYE RTI data to produce labour market statistics on transitions in and out of employment, job-to-job transitions and earnings mobility.

We explored the impact of COVID-19 on the labour market, addressing two main questions. Firstly, what the impact has been on self-employment relative to paid employment and secondly, what has happened to local labour markets during this time and the extent to which they appear to rebound from the economic shock of COVID-19. We explored these questions initially through descriptive analysis of the Labour Force Survey and online job vacancy data before conducting more formal econometric modelling. Our case study of the creative workforce focused on a subset of the population but also examined whether the impact of COVID-19 is any different within geographically-specific creative clusters.

We used the universe of business owners’ administrative tax records provided by HM Revenue and Customs (HMRC) to learn more about business owners and their businesses than has previously been possible using survey data. Specifically, using these data, we documented the numbers, characteristics, incomes and business activities of business owners in the UK. We tracked the same business owners over time – something that has not been possible before – and used this to analyse patterns of business start-up and closure and to explain substantial falls in sole trader incomes since 2008. Our analysis relied on linking a number of administrative datasets.

We found that during the pandemic there was considerable variation within the creative workforce, both employed and self-employed. Those employed in IT, software and computer services appear to have fared better than the non-creative workforce, while other creative occupation groups such as Music, performing and visual arts were harder hit with sustained falls in employment and reduced hours.

While the business owner population has grown especially quickly since 2007/08, this growth has been accompanied by a decline in reported capital investment. Sole traders, who account for 90% of the net growth in business owners since 2007/08, have seen large falls in income. These findings highlight that the ‘number of people running a business’ is a poor measure of the economic contribution of that sector.

Obtaining a detailed, accurate and timely understanding of changes in the labour market helps policymakers, businesses and employees respond and make informed strategic decisions.

Having a better understanding of the business owner population is important for public discourse and policymaking. This project has shown the benefits of using administrative tax data to study the UK population of business owners. We can say much more than was previously possible about the legal forms people use, the activities they engage in and the ways in which they take their income.

Egyei, R. ‘Essays on Migration, Networks and Labour Market Transitions’ King’s College London Doctoral Thesis, Jul 2023

Dorsett, R., Hug, J. ‘Local variations in the labour market impact of COVID-19‘ ESCoE blog, July 2022

Dorsett, R., Hug, J. ‘Local variations in the labour market impact of COVID-19‘ ESCoE Discussion Paper, July 2022

Bowyer, A. and Dorsett, R. ‘What has been the impact of COVID-19 on self-employment relative to paid employment in the UK?‘ ESCoE Discussion Paper 2022-07

Dorsett, R., Hug, J. ‘Using UK tax records to produce new statistics on labour market transitions‘ ESCoE blog April 2022

Dorsett, R., Hug, J. ‘Using UK tax records to produce new statistics on labour market transitions’ ESCoE Discussion Paper April 2022

Egyei, R. (2001) ‘Immigration Stocks and Flows, APS and Electoral Register Data’ ESCoE Conference on Economic Measurement 2021 Poster Exhibition, 11-13 May 2021. Poster Presentation.

Bakhshi, H., Bowyer, A., Dorsett, R. and Hug, J. ‘How differently has the creative workforce fared under COVID-19?‘ May 2021

O’Connor, M. and Portes, J. ‘Estimating the UK population during the pandemic‘ January 2021

O’Connor, M. and Portes, J. ‘Estimating the UK population during the pandemic‘ ESCoE Blog, 14 January 2021

de Coulon, A., Egyei, R. and Wadsworth, J.  ‘Immigration Stocks and Flows, LFS and Electoral Register Data’ ESCoE Conference on Economic Measurement 2020, Contributed Sessions O: Labour Markets and Migration

de Coulon, A., Egyei, R.K. and Wadsworth, J. (2020) ‘Immigration Stocks and Flows, APS and Electoral Register Data‘ ESCoE Discussion Paper Series, ESCoE DP 2020-07

de Coulon, A. ‘Using electoral registers to quantify immigration in UK local authorities‘ ESCoE Research Seminar, 28 January 2020

Cribb, J., Miller, H., Pope, P. ‘Who are business owners and what are they doing?‘ ESCoE Discussion Paper 2019-12

Portes, J. ‘Population estimates and migration statistics compared‘ ESCoE Blog, 22 June 2018

Wadsworth, J. ‘Does Immigration Affect the Training of Native-Born Workers (and Are There Different Effects From EU and Non-EU Migrants)?‘ Royal Economic Society Annual Conference, ESCoE-ONS Special Session ‘Economic Statistics for Times of Transition’ 26 March 2018