The UK Longitudinal Business Database – What it is, what it can do, and why we need it


The UK Longitudinal Business Database – What it is, what it can do, and why we need it


How quickly does firm entry and exit respond to changes in the economy? How do young businesses scale up employment over the first few quarters of their existence? How do firms organise their activities across space in different industries, and does this change in response to trade shocks?

The range of policy-relevant questions we can answer is always constrained by the data infrastructure we have built. And for economic researchers in the UK, the range of questions that are answerable has just increased substantially, thanks to a years-long data development push with former colleagues at the Office for National Statistics (ONS). The new UK Longitudinal Business Database (or LBD for short), explained in an ESCoE Technical Report published today, is an effort to improve the quality of firm-level microdata available for researchers.

At its core, the LBD is a set of data spines, or longitudinally linked firm references, for all business units on the Inter-Departmental Business Register (or IDBR, the UK business register). These spines are linked to each other as well, allowing researchers to track establishments, individual firms, and enterprise groups over time at business-cycle frequency, and to observe changes in the business structure. Additionally, by applying unit-specific filters to the data, the LBD aims to preserve only those businesses that are economically active in a meaningful sense.

Datasets generated with the LBD spines provide clear advantages over existing data products for researchers interested in productivity, business demography and the micro-foundations of macroeconomic outcomes.

First, the LBD is the only UK firm-level data product to provide firm entry and exit, employment and other firm characteristics available at business-cycle frequency. It therefore equips researchers with the tools to address a wide array of research questions on the dynamics of firm responses to demand or supply shocks, and better understand the dynamics of the Great Financial Crisis or the Covid-19 pandemic.

Second, by applying unit-specific economic activity criteria, the LBD gives researchers firm activity start and end dates that are more economically meaningful than the birth and death dates on the business register, which had never been intended for microdata research use. The filters also allow researchers to track more complicated patterns of activation, de-activation and re-activation and how these have changed both over the business cycle and over the long run.

Finally, the LBD spines are an ideal vehicle for linking microdata across sources in a consistent way. While for now the first LBD data product only contains information from the business register, in the future we hope it becomes a vehicle for bringing administrative, survey and web-scraped firm-level microdata together in a consistent way.

As an illustration of what the LBD might be useful for, Figure 1 shows the employment associated with firm births and deaths at quarterly frequency. Employment associated with firm births and deaths represents around one percent of overall employment, with large fluctuations. Between 2013 and 2021, firm births accounted for systematically more employment than firm deaths. As a result, the net contribution to employment growth from the entry and exit margin has been positive in recent years. However, there are signs that this gap may have been closing more recently.

The first dataset created with the UK Longitudinal Business Database will soon be available to all accredited researchers on the Secure Research Service (SRS). For those interested in the technical details behind its construction, my data science colleagues will explain the data pipeline in more detail in an upcoming ONS Data Science Campus blog. Full details about the Longitudinal Business Database are available in the aforementioned ESCoE Technical Report, published today.

Figure 1: percentage of overall employment associated with firm births and deaths, 1999-2022.

ESCoE blogs are published to further debate.  Any views expressed are solely those of the author(s) and so cannot be taken to represent those of the ESCoE, its partner institutions or the Office for National Statistics.