By Alex Botsis and Kevin Lee
Surveys – like the Suite of Business Surveys of the Confederation of British Industry (CBI) or the Business Insights and Conditions Survey (BICS) of the Office for National Statistics – can inform us in a more timely manner than official statistics about recent changes in the level of economic activity. This is achieved by collecting qualitative information from firms, which can usually be made available more rapidly than the quantitative surveys collected for official statistics. Intuitively, firms can more rapidly indicate whether their level of activity increased, decreased or remained the same during a given period than computing the exact growth rate for that period. In other words, there is a trade-off between speed and accuracy. Additionally, qualitative surveys focus directly on firm-level economic activity; other high frequency data, such as Google Mobility, only proxy for economic activity, but offering the advantage that they are available daily or weekly.
In our ESCoE Technical Report we seek to assess whether qualitative surveys can provide adequate measures (nowcasts) of output in a more timely manner than ONS measures of output, which are based on more accurate but less timely quantitative data. Next, we look at whether the answer to this question changes when we look at specific sectors of the economy, such as manufacturing and services, rather than the whole economy. Finally, we also examine whether using the forward-looking expectations given by survey respondents adds values to these nowcasts.
For our purposes we use monthly survey-based qualitative data from the CBI, for the UK and for a period from January 2005 until July 2021. In Figure 1 we demonstrate how firm-level responses on their activity levels co-vary with the growth of the whole economy (Gross Value Added or GVA) both before and during the COVID-19 pandemic. This co-movement is particularly evident in the event of larger swings in output, for instance in the period following the global financial crisis from 2008-2010 and during the global pandemic in 2020.
With the CBI data and using model weighting techniques that combine different nowcasting models together, we aim to assess the performance of the surveys in nowcasting the month-over-month growth rate of the GVA index published by the ONS. We find that a model that combines survey responses on firm-level activity, both expected and realized, with an autoregressive model of order 1 (AR(1)) offers superior nowcasting accuracy relative to using only either of the realizations, expectations or the AR(1). We also demonstrate that at the sectoral level, manufacturing and services, this is the case only for the manufacturing sector. During the pandemic it is less clear that the use of the survey-based data can improve our nowcasts. One reason for this might be the short period that our sample covers during the pandemic, from January 2020 until July 2021.
Our paper also offers some lessons-learned regarding the CBI surveys that can also apply to similar surveys like the BICS that has been run by the ONS since 2020. Based on our findings, the BICS offer some potential advantages over the survey data from the CBI. First, it includes a larger sample of firms than the CBI. Second, it asks all firms explicitly about their turnover, whereas the wording in CBI survey questions is different for each sector. CBI asks about the volume of output in Manufacturing, of sales in Distributive Trades and the volume of business in Services. What the `volume of business’ might cover is less precise than turnover, which is the focus in BICS. Third, responses in BICS have more than three bins so they can capture more than the mere direction of change that is captured by the three bins in CBI surveys (up, down, no change). Essentially, having five bins strikes a fine balance: eliciting more nuanced responses than three bin questions without requiring respondents to provide an explicit growth rate, such as that requested in the quantitative Monthly Business Survey (MBS). This allows firms to respond more quickly than in the MBS, which the ONS uses to compute the monthly indices of manufacturing and services.
Overall, using the CBI data we found that qualitative surveys, like the one we used or the BICS, can offer adequate nowcasts for aggregate economic activity, in the absence of any additional indicators, in a more timely manner than quantitative surveys. Additionally, given the findings of our paper we think that for qualitative surveys to strengthen their potential, they need to: (i) include a large sample size (ii) ask all firms explicitly about their turnover which is comparable across all sectors of the economy; (iii) collect forecasts of the firms about the short term evolution of their turnover; (iv) allow responses to have more than three bins so they can capture more than the mere direction of turnover.
Read the full ESCoE Technical Report here.
ESCoE blogs are published to further debate. Any views expressed are solely those of the author(s) and so cannot be taken to represent those of the ESCoE, its partner institutions or the Office for National Statistics.