Presented by Christopher Kurz, Federal Reserve Board
This webinar examines the link between industrial production and the sentiment expressed in natural language survey responses from U.S. manufacturing firms.
We compare several natural language processing (NLP) techniques for classifying sentiment, ranging from dictionary-based methods to modern deep learning methods. Using a manually labeled sample as ground truth, we find that deep learning models–partially trained on a human-labeled sample of our data–outperform other methods for classifying the sentiment of survey responses.
Further, we capitalise on the panel nature of the data to train models which predict firm-level production using lagged firm-level text. This allows us to leverage a large sample of “naturally occurring” labels with no manual input. We then assess the extent to which each sentiment measure, aggregated to monthly time series, can serve as a useful statistical indicator and forecast industrial production.
Our results suggest that the text responses provide information beyond the available numerical data from the same survey and improve out-of-sample forecasting; deep learning methods and the use of naturally occurring labels seem especially useful for forecasting. We also explore what drives the predictions made by the deep learning models, and find that a relatively small number of words–associated with very positive/negative sentiment–account for much of the variation in the aggregate sentiment index.