By Cath Sleeman
While the phrase ‘good jobs’ is a firm favourite of politicians across the political spectrum,[1] there is no official set of job quality indicators for the UK. This set might include the likes of pay, progression, employee voice and numerous other factors.[2] There is also no official system to continually monitor how the mix of these indicators is changing over time. This makes it much harder to design policies that improve worker wellbeing, which in turn, may even be acting as a drag on productivity.[3]
The aim of this study as published in the ESCoE Discussion Paper here, is to explore whether a novel dataset might provide a useful starting point for creating a job quality monitoring system, and one that would complement existing efforts by the Office for National Statistics (ONS) and The Chartered Institute of Personnel and Development (CIPD). Using 1.5 million reviews by employees, posted on Indeed’s UK website, we extracted keywords and grouped them to build a multi-level taxonomy of job quality for the UK. The text in these reviews has never previously been used to study job quality.
Data
The public reviews, posted on Indeed’s UK website, are intended to help job seekers learn about different workplaces. Most reviewers describe the tasks and duties in their role as well as the features of their job, which impacted their wellbeing. It is this second component that forms the focus of this study.
In the reviews, workers are free to write about any aspect of their work experience, and in as much detail as they wish. This allows us to surface more granular dimensions of job quality than would be possible from a survey. Another benefit is that new reviews are continually being added to the site (although our specific dataset ends in 2019) and reviews can be collected at no cost.[4] In contrast, surveys can be expensive to run and there is inevitably a delay between the launch of the survey and the publication of results.
On the flipside, employee reviews can be both biased and noisy. A particular concern is that workers who leave reviews may not be representative of the wider workforce. To investigate this bias, we identify occupations that are under or over-represented. A second source of potential bias lies in the content of reviews. Contributors know that their reviews will be public and they can read what their colleagues have written – both factors may influence the topics that reviewers mention. A final hurdle is the very informal style in which the reviews are written. Misspellings and mistakes in grammar are common, which caused issues in tagging parts-of-speech (i.e. confusion between nouns and verbs).
Method
The Quality of Work taxonomy is built by training a model to create word embeddings, which are numerical representations of words. These embeddings enabled us to construct a network that links the most frequent nouns, whereby the strength of connection (or edges in the network) reflects their semantic similarity. A hierarchical clustering algorithm is then applied to find groups within this network (of similar nouns), which formed the branches of the taxonomy.
Findings
The taxonomy reveals aspects of job quality that typically do not receive much attention. One such aspect is ‘non-financial recognition’, where workers refer to being thanked, appreciated or acknowledged. Another interesting aspect is references to the broader organisation – including comments on the visibility and transparency of senior management, as well as the organisation’s philosophy, ethos and vision. These factors do appear to impact workers’ wellbeing and could be added to surveys of job quality.
The study also finds differences amongst workers in regard to the topics that they mention. For instance, when referring to negative work experiences, those who are not in managerial roles speak more frequently about interpersonal difficulties, while managers tend to speak about concerns around restructuring, shortages, downturns and complaints. There are also interesting differences across occupations. Workers in professional[5] and administrative roles are most likely to mention the atmosphere or environment of their organisation. In contrast, those in skilled trades and logistics are more likely to mention pay than other occupations.
Finally, there is some evidence to suggest that the dimensions of work quality may be changing over time. There have been steady rises in references to work terms and benefits, including pay and hours. Of course, these components may have become more important, or it may simply be that workers have become more willing to speak about these aspects of work on the website.
Applications and further research
One potential use of the taxonomy is as a ‘first draft’ or starting point for an official Quality of Work taxonomy for the UK. The determinants of work quality that we identified could be combined with other sets of indicators, to build an official system of measuring work quality.
Another use of the taxonomy is as an early warning indicator to detect changes in the dimensions of work quality. Tracking changes in the taxonomy, as new reviews are added, could help to identify new aspects that are affecting employees wellbeing, which could then be investigated further.
There are also numerous avenues for further research. One avenue would be to explore how COVID has changed the dimensions of work quality, particularly given the rise in remote working. Another element could explore the impact of the cost-of-living crisis, or more broadly, the influence of the economic cycle on the determinants of job quality.
The Discussion Paper can be downloaded here.
ESCoE blogs are published to further debate. Any views expressed are solely those of the author(s) and so cannot be taken to represent those of the ESCoE, its partner institutions or the Office for National Statistics.
[1] For example, Labour (Co-op) MP James Murray made a reference to ‘good jobs’ in the Commons on December 13th (link). Conservative MP Mel Stride made a reference to ‘good jobs’ in the Commons on November 27th (link).
[2] Several groups have created their own taxonomies of job quality and these are discussed in the paper.
[3] Arends, I., C. Prinz and F. Abma (2017)
[4] The dataset for this study was provided by Indeed to Nesta, under a Data Sharing Agreement.
[5] This term is used in the Standard Occupational Classification. ‘Professional occupations’ broadly cover roles in STEM, health, teaching, business, the media and the public service.
