Making Sense of Skills

Skill shortages are costly and can hamper growth, but these shortages are not currently measured in a detailed or timely way. To address this, we developed the first publicly available data-driven skills taxonomy for the UK. A skills taxonomy provides a consistent way of measuring the demand and supply of skills. It can also help workers and students learn more about the skills they need, and the value of those skills. We are using our skills taxonomy to develop measures of skills shortages in local areas.

Skill shortages arise when there are not enough people with particular skills to meet demand. A mismatch between the skills of the people in a given region and the skills that local employers need can significantly hamper regional growth. The Open University estimates that skill shortages cost the UK £2 billion a year in higher salaries, recruitment costs and temporary staffing bills. As well as being costly to businesses, skill mismatches adversely affect the earnings mobility of workers. Looking ahead, skill mismatches may worsen because the skills we need are changing, for example due to factors such as automation. Despite the importance of skill shortages, these are not currently measured in a detailed or timely way.

The first step to measuring shortages is to build a skills taxonomy which defines accepted groups of skills. Despite having well-established taxonomies for defining occupations and industries, the UK does not have an accepted skills taxonomy. We have developed the first skills taxonomy for the UK. We are further developing this, taking into account new data sources that will enable us to make the full taxonomy available for open use.

To build a skills taxonomy for the UK we began with a list of more than 10,000 unique skills that had been mentioned within tens of millions of UK job adverts. These ‘skills’ are more like job requirements: in addition to skills they include specific tasks, knowledge, software programmes, and even personal attributes. To generate the taxonomy, we employed machine-learning methods such as word embeddings, network community detection algorithms and consensus clustering. We modelled skills mentioned in job adverts as a graph and once skills were represented as a network, we hierarchically grouped them into clusters. The greater the probability of two skills appearing in the same advert, the more likely it was that they ended up in the same branch of the taxonomy.

Using information from online job adverts and ONS labour market statistics, we are producing estimates of regional skill demand. We aim to produce estimates for Travel-To-Work Areas and Local Authorities.

The skills taxonomy identified over 140 clusters of specific skills. We’ve also shown that our methodologies can be used to create further skill taxonomies which provide even more detailed and refined pictures of skills demand. Our skills taxonomy is a first step towards a real-time map that links skills, occupations and qualifications.

The taxonomy provides estimates of the demand for each skill cluster, based on the number of mentions within adverts. Users can search the taxonomy by job title and discover the skills needed for a wide range of jobs. We found that the five clusters containing the most frequently demanded skills are social work and caregiving, general sales, software development, office administration, and driving and automotive maintenance. Around 60 per cent of job adverts mention a salary. This information can be used to provide estimates of skill values for the UK. The five skill clusters with the highest median annual salaries are data engineering, securities trading, IT security operations, IT security standards and mainframe programming. To date, workers and students have had to decide between these skills without access to this information. Skills that attract high annual salaries and for which demand has been growing include data engineering, IT security implementation, IT security operations, marketing research, app development and web development. This may reflect a shortage of workers who have these relatively new skills.

Our work on occupations is being used by the ONS’s Classifications Team to inform their update of occupation classification codes. Google Digital Garage used our skills taxonomy to build their recently launched Profile Builder. The project’s data visualisations, viewable here, have been shortlisted for an Information is Beautiful award in the Science and Technology category. The World Economic Forum report on ‘Strategies for the New Economy: Skills as the Currency of the Labour Market’ cited the skills taxonomy as one of the emerging initiatives for creating a skills-based labour market. Our research team continue to receive many queries from stakeholders who wish to share lessons learnt and to explore opportunities for applying the taxonomy and replicating the research.

The new skills taxonomy using TextKernel data is available here.

The prototype skills taxonomy is available to access here.