Data science and artificial intelligence skills are sorely lacking in the UK jobs market, according to new government research, and vacancies are being left unfilled.

Out of 118 firms surveyed by the government, nearly half (49 percent) said they were affected by a lack of candidates with technical skills in AI, and 32 percent by a lack of relevant non-technical skills.

Other barriers to employment include a dearth of work experience and industry sector knowledge.

Among firms reporting an AI skills gap among existing employees, 55 percent said the gap was in employees’ understanding of AI concepts and algorithms, 52 percent in coding skills and programming languages, 52 percent in systems and software engineering, and 51 percent in user experience.

Sixty-two percent of respondents said employees in AI roles had received at least some training in the past 12 months to improve their skills. However, less than a quarter had been trained in the ethical issues surrounding AI deployments.

Over two-thirds of firms (67 percent) said they believed demand for AI skills will increase over the next 12 months. This suggests that the gap between organisations’ AI aspirations and their ability to act on them will only increase until more skilled individuals enter the market.

Two-thirds of the firms surveyed have tried to recruit in the last two years, and 69 percent said that at least one vacancy had proved difficult to fill.

According to the government, the bulk of hard-to-fill vacancies were among middle-management and other senior roles, which required three or more years of experience.

For more junior roles, most firms were unlikely to hire staff through internships (just 11 percent did) or apprenticeships (just three percent reported hires from that pathway into employment).

Poor diversity scores

Lack of diversity remains another big challenge for the industry. Fifty-three percent of the firms surveyed employed no women at all in AI and data science roles, and 40 percent no ethnic minorities.

Overall, less than a quarter of the AI workforce (24 percent) is female and just over one quarter (27 percent) comes from ethnic minorities.

However, the figures are higher than for the IT/STEM sector overall, where previous studies have shown that 85-90 percent of the workforce is male and over 90 percent of employees are white.

With products and services needing to be developed for use by everyone in society, it is essential that the industry more closely reflects the diversity of the population to avoid bias and/or lack of appropriate design entering the system.

Despite the pandemic, 2020 saw the highest number of advertised AI and data science job vacancies, with a 16 percent year-on-year increase.

In total, 110,495 job postings related to AI and data science appeared online between January and December last year – more than double the number in 2014.

More than half of the vacancies were in London and the South East, followed by Cambridge (5,453 postings), Manchester (3,619), Bristol (2,505), Edinburgh (2,365), Oxford (2,311) and Birmingham (2,095).

The mean advertised salary was £54,800 for an AI and data science job.