Dutch AI monitor 2024

Chapter 1 Introduction

The term artificial intelligence (AI) describes machine-based systems that can, based on the inputs provided, infer how to generate output for specific purposes. AI is an example of a systems technology, and in this sense can be compared with electricity or the internal combustion engine. AI has been high on the agenda of policymakers in recent years. Any new technology with the potential significance of AI demands systematic, ongoing monitoring. The current report is another step towards the development of a national AI monitor. It reports on the development of methods for creating new statistics, but it also provides (preliminary) figures. The report focuses on companies that produce AI, companies that use AI, educational programmes that focus on AI and how these are feeding through into the labour market, and the demand for workers with AI skills. The research was commissioned by TNO (the Netherlands Organisation for Applied Scientific Research) and also represents a contribution to the monitoring activities of the AiNed programme.  

Chapter 2 Use of AI by businesses

• Statistics Netherlands’ survey of ICT Usage in Enterprises includes questions regarding the use of seven AI technologies by businesses in sectors C to N and Q that have 10 or more employees. The seven technologies are: machine learning, service robots or autonomous vehicles, robot-assisted process automation, speech recognition, image recognition, text mining and natural language generation.
• In 2024, 23 percent of companies with ten or more employees made use of one or more of the AI technologies listed. This represents an increase of nearly 8 percentage points since 2023.
• Companies that use AI technology accounted for more than half (51 percent) of the total revenue of companies in the Netherlands in 2024.
• In 2024, among companies that used AI technology, it was used most widely for marketing and sales (36 percent) and administrative processes or management tasks (30 percent).
• The most common way in which companies obtained AI technology in 2024 was the direct use of commercially available software.
• Among those companies which, in 2024, had considered adopting AI technology at some point, but which were not currently using it, ‘lack of experience’ was by far the most commonly given reason for this (75 percent).

Chapter 3 Companies that produce AI technology

• In order to ascertain the number of companies in the Netherlands produce AI technology, a method based on machine learning was developed to differentiate ‘AI companies’ from ‘non-AI companies’ by analysing the text on their websites.
• ‘AI system’ was defined using the definition of the OECD. An ‘AI company’ was defined as ‘a company whose principal activity is the production of AI systems’.
• By applying this method, we produced a set of 450 websites belonging to 402 AI companies with a presence in the Netherlands that were active in 2024.
• Of the AI companies identified in 2024, 97 percent were small and medium-sized enterprises, defined as companies with fewer than 250 workers.
• Most AI companies are active in sector J, the Information and Communications sector (63 percent).
• Most AI companies were limited liability companies (83 percent)
• The provinces with the most AI companies were Noord-Holland (32 percent) and Zuid-Holland (24 percent). 5 percent of the companies identified were subsidiaries of companies based abroad.
• In 2023, the majority of AI companies (43 percent) had a turnover of between 100 thousand euros and one million euros. AI companies often have higher than average revenues compared with other companies in the Netherlands.
• Among AI companies with at least one worker, the majority (85 percent) had a turnover of less than 5 million euros in 2022. A small proportion (5 percent) had a turnover of over 50 million euros. The remainder (10 percent) had a turnover of between 5 and 50 million euros.
• The majority of AI companies with at least one worker (81 percent) had operating expenses of less than 5 million euros in 2022. Around 7 percent had operating expenses of more than 25 million euros. The remainder (13 percent) had operating expenses of between 5 and 25 million euros.
• This research method is still under development, and therefore these figures remain provisional.

Chapter 4: Educational programmes featuring AI

• Statistical information on educational programmes that focus on AI (e.g. numbers of current students and graduates) is produced by identifying - using existing data on educational programmes - which programmes have AI as their core focus (‘AI-narrow’), and programmes in which AI forms one aspect, but is not the core focus (‘AI-broad’, which includes all AI-narrow programmes).
• Students enrolled in AI-narrow programmes were a very small proportion (4-7 percent) of the total number of AI students (AI-broad).
• The number of students enrolled in AI-broad programmes increased by 23 percent between 2018/’19 and 2022/’23. The number of students in AI-narrow programmes doubled between 2018/'19 and 2023/'24.
• AI-broad programmes had a total of nearly 104 thousand students enrolled in the 2023/’24 academic year.
• Within AI-broad, programmes concerning ‘information technology general’ had the most students in 2023/'24 (34 thousand), followed by programmes on ‘marketing, commercial economics’ (21 thousand students).
• In AI-narrow programmes, the share of women was around 22 percent during the period studied. The majority of students were enrolled in Bachelor’s or Master’s programmes.
• In higher education, 128 thousand international students were enrolled in the 2023/’24 academic year: 18 thousand in AI-broad and 2,500 in AI-narrow.
• Choices around how to define which educational programmes focus on AI affect the numbers greatly. Because this definition remains under development, these figures on AI study programmes remain provisional.

Chapter 5 Position of students leaving AI-related programmes in the labour market

• We looked at students who had left a study programme that includes AI (i.e. a programme where AI is at least one aspect of the programme, even if it is not the only focus of the programme), and how they fared in the subsequent years.
• The number of students leaving AI study programmes in the academic years 2018/'19 to 2022/'23 ranged between around 15 thousand (2019/’20 academic year) and nearly 22 thousand (2022/’23 academic year).
• Around two-thirds (67 percent) of them left with an AI diploma. In the case of ‘AI-narrow’ programmes (i.e. programmes where AI is the core focus), the share was 73 percent.
• Women were more likely to graduate with an AI diploma from an AI programme (77 percent) than men (63 percent).
• Most of the almost 16 thousand graduates from AI programmes in the 2018/'19 academic year (with or without diplomas) found employment immediately. Self-employment or living on welfare were rare, and remained so for the first four years after leaving such a study programme.
• Students who leave with a degree from an AI study programme were most likely to find employment in the information and communication sector, specialised business services sector or trade sector. Those leaving without a degree from an AI study programme were mainly employed in trade (particularly in supermarkets) and in renting/leasing and other business services (particularly for temporary employment agencies).
• Among international students leaving AI study programmes, 26 to 29 percent found employment in the Netherlands, but the majority left the Netherlands.
• The majority of workers who had left an AI study programme in the previous four years earlier were working 35 hours or more.
• Choices around how to define which educational programmes focus on AI affect the numbers greatly. Because this definition remains under development, these figures on students leaving AI study programmes remain provisional.

Chapter 6 Demand for workers with AI skills

• In order to understand the demand for workers with AI skills, we assessed the effectiveness and feasibility of a method that, using job vacancies and modelling, can distinguish AI vacancies from non-AI vacancies.
• An ‘AI vacancy’ was defined as ‘a job vacancy involving the use or production of AI systems (based on the OECD definition). One criterion is therefore that the job, as described in the job vacancy, can only be done by a person with (in-depth) knowledge of AI systems.’
• Various models were trained using a manually labelled set of AI vacancies and non-AI vacancies. The best-performing model was a machine learning model using tf-idf encoding.
• This model was applied to a dataset of 7.5 million online job ads, which produced a set of 8,725 AI job ads in the period 2018-2024.
• An increase in AI vacancies can be seen over the period 2018 to 2022, with a peak at the beginning of 2022. After this, the number of vacancies drops, and remains stable at around 430 per quarter.
• The total number of AI vacancies between Q1 2018 and Q2 2024 was highest in the provinces of Noord-Holland (2,770), Zuid-Holland (1,435) and Noord-Brabant (1,205).
• Among AI vacancies, the five most common occupational groups are: systems analyst; statistical and mathematical specialists; software developers; professors and other teaching staff in higher education; and managers in the field of information and communication technology.
• The most common industrial sectors for companies with AI vacancies are: education; information and communication; specialist business services; trade; and manufacturing.
• The 10 organisations with the most AI vacancies to their name advertised 2,000 AI vacancies during the period studied. Of these, seven were universities in the Netherlands.
• This research method is still under development, and therefore these figures on AI vacancies remain provisional.

The term artificial intelligence (AI) describes machine-based systems that can, based on the inputs provided, infer how to generate output for specific purposes¹⁾. These systems are capable of making predictions, providing advice, or taking decisions that affect our physical or virtual environment, and they have some degree of autonomy. Examples of AI systems include autonomous robots, self-driving vehicles, machine learning models used for data analysis, AI-driven image analysis and generative AI models that produce text and/or images based on a prompt.

AI is an example of a systems technology, and in this sense it can be compared with electricity or the internal combustion engine²⁾. Systems technologies are technologies that have a major impact on society which is not fully understood when they are first introduced. According to one OECD study, AI could contribute to scientific progress, economic productivity and growth, improvements in health care and education, and climate change mitigation efforts³⁾. At the same time, the potential risks associated with AI include increased opportunities for criminals (such as online criminality), the spread of disinformation, the undermining of privacy and increased societal inequality³⁾.

AI has been high on the agenda of policymakers in recent years^4),5). It is now an integral element of the European data and digitisation strategy, and new legislation on AI has been drafted in order to regulate its us^6),7). In the Netherlands, the Netherlands AI Coalition (NLAIC) has been established, with the aim of placing the Netherlands at the forefront of expertise in and the application of AI in order to promote prosperity and well-being. One of the ways in which this ambition is taking shape is through a Growth Fund proposal involving approximately 150 million euros. This has been honoured and its implementation will be overseen by the AiNed Foundation.

A new technology with the potential impact of AI demands systematic, ongoing monitoring. However, the statistical data that we need in order to quantify the impact of AI remains patchy. Statistics Netherlands (CBS) is committed to addressing this gap and to developing and implementing a coherent system of monitoring for AI. Some small-scale projects have already been completed, at the request of NLAIC and TNO (the Netherlands Organisation for Applied Scientific Research). A survey of the AI indicators currently available was conducted in 2022⁸⁾. In the same year, a table set with a statistical description of companies receiving AiNed grants was provided⁹⁾. In 2023, statistics on Dutch companies seeking to fill AI-related vacancies were delivered, and an investigation of the use of AI technology by Dutch companies with and without AI vacancies was completed¹⁰⁾.

The current report is another step towards the development of a national AI monitor. It reports on the development of methods for creating new statistics, but it also provides figures. The report focuses on companies that produce AI, companies that use AI, educational programmes that focus on AI and how these are feeding through into the labour market, and the demand for workers with AI skills. The study was commissioned by TNO and also represents a contribution to the monitoring of the AiNed programme.  

Outline

The publication is structured as follows. Chapter 2 focuses on the use of AI technologies by Dutch companies and provides an insight into what these technologies are used for and how they have been obtained. It also looks at companies that do not use AI technology, whether they have considered using it and, if so, why they decided not to. Chapter 3 presents a new method for identifying ‘AI companies’ (companies whose main activity is the production of AI technology). The AI companies identified in this way are then described in terms of their demographic and business characteristics. Chapter 4provides key figures on educational programmes that include AI. These include figures on the number of students enrolled in study programmes in which AI is a central theme or addressed as part of a broader curriculum, and numbers of graduates. Chapter 5 builds on this, looking at the labour market for graduates of these study programmes. It looks at the position of these graduates in the labour market one year after graduating and which sectors do they end up working in, for example. Finally, Chapter 6 presents a new method of classification for job vacancies involving AI. Based on this, the chapter also provides initial descriptive statistics on the demand for workers with AI skills. 

This chapter examines the use of AI technologies by Dutch companies with ten or more employees.

2.1 Use of AI technology

In 2024, 22.7 percent of companies with ten or more employees used one or more of the seven AI technologies listed above; an increase of nearly 9 percentage points compared to 2023. Between 2020 and 2023, the use of AI technology was relatively stable, with a small spike in 2022.

The AI technologies most often used by companies in 2024 were text mining (13.5 percent) and natural language generation (12.3 percent). Compared to a year earlier, the share of companies using text mining was two and a half times greater. The use of natural language generation nearly tripled in 2024, compared with the year before. The use of speech recognition increased from 3.7 percent in 2023 to 6.5 percent in 2024.

Companies with a larger number of employees were more likely to use AI technology in 2024 than smaller companies. The use of AI technology was highest among companies with 500 or more employees (59.2 percent) and lowest among companies with ten to nineteen employees (17.8 percent). This pattern was also present in the years before, between 2021 and 2023 (not shown in figure).

The Information and Communication industry saw the most frequent use of AI technology in 2024, at 58 percent. Just a year earlier, 37 percent of companies in this industry used AI technology. Companies in specialised business services and financial services also used AI technology relatively often in 2024 (39.8 and 37.4 percent, respectively). As in the ICT industry, the use of AI technology in these industries was lower (24.8 and 27.4 percent) in 2023 as well.

Eurostat examines the use of at least one of seven AI technologies by companies with ten or more employees in EU countries¹¹⁾. In 2024, there were five EU-27 countries (Denmark, Sweden, Belgium, Finland, and Luxembourg) with a greater proportion of companies using AI technology. Denmark had the highest proportion of companies using AI technology (27.6 percent); the EU-27 average for companies' use of AI technology was 13.5 percent in 2024.

2.2 Revenue share of companies with AI technology

Companies that used AI technology accounted for more than half (51.1 percent) of the total turnover in 2024 by companies with ten or more employees in sectors C through N and Q; companies that did not use AI accounted for the remaining 48.9 percent of turnover. The revenue share of companies that used AI technology was greater among larger companies. For example, among companies with ten to nineteen employees, those that used AI technology accounted for 18.7 percent of turnover, compared with 79.5 percent for companies with 500 or more employees. This positive relationship between revenue share and company size can be partly explained by the fact that larger companies are more likely to use AI technology.

The revenue share by companies that used AI technology varied by industry in 2024. The revenue share was highest among companies in financial services (87.9 percent), information and communication (75.3 percent) and energy, water and waste management (74.3 percent), and lowest among accommodation and food services (25.3 percent).

2.3 Purpose of AI use

In 2024, companies that used AI technology most often did so for marketing and sales (36.4 percent) and administrative processes or management tasks (30.3 percent). The least common use of AI technology was for logistics purposes (6.5 percent). Some of the response categories are different in 2023 and 2024 than in the two previous years. For three of the four purposes consistently asked after throughout the 2021-2024 period (ICT security, marketing or sales, production processes, and logistics), there is a decrease in the proportion of firms mentioning these purposes during the 2021-2023 period. It cannot be ruled out that this decrease is (partly) related to the adjustment in response options.

* provisional figures
¹⁾Not all purposes were included in every year

Across all size classes in 2024, AI technology was used the least for logistics. Companies with fewer than a hundred employees deployed AI technology most often for marketing and sales (35.5-38.8 percent). For companies with a hundred or more employees, the deployment of AI technology most often had ICT security as a goal (30.1-44.6 percent).

AI technology was deployed for a variety of purposes across all industries in 2024, but the main purpose varied by industry. In renting, buying and selling of real estate, AI technology was used most often for marketing or sales (54.1 and 40.3 percent). Meanwhile, companies active in information and communication or financial services most often deployed AI technology for research & development or innovation (46.4 and 46.8 percent). Companies in transportation and storage or human health and social work primarily used AI technology was for administrative processes or management tasks (41.4 and 29.9 percent).

2.4 Method of obtaining AI technology

There are various ways to obtain AI technology for use within a company. Software can be developed entirely by in-house employees, outside vendors can be hired to develop or modify software, or a company can use commercial or open source software. Direct use of commercial software is the most common way companies obtained AI technology in 2024: more than half of the companies (55.6 percent) obtained AI technology this way.

For companies of all size classes, direct use of commercial software was the most common method of obtaining AI technology. Compared to smaller companies, larger companies were more likely to hire outside vendors to develop or modify software.

Across all industries, AI technology was most often obtained from a commercial party. This software might then be modified by in-house employees, which happened most often in financial services. In all other industries, commercial software without further modification was used the most often. 

2.5 Reasons for not using AI Technology

For companies that considered using AI technology in 2024 but decided not to use it, ‘lack of experience’ was by far the most important motivation (74.6 percent). This was true for companies of all size classes and in all industries. Privacy also played a role in the considerations: more than half of companies with a hundred or more employees mentioned privacy as a reason for not using AI technology.

* provisional figures
¹⁾ Percentage of companies not using AI, but have considered using it

Annex: methodological consideration

Averages and 95 percent confidence intervals were calculated using R’s 'Survey' package V4.2-1. The percentages are weighted to the population of Dutch companies with ten or more employees. Confidence intervals were calculated using 'svyciprop' for confidence intervals for proportions. The ‘logit’ method was used; this method fits a logistic regression model and calculates a Wald-type interval on the log-odds scale, which is then transformed to the probability scale.

There was a slight decrease in the use of AI technology in 2023 (15.8 percent in 2022 to 14 percent in 2023). Denmark and Norway (2023 vs. 2021; see here) also reported a drop in the use of AI technology. These observations seem somewhat contradictory to the rapid technological developments currently taking place in the AI field of. It is possible that this somewhat inconsistent picture is partly related to the fact that it can be difficult for a company, or more specifically the person filling out the questionnaire, to assess whether and where in the company specific AI technologies are being used; AI technology may be part of a larger software package or system without this being evident. The survey on ICT usage in enterprises is designed such that a new sample is drawn each year; businesses with a hundred or more employees are an integral part of the sample survey. A large proportion of companies (71.4-75.6 percent) responded in two (or more) consecutive years. An analysis of subsamples of companies that responded in contiguous years showed that more than a quarter of companies reported using AI technology one year, but stopped doing so the following year. This percentage of ‘stoppers’ was higher in 2023 (43.6 percent) than in 2022 (32.7 percent) and 2024 (28.4 percent) and likely explains (in part) the observed decline in AI use in 2023. With the available data, it is impossible to verify the extent to which noise from self-reporting explains the higher rate of quitters and the decrease in AI use in 2023.

To describe the characteristics of companies in the Netherlands that produce AI, it is essential to know which companies make up that population. Although some sub-populations of AI companies are available¹²⁾, there is not yet a complete overview of AI-producing companies. Not only that, but these sub-populations are also compiled using ambiguous definitions.

In 2021, Statistics Netherlands (CBS), together with the company InnovatieSpotter, conducted an exploratory study that examined whether it was possible to identify AI companies based on the texts on their websites. Within this study, it was not possible to reliably estimate the number of AI companies. This was partly because a (too) broad operationalisation had been used in compositing the training set of AI companies, which meant that the models developed could not reliably distinguish between AI companies and non-AI companies. But since that first study, great strides have been made in modelling the identification of business types based on their website texts. Examples include the identification of innovative companies¹³⁾, online platforms¹⁴⁾, companies in the creative sector, and drone companies¹⁵⁾.

The current project goal is to use new web scraping and machine learning techniques to map the population of AI companies in the Netherlands. The project specifically targets companies that produce AI systems. Section 3.1 of this chapter describes the methodology used, section 3.2 provides a statistical description of the identified AI companies, and section 3.3 provides a methodological review of the method used.

3.1 Method

This section describes the method by which the population of AI firms was composed.

3.1.1 Conceptual delineation of AI companies

To identify AI companies, it is essential to first define what an AI system is, what an AI company is, and how to assess whether a company is an AI company in practice.

Definition AI system

This project uses the most recent definition of an AI system as established by the Organization for Economic Cooperation and Development (OECD) in 2023. CBS is thus consistent with the definition of an AI system in the European AI Regulation (2024/1689). According to this definition, an AI system is: 'a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. Different AI systems vary in their levels of autonomy and adaptiveness after deployment.’ A non-exhaustive set of examples of AI systems under this definition include: autonomous robots, self-driving cars, machine learning models used for data analysis, AI-driven image analysis, and generative AI models that produce prompt-based text and/or images.

Definition of AI company

An AI company is defined as: 'A company whose main activity is the production of AI systems.' Thus, according to this definition, a company that merely uses AI systems is not an AI company. In contrast, a company that focuses primarily on the production of AI systems is an AI company. The key question here is whether the company would not exist, or would be substantially different in nature, if it did not produce AI systems.

Operationalisation of AI company

A company is considered an ‘AI company’ if its website shows that it meets the definition of an AI company. It is impossible to avoid at least some subjectivity in manual assessment. This is especially true for the question of whether a company would not exist, or would be substantially different, if it did not produce AI systems.

3.1.2 Population and training set

To identify company websites, a very large list of websites¹⁶⁾ was linked to the CBS business registry in the first quarter of 2024. This yielded more than 713 thousand websites linked to businesses in the Netherlands. These websites constituted the population of business websites in the Netherlands. For each website, we then scraped up to 200 webpages. A training set of manually classified websites of AI companies and non-AI companies was compiled in the third quarter of 2024. The positive classifications consisted of websites of companies participating in the Dutch AI Coalition (NL AIC) or those that had received a grant through the AiNed Investment Programme, supplemented with the websites of AI companies found through a Google search. One condition was that all of these companies met the criteria for the operationalisation of AI companies. The negative classifications consisted of company websites from the aforementioned sources that did not meet the standard of operationalisation of an AI company, supplemented by a sample of non-AI company websites. To further extend the training set, we used it to train a logistic regression model and applied that to the population of corporate websites. The websites with a high probability of belonging to an AI company were reviewed and added to the training set. This yielded a final training set of 294 websites of AI companies and 2 436 websites of non-AI companies.

3.1.3 Development of the model

Different types of models were trained on the final training set. Of these models, the random forest model performed best and was therefore applied to the population of corporate websites. However, the first random forest model still produced many false positives. A manual review of 200 random potential AI websites showed that only four were actually from AI companies. In an effort to reduce the rate of false positives, six more random forest models were trained on training sets that varied in the number of positive and negative classifications. For the negative classifications, this involved an additional distinction between websites that did or did not have characteristics of websites of AI companies.

The scientific literature describes positive results for the combined use of (multiple) models. After all, each model makes classifications based on different properties¹⁷⁾. Because a combination of models was expected to be better able to distinguish AI- from non-AI websites, the choice was made to use a combination. The choice of specific models in the combination was made by evaluating which (combinations of) models yielded the smallest possible subset of potential AI websites, which did include as many known websites of AI companies as possible. A website was then identified as a ‘potential AI website’ if it was given a high probability of belonging to an AI company by at least six of the seven models. This resulted in a list of 1,281 potential AI websites, which included more than 90 percent of websites that had already been identified. After manually reviewing this list, 293 were actually found to belong to an AI company. Combined with the websites in the training set, this yielded a population of 450 AI company websites.

3.2 Results: statistical description of AI companies

This section describes some demographic and financial characteristics of the identified AI companies. For this purpose, the websites of AI companies (Section 3.1.3) were linked to Statistics Netherlands’ business register and enriched with revenue data and other financial information for the respective companies. At the time of analysis, revenue data were available until 2023; other financial information was only available until 2022. The 450 websites belonged to 402 different AI companies operating in the Netherlands in 2024. This section provides some key findings. A more detailed statistical description of these AI companies can be found in the table set accompanying this chapter. All figures are provisional, as the methodology used could be improved in the future (see Section 3.3).

3.2.1 Demographic characteristics

Of the AI companies, 97 percent belonged to small and medium-sized enterprises in 2024¹⁸⁾. Only 3 percent of the AI companies had 250 or more employees.

Most AI companies belonged to sector J Information and communication industry (63 percent). Other common industries were sectors M Specialised business services (23 percent) and C Manufacturing (5 percent).

In most cases, an AI company was a private company under Dutch law (83 percent). A smaller proportion were sole proprietors (12 percent) or had some other legal form (5 percent).

Nearly half of the AI companies (47 percent) had been founded within the past five years. About one in ten AI companies had existed for more than 15 years.

Most of the AI companies were located in the provinces of Noord-Holland (32 percent) and Zuid-Holland (24 percent). Utrecht (11 percent) and Noord-Brabant (10 percent) also had relatively high numbers of AI companies.

Most AI companies (95 percent) were headquartered in the Netherlands. The remainder (5 percent) of the companies belonged to a parent company based abroad. A significant portion of this group (37 percent) belonged to a parent company from the United States.

3.2.2 Financial characteristics

This section describes some financial characteristics of AI firms, such as turnover, revenues, wages, operating expenses, operating results, and value added. Because a number of industries fall outside the observational domain of financial data, only the following industries are reported: wholesale and commission trade, transportation and storage, accommodation and food services, information and communication, specialised business services, other business services, and other services. Because of missing values, the revenues, wages, and operating expenses for some companies have been imputed. Additionally, due to a high number of missing values for smaller companies, operating results and the value added are only reported for businesses with ten or more employees. For more background and details, see the Explanation tab in the table set accompanying this chapter.

By 2023¹⁹⁾, the majority of AI companies (43 percent) had a turnover of between 100 thousand and 1 million euros. Compared to other companies in the Netherlands, AI companies were more likely to have higher turnover.

Of the AI companies with at least one employee, the majority (85 percent) had revenues²⁰⁾ of less than 5 million euros in 2022²¹⁾. Only a small portion (5 percent) had revenues of more than 50 million euros. The remainder (10 percent) had revenues between 5 and 50 million euros.

Of the AI companies with at least one employee, the majority (81 percent) had less than 5 million euros in operating expenses in 2022²²⁾. About 7 percent had operating expenses in excess of 25 million euros. The remainder (13 percent) had revenues between 5 and 25 million euros.

Of the AI companies with at least one employee, the majority (94 percent) had less than 10 million euros in wage expenses in 2022²³⁾. Only a small portion (6 percent) had revenues of 10 million euros or more.

More than half (55 percent) of AI companies with 50 or more employees had operating results²⁴⁾ of less than 2.5 million euros by 2022. Over one-third (37 percent) had higher operating results.

More than half (63 percent) of AI companies with 50 or more employees had value added²⁵⁾ of less than 23 million euros by 2022. Almost 30 percent had higher value added.

3.3 Methodological review

This section provides a methodological review of the method used as described in Section 3.1.2. One question raised here is to what extent this method makes it possible to identify the entire population of AI companies in the Netherlands.

3.3.1 Use of website texts

The method we used identified AI companies based on website texts. According to figures from Statistics Netherlands (CBS), 86 percent of companies with two or more employees in the ICT sector had a website in 2024. Since it can be assumed that this is also true for AI companies, it is likely that the vast majority of those companies in the Netherlands have a website that can be used to identify the company as an AI-producing company. However, while the list of websites linked to the business register is very extensive, it did not include all websites of companies in the Netherlands. This was evidenced by the fact that nearly 140 of the AI websites found manually did not appear in the population of company websites. One reason may be that websites were created and/or companies were established after the link with the business registry was made (March 2024). In addition, when companies have a website that represents only a part of the company's activities, it is possible that the production of AI systems may be mistakenly identified as the main activity, and therefore the company may be mistakenly classified as an AI-producing company. It is unclear for how many companies this is the case. But despite these caveats, the method, which supplements existing sources with newly found AI-producing companies, is likely to provide adequate insight into the current population of those companies in the Netherlands. Since it is plausible that the most influential AI companies are affiliated with the Dutch NL AI Coalition and/or have received a grant through the AiNed Investment Programme in recent years, no significant companies are expected to be missing.

3.3.2 False positives and false negatives

A combination of random forest models proved most appropriate for identifying potential AI websites. The models in the combination did still yield a relatively large share of potential AI websites that turned out not to be from AI-producing companies after all (‘false positives’). These false positives were removed from the final population of AI companies through manual review. However, the combination of models may have also missed websites that did belong to AI-producing companies (‘false negatives’). Because those companies are only a (very) small part of the total company population in the Netherlands, it is rather difficult to determine the percentage of false negatives.

3.4 Conclusion and recommendations

This project used web scraping and machine learning to identify a population of 402 AI-producing companies operating in 2024. It is possible that some AI companies were not identified by the current method, but these are not likely to be large companies.

The current method could be improved in a number of ways in the future. One time-consuming step is the manual review of all potential AI websites. As the number of AI companies increases in the future, the assessment time required will also increase. It is therefore important to investigate whether it is possible to reduce the share of false positives. One starting point for this is to use the full set of AI websites identified in this project to train a new model. The linking of Dataprovider.com's list of websites to the business directory could also be improved. These improvements may help reduce the proportion of false negatives. Additionally, the assessment of whether a company is an AI-producing company is somewhat subjective (see Section 3.1.1). In this study, the manual assessments were done separately by two people. To increase the reliability of the ratings, future work could look at ways to increase consistency between the various classifications.

Finally, it is important to mention that AI is a constantly evolving phenomenon. As a result, new AI terms may emerge over time that are not recognised by the currently developed models. Additionally, common terms on websites of AI-producing companies could increase in use on websites of non-AI companies and vice versa. As a result, models that perform well now may not perform as well in the future. It is important to continue to evaluate the effectiveness of the developed models, and adjust them as needed.

4.1 Key figures AI in education

The key figures for educational programmes featuring AI are the number of enrolled students and  the number of graduates. These are listed in table 4.1.1 along with the total of all enrolled students and graduates in the Netherlands. The table distinguishes between two categories: ‘AI-broad’ programmes and ‘AI-narrow’ programmes. ‘AI-broad’ involves a selection²⁶⁾ of all educational programmes in which AI is addressed as part of a broader curriculum. ‘AI-narrow’ consists of educational programmes where AI is a central theme. AI-narrow is thus a subset of AI-broad.

The number of students currently enrolled is many times greater than the number of graduates. This is because enrolled students include both first-year students and senior students, while graduates are exclusively senior students at the end of their studies.

The table set for this chapter contains too much data for the scope of this chapter. We have presented the key findings below.

4.2. Explanation of choices made

4.2.1. Selection of educational programmes

For this chapter, we made a selection of educational programmes with an AI component as well as programmes where AI is at the core of the curriculum. This selection was based on the international classification for classifying education of UNESCO. Within this classification, educational programmes are grouped based on their content. This is done at three levels: broad field, narrow field and detailed field. Statistics Netherlands has further subdivided the detailed fields by section. Some of these sections contain programmes from a single type of education, while others contain programmes from multiple types of education. We selected the AI programmes based on these sections. The reason for this is that tracking individual programmes over time can be difficult. Programmes can be split up into components or their names can be changed, for example, and keeping track of these changes is time-consuming. Not only that, but new educational programmes are added every year. These would all have to be assessed to ascertain whether they are AI programmes. This is unnecessary when looking at sections, as the division of programmes into sections has already been done. 

We have opted for two delineations of AI programmes based on sections. The category of ‘AI-narrow’ corresponds to the section ‘artificial intelligence and knowledge technology’, which consists solely of educational programmes where AI is the core component. The category of ‘AI-broad’ consists of AI-narrow, supplemented by programme sections where AI is part of the curriculum but not central to it. Statistics Netherlands made an initial selection of sections for ‘education featuring AI’ (AI-broad) in consultation with the Netherlands Organisation for Applied Scientific Research (TNO). These sections were then supplemented with programmes from an inventory of AI programmes by the Netherlands AI Coalition (NLAIC)  from 2022. Finally, in consultation with TNO, we selected sections ‘in which there is room for AI-related skills.’ 

The sections and underlying programmes included in AI-broad are listed in this table set. The selection of these sections has a major effect on both the scope and development of AI-broad. However, for follow-up analyses, the AI-broad selection still needs to be finalised with input from subject matter experts.

4.2.2. Enrolments and graduates

In this chapter, we present figures on student enrolment in educational programmes featuring AI. In order to ensure that this large amount of information remains easy to read, the figures on graduates²⁷⁾ are only included in the table set on education featuring AI. When interpreting these figures, it should be noted that changes in the number of first-year students are not immediately evident for various reasons. In the case of the figures on enrolled students, it is because they include not only first-year students but also more senior students. With the figures on graduates, it is because these refer to students at the end of their studies.

4.2.3. Different table sets

Information on enrolled students and graduates is presented separately in the table set  that accompanies this report, because these groups cannot be meaningfully compared within the same academic year. Additionally, there are separate tables for secondary vocational education (MBO) and higher education (HO), as these types of education have different subdivisions. MBO is divided by level of education; within higher education, distinctions are made between international- and non-international students, as well as the form of education (full-time, part-time).

4.3. Growth of education featuring AI

AI-narrow is a subset of AI-broad. Educational programmes that fall within AI-broad, but not within AI-narrow, are referred to here as ‘AI-applied’. The number of people enrolled in AI-narrow were a very small proportion (4 to 7 percent) of the total number of AI students (AI-broad). 

The number of students enrolled in AI-broad increased from 2018/'19 to 2022/23'. In 2023/'24, the number of students enrolled was similar to that of the previous year. The total number of enrolled students in the Netherlands also increased during this period, but this increase was less pronounced than that in the number of students in AI-broad.

The number of students in AI-narrow doubled from 2018/'19 to 2023/'24. Each year, it increased more than the total number of students did, as well as more than the number of students in AI-broad. There was also no stagnation as seen with AI-broad. The category of AI-narrow was clearly on the rise, although actual student numbers in AI-narrow remained relatively low (see Figure 4.3.1).

4.4.Most AI students in IT General

AI-broad had a total of 103,960 enrolled students²⁸⁾ in the 2023/'24 academic year. These were distributed between several fields of study and the underlying programmes. A section can contain programmes from different education types (MBO, higher vocational education (HBO) and university education (WO)), but this is not the case for every section.

Within AI-broad, ‘information technology general’ was the largest section in 2023/'24 for both HBO and WO students (34 thousand), followed by ‘marketing & commercial economics’ (21 thousand) with mainly HBO students. The section ‘application building & programming’ consisted of mostly MBO students. This section included a new MBO programme called ‘software developer’, which managed to draw a lot of students (see further in this section). 'Artificial intelligence & knowledge technology' (i.e. AI-narrow) ranked fourth among the selected AI sections with 7 thousand students enrolled.

Student numbers for most sections remained stable over the period starting in 2018/'19. The four largest sections were the main determinants in the development of AI-broad. ‘Information technology general' steadily increased from 2018/'19 to 2022/'23, after which the number of students in this section stagnated. The number of students in 'marketing & commercial economics' decreased slightly, starting in 2020/'21. The aforementioned new MBO programme ‘software developer’ started in 2019/’20 within the section ‘application building & programming.’ As a result, the number of students within this section increased substantially, although its contribution within the AI-broad total remained small. AI-narrow was ranked fourth and steadily increased in student numbers over these five years.

4.5.Profile of AI students

Generally speaking, AI students were men²⁹⁾ studying full-time for a Bachelor’s or Master's degree.
Within AI-broad, the proportion of women was around 22 percent during the period investigated. The majority of students were enrolled in Bachelor’s or Master's degree programmes, although the new MBO programme of ‘software developer’ did cause a marked increase in the number of MBO students. In higher education, AI-broad programmes were taken almost exclusively full-time (95 percent) in 2023/’24; part-time and dual education were very rare.

Students in AI-narrow were similar in this regard, except that the proportion of women was slightly higher, hovering between 31 and 34 percent. Students in AI-narrow studied exclusively full-time and were enrolled in an Bachelor’s or Master’s programme. The number showed no meaningful change over the period.

4.6. International AI students

4.6.1. Number of international AI students increases

As international students are distinguished only within higher education, comparisons with non-international students are made only for this type of education.

There were 128 thousand international students enrolled in higher education in the 2023/'24 academic year (see here); 18 thousand within AI-broad and 2500 within AI-narrow. That year, the proportion of international students in higher education was 16 percent (see: here). It was 20 percent within AI-broad and 35 percent within AI-narrow. Thus, AI students in higher education were relatively often international students.

The number of international students increased sharply over these years (see figure 4.6.1.1). Between 2018/'19 and 2023/'24, international enrolment increased by 49 percent. Within both AI-broad (including AI-narrow) and AI-narrow, international enrolment increased more rapidly than overall enrolment (see figure 4.6.1.1). Enrolment increased by 103 percent (i.e., more than doubled) for AI-broad and by 222 percent (i.e., more than tripled) for AI-narrow.

The proportion of women within the group of international AI students remained unchanged at 33 percent; slightly higher than the proportion of women within non-international AI students (22 percent).

4.6.2. International AI students choose different programmes

The proportion of international students in AI-related educational programmes (AI-broad) in higher education was 20 percent in 2023/'24 (see 4.6.1). That proportion varied widely by section.

The sections (AI-broad) with the highest numbers of enrolled AI students (both international and non-international) were ‘information technology general’, and ‘marketing & commercial economics’ (see figure 4.4.1). International students were more likely to choose different educational programmes and often ended up in ‘computer technology & computer engineering’ and ‘artificial intelligence & knowledge technology’ (i.e. AI-narrow). Consequently, the student population in these sections was more international in nature.

Chapter 4 described educational programmes featuring AI based on enrolment numbers. In the following paragraphs, we will describe how students of AI education have fared in the years after graduating or leaving their programmes. For this purpose, we used the figures on students who left in academic years 2018/'19 through 2022/'23.

The annual number of students leaving AI-broad³⁰⁾ programmes between 2018/'19 and 2022/'23 was between approximately 15 thousand (academic year 2019/'20) and nearly 22 thousand (academic year 2022/'23) students. The category of ‘AI-broad’ involved a selection³¹⁾of all educational programmes with an AI component. In sum, more than 91 thousand students were involved over the years investigated. The number of students leaving AI-narrow programmes was smaller during this period, totalling over all years 4,250 students. The category of ‘AI-narrow’ consists of programmes where AI is central to the curriculum. AI-narrow is a subset of AI-broad.

The figures in this chapter mainly concern students leaving AI-broad (including AI-narrow) programmes. Where possible, students leaving AI-narrow programmes are discussed separately. In the table set that accompanies this chapter, more data is provided than we can cover here. This chapter is limited to the most important findings.

5.1 Two-thirds graduated from educational programmes featuring AI

Around two-thirds (67 percent) of students left educational programmes featuring AI with an AI diploma³²⁾ , while one-third left without a matching diploma. Across all years, 73 percent of all students who left AI-narrow did so with a degree. That proportion was therefore slightly higher for AI-narrow than for AI-broad.

The proportion of students who left AI education with an AI degree is relatively stable (figure 5.1.1). The academic year 2019/’20 saw the highest proportion of students leaving with a degree (70 percent); the lowest proportion was 64 percent in the 2021/’22 academic year (during the COVID-19 pandemic).

Women were more likely to graduate from AI education with an AI degree (77 percent) than men (63 percent). In other (non-AI) courses, women also graduated more often than men, in both secondary vocational education (MBO) and higher education.

Whether students left AI education with an AI diploma depended on the type of education (figure 5.1.2): the proportion of students who graduated was lowest in MBO, and highest in university education (WO).

This difference was caused in part by the start of a new AI programme in MBO in the 2019/’20 academic year: ‘software developer’, part of the ‘application building and programming’ section. Most persons who left this program were still ungraduated, as the period we examined was quite short for obtaining a diploma. 

5.2 Majority of students leaving AI programmes became employees

Of those students who left AI education, it was examined immediately after exit and in subsequent years whether they went to work as employees or became self-employed, whether they received benefits, and whether they returned to education. We also investigated whether they were still registered in the Netherlands in the population register (BRP). For each person, a check was carried out to determine whether they were enrolled in education and the BRP on 1 October; with respect to employment and benefit status, a check was carried out for the month of October.

For each year, it was determined whether individuals returned to education that year or earlier³³⁾ . If they did not, we checked whether they were still registered in the BRP on the reference date. If individuals did not return to education and were registered in the BRP on the reference date, they were included in the labour market population. Within this group, a prioritisation was made, as a person can be both employed and self-employed, and can both work and receive benefits. This prioritisation is as follows: 1. employee, 2. self-employed, 3. benefits recipient. Some individuals did belong to the labour market population, but they were neither working nor receiving benefits. This group is included in the presentation of the results, along with the group not registered in the BRP at the reference date, under the heading ‘other: unemployed, no benefits, not in BRP’.

In the 2018/'19 academic year, 15,800 students exited AI-broad education, both with and without degrees. Most of them immediately found employment. Self-employment or receiving benefits were rare, and remained so for the first four years after exit.

MBO students and HBO students in particular returned to education, usually after leaving without a diploma: of those who left without a diploma, more than a quarter (28 percent) returned to education versus only 5 percent of those who left with a diploma. The results are similar for other cohorts, and will therefore not be discussed.

Students who had left AI-narrow (540 persons total) were more likely to end up in the category ‘other: no work, no benefits, not in BRP’. This was probably because programmes within AI-narrow were relatively often followed by international students. After leaving education, this group was more likely to leave the Netherlands. They did not always deregister from the BRP when leaving. This made it appear that these individuals were neither working nor receiving benefits, when in reality they had already left the Netherlands. Some of the international students did deregister from the BRP. They also ended up in this category (see also Section 5.4).

5.3 Most employees end up in information and communication sector

For students who left educational programmes featuring AI and started working as employees, the relevant economic sector was recorded. Some sectors received very few employees from AI education. Additionally, to prevent data being traceable to individual persons, some sectors will not be shown in this chapter. These include, for instance, the sectors Agriculture, forestry and fishing, Mining and quarrying, and Real estate activities.

October 2023 was used as the reference date for determining the sector. All cohorts were examined. This means that the number of years that individuals were employed varies: the cohort 2022/'23, consists of people who had become employees immediately after leaving education, while the 2018/’19 cohort consisted of employees who had left four years earlier. In October 2023, most people were employed in the Information and communication sector.

Those who left AI education with an AI degree and those who left without a degree tended to work for companies in different sectors. Of those who had left with a degree, most worked in the following sectors: Information and communication, Specialised business services, and Trade. Of those who had left without a degree, the largest number of people were employed in the Trade sector (especially in supermarkets), as well as in Renting, leasing and other business support services (especially at temporary employment agencies). Those leaving education were more likely to work in these sectors immediately after their leaving education than they were to do so a few years later. Those who left education without a degree were also relatively likely to work in the information and communication sector in 2023.

There are some differences between AI-broad and AI-narrow in terms of the sectors where employees end up. These differences are illustrated using the 2018/'19 cohort four years after graduation in October 2023, which includes both people with and without a diploma.
People leaving AI-broad or AI-narrow education were equally likely to end up working in the Information and communication sector. However, people leaving AI-narrow were significantly more likely to work in education – and particularly university education. It is possible that these are individuals who went on to work at a university as PhD candidates after completing their studies. This is also consistent with the relatively large numbers of AI job vacancies in education (see chapter 6). People leaving AI-narrow were less likely to enter the manufacturing or trade sectors than those leaving AI-broad.

5.4 Most international AI students left the Netherlands

International students are only distinguished within higher education. The following findings therefore relate exclusively to AI programmes within HBO and WO education.

International students were as likely to leave AI education with an AI degree (67 percent) as non-international students (68 percent). This was true across all cohorts. It was the international students who mainly –and usually immediately after leaving– ended up in the category ‘other: no work, no benefits, not in BRP’. As indicated in Section 5.2, most of these were individuals who no longer resided in the Netherlands but had not deregistered from the BRP.

Over a quarter of international students (26 to 29 percent, across all cohorts) found employment in the Netherlands. This proportion was close to the average of international students in the Netherlands. One or two internationals returned to education. Like their non-international peers, international students rarely became self-employed, and they almost never received benefits.

International students from all cohorts were more likely than non-international students to choose programmes in the sections³⁴⁾; Computer Science & Computer Engineering and Artificial Intelligence & Knowledge Technology (i.e. AI-narrow). They were less likely to choose courses in the sections Marketing & Commercial Economics and General Information Technology. International students' preferences for specific programmes may have helped to determine the specific sectors in which they came to work post-education.

In October 2023, international students who started working in the Netherlands were as likely to be employed in the information and communication sector as non-international students. For this, we looked at all cohorts on the reference date. In 2023, international students were more likely than non-international students to work in manufacturing, specialised business services, financial services, and education. In the latter sector, they were often employed as PhD candidates. International students were proportionally less likely to enter the sectors Public administration and public services and Trade.

5.5 Most AI graduates worked at least 35 hours per week

Working hours per week were generally similar for workers across the different cohorts. For this reason, we have opted to discuss those leaving educational programmes featuring AI in the 2018/’19 cohort, since this cohort had the most reference dates available.

Immediately after leaving AI education, people were more likely to work in jobs with shorter hours, especially those leaving without a degree. This was likely because former AI students were more likely to work in the sectors trade (including supermarkets) and renting & other business support services (including employment agencies). These sectors are characterised by flexible jobs with shorter hours. Two years after leaving education, those without a diploma had either found more stable employment than their former jobs at supermarkets or employment agencies, or they were back in education. At that point, the graduates had been active in the labour market for a little longer. The working hours per sector are shown below, taken over the four years after leaving education. These provide a picture of the more stable working hours.

Most employees worked 35 hours per week or more. There were more jobs with shorter hours in the food and accommodation services sector, but there were very few AI employees still working there four years after leaving education. Part-time jobs with longer hours (20 to 35 hours) were somewhat more common in the education and health & social work sectors. In these sectors, shorter working hours were more common.

Of those leaving AI-narrow, 79 percent were working 35 hours a week or more after four years. The number of people in this group is too small to provide a breakdown by individual sectors.

In order to understand the role of AI in today’s labour market, it is important to have a clear picture of the demand for workers with AI skills. If the supply of workers with AI skills does not meet the demand, this could lead to a number of challenges, including a delay in the adoption of AI technologies by businesses and agencies. One way of measuring this demand is to survey the number of AI vacancies in the Netherlands. When counting job openings, standard classifications – such as the ISCO classification of occupations – are often used to distinguish job openings for specific occupational groups. However, conventional classifications do not include a specific AI category. For that reason, this chapter examines the effectiveness and feasibility of an alternative method that – based on job postings and modelling – distinguishes AI vacancies from non-AI vacancies. To this end, a conceptual delineation of the term AI vacancy is first provided in Section 6.1. Next, different methods of identifying AI vacancies are described and compared in Section 6.2. Using those methods, we assess the feasibility of identifying AI vacancies based on vacancy text data. The final section, 6.3, shows the results of the model that performed the best; we will also discuss the characteristics of the AI vacancies identified.

6.1 Conceptual delineation of AI vacancies

In order to identify AI vacancies, it is important to create a conceptual delineation. We must first define the terms ‘AI system’ and ‘AI vacancy’ and operationalise them.

6.1.1 Definition of AI system

As in chapter 3, we have opted to use the most recent definition of an AI system as established by the Organisation for Economic Cooperation and Development (OECD) in 2023. Statistics Netherlands is thus consistent with the definition of an AI system in the European AI Regulation (2024/1689), which came into effect on 1 August 2024. According to this definition, an AI system is ‘A machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments.’³⁵⁾ A non-exhaustive set of examples using this definition includes: autonomous robots, self-driving cars, machine-learning models used for data analysis, AI-driven image analysis, and generative AI models that can produce text and/or images based on prompts.

6.1.2 Definition of AI vacancies

An AI vacancy is defined as follows: 'A vacancy for a job in which the use or production of AI systems (according to the OECD definition) is a core component. This requires that the job, as described in the job posting, can only be performed by a person with (in-depth) knowledge of AI systems.’ This means that a central aspect of the job is the production or use of AI systems. Production of AI systems can include writing code to develop, maintain, or improve AI systems; testing already existing AI systems; or contributing content to the development process of AI systems. Using AI systems can include data analysis with machine learning (ML) models or doing research using AI systems.

Vacancies for employees who are involved in the production of AI systems, but who do not require knowledge of AI systems – such as managers, IT specialists, or employees who make professional contributions in the scope of the AI system – are not considered AI vacancies. The same applies to job openings for workers who do have (in-depth) knowledge of AI systems, but who are not directly involved in the production or use of AI systems, such as ethicists, legal professionals, and journalists specialising in AI systems.

6.1.3 Operationalisation of AI vacancies

The following operationalisation was used in the manual classification of vacancies: 'The vacancy text and title demonstrate that the vacancy meets the definition of an AI vacancy.' Using manual assessment, it is impossible to avoid at least a degree of subjectivity. This mainly applies to determining the following: whether the job posting indicates that AI systems are central to a job; whether AI systems make up a large part of the job description; and whether (in-depth) knowledge of AI systems is a requirement for the position.

6.1.4 Data source: differences between online job ads and vacancies

This study used a sample of more than 7.5 million online job ad texts from the company Textkernel. These texts were posted on the internet between Q1 of 2018 and Q2 of 2024. The model-based approach being developed to identify AI vacancies (Section 6.2) makes direct use of these online job ad texts. A weighted model was applied to the number of online job ad texts found in order in order to arrive at nationally representative vacancy figures (Section 6.3). A detailed description of the weighted model can be found here (only available in Dutch). A vacancy is defined as a job opening for which employees are being sought outside a company or institution, and who can be put to work as soon as possible. Online job ads (henceforth: ads) are announcements of open positions posted online by employers. A single vacancy can have one, multiple, or no ads.

6.2 Methodology of identification of online AI job ads

This section presents the two methods tested: an ML classification model and a search terms model. The results of these models are compared with each other as well as a method previously developed by TNO and Jobdigger. Based on this study, an assessment is made of the reliability and feasibility of model-based identification of AI vacancies. For this study, we decided to investigate only those ads with text in Dutch or English. Internships (paid or unpaid) were not included in the population.

6.2.1 Machine learning model

Several machine learning (ML) classification models have been developed to identify AI vacancies. This section provides a brief overview of the best performing ML model. For further technical details, as well as an overview of other ML models tested, Technical Annex to Chaper 6 (only available in Dutch).

Encoding

Because ML classification models use a numeric input, the job texts are first converted to numeric values using an encoding method. Of the various methods tested for this purpose, term frequency - inverse document frequency (TF-IDF) encoding was part of the best performing combination of encoding method and ML classification model. This approach involves counting how often relevant words appear in the job posting. That number is then weighted based on the frequency with which ads normally contain these words. In addition to the standard TF-IDF encoding, additional information about ad characteristics and word embedding vectors have been added. These vectors map the interrelationship between words within the job posting.

Classification Model

The best performing classification model is a combination of regularised logit models. A regularised logit model is a standard logit model that selects the most relevant features in the dataset, based on an algorithm. In the case of TF-IDF encoding, individual features correspond to words. This means that the model first selects relevant words and then determines the extent to which the presence of these words in the job posting is associated with an increased probability that the ad meets our definition of an AI ad. The logit model then assigns each input  a probability score between 0 and 1. The higher this score, the more likely it is that the job posting concerns an AI vacancy, according to the model. As in Chapter 3, we chose to work with a combination of models trained on different data sets. The average probability score of four models is calculated; if this score exceeds 0.5, the ensemble classifies the ad as an AI ad.

Selection process

We compiled a manually labelled dataset to select the best performing combination of encoding method and ML classification model. The dataset consisted of 807 ads, 235 of which were AI ads. Of this dataset, 20 percent was used as test data and 80 percent as training data. The test data was used to measure the performance of the models and remained unchanged throughout the process. The training data was used to train the classification models; it was supplemented several times during the process.

Based on the initial manually labelled dataset, multiple combinations of encoding methods and ML classification models were tested. The best performing combinations were then applied to new unlabelled data from the full dataset. The ads that emerged as potential AI ads using this method were labelled manually. In this way, the manually labelled set expanded twice: first to 958 with 253 AI ads, then to 1,092 with 274 AI ads. These iterative steps were carried out to supplement the training dataset with ads whose status was not immediately clear. The inclusion of such hard-to-classify vacancies is valuable in order to improve the performance of the model. The best performing combination of encoding method and ML classification model was chosen based on all the datasets collected.

6.2.2 Search Terms Model

We also researched a simpler search term model that only looks to see if certain AI-related words appear in the job posting. An initial list of AI terms was compiled for this purpose. Several sub-selections were then made by taking a random number of AI terms from the initial list. Simultaneously, the minimum number of required words from a sub-selection that must appear in the job posting to classify it as an AI job posting was determined. We then determined the best combination of the chosen AI-related words and required word count by comparing the performance of each combination tested on the manually labelled dataset. The initial glossary and the final selected glossary can be found in Appendix 2.

6.2.3 TNO Search Terms Model

Previously, Statistics Netherlands participated in a study on AI vacancies. In that study, TNO and Jobdigger identified AI vacancies (using Jobdigger vacancy data) and Statistics Netherlands provided a statistical description of the companies with those vacancies. The study compiled a list of ads that met one of the following two conditions: 1) At least one of the words used in the job ad appeared in a pre-established glossary (see glossary B2.2.1 in Appendix 2); or 2) at least one of the skills in the ad specified by Jobdigger appeared in a pre-established list of AI skills. It then looked at the ISCO (International Standard Classification of Occupations) code of each ad in the list. When this ISCO code occurred in a predetermined selection of AI-related ISCO codes (see B2.2.2 in Appendix 2), the ad was identified as an AI ad.

In order to evaluate whether the models developed in this project outperform the method developed by TNO and Jobdigger, this approach was also applied to the Textkernel data used in the current project. However, because the skills specified by Jobdigger are not part of the Textkernel data, the approach taken by TNO and Jobdigger cannot be fully replicated. The comparison below will present two versions of this method: one with and one without the ISCO code requirement.

6.2.4 Results of feasibility study

Comparison tested model

The different methods were compared based on their balanced accuracy score³⁵⁾ on the test dataset. We also looked at the number of positive ads identified by applying the model to the entire dataset. These results are shown in table 6.2.4.1.

The ML model is the best performing method. This model scored the highest on balanced accuracy, and would thus appear to result in the most accurate selection. Interestingly, this method also identified the smallest number of AI vacancies, suggesting that the other models resulted in an overestimate. While the search term model performs poorly in comparison, it may be preferable in certain situations due to its speed and simplicity.

Both versions of TNO’s search terms model identified a much higher number of AI vacancies than the new models. The main reason for this is that they used an extensive glossary of possible AI terms. It includes several terms that can also be used outside an AI context, such as 'statistics,' ‘cloud’, and 'data cleaning'. As a result, these models are expected to overestimate the number of AI ads significantly. The model with the ISCO filter did identify fewer AI ads, but it also achieved a significantly lower balanced accuracy score compared to the unfiltered model. The reason for this is that the list of ISCO codes appears to be too limited. Many AI ads use ISCO codes outside the pre-established filter list, e.g. ISCO code 3314. As can be seen in section 6.3.2, that was one of the most common ISCO codes for AI vacancies found with the ML classification model.

Performance of the best-tested model

The percentage of false positives (i.e. ads falsely classified as AI vacancies) in the ML classification model was evaluated using a sample of 150 AI ads marked by the model. A manual review showed that thirteen ads (9 percent) had been incorrectly classified as AI ads. Eleven of those thirteen ads were AI-related but did not meet the specific operationalisation of AI vacancies. For example: there was a vacancy for a data engineer position, which involved developing a data pipeline in close cooperation with AI specialists. Although in this case the data engineer would contribute to an AI system, the job itself does not require AI expertise.

Due to the relative scarcity of AI vacancies, it is difficult to determine the percentage of false negatives (vacancies incorrectly marked as non-AI) using similar sampling methods. However, an evaluation of all manually-labelled AI ads showed that 15 percent of positive cases were not identified as such by the model. This is further compounded by the AI ads that were overlooked due to their lack of representation in the training data. We conclude that it is likely that the model-based identification slightly underestimates the actual number of AI ads.

6.2.5 Points of improvement

This project provided us with a reliable ML classification model for identifying AI vacancies. However, the modelling approach still has room for improvement in the future. This section identifies several areas that warrant revision or further scrutiny.

Context in encoding

The TF-IDF encoding method counts how often words appear in job texts and then weights them accordingly. Consequently, this encoding method does not take context into account. That context can be expressed at three different levels: word level, sentence level, and paragraph level. Words can have multiple meanings, and sentences and paragraphs contain information that may or may not relate to the candidate sought. A lot of useful information is lost due to the lack of context in the current method. It would therefore be relevant for follow-up research to look at whether model performance improves when an encoding method is used that takes context into account. We did not do this in the current study, as it was impossible to do for the entire dataset in a realistic timeframe, given the computing power available to Statistics Netherlands.

Dependence on training data 

In TF-IDF encoding, a set of words is determined from the training data, which are then used as features in the classification model. The only words included in the final input for the classification model are those that appear in more than m texts in the entire training dataset; the parameter m is defined beforehand. This method ensures that infrequent or less relevant words do not disrupt the classification procedure. However, one disadvantage is that the results are even more dependent on the training data used. The composition of the training dataset directly affects which words are considered relevant for classification. One potential issue with this is that the model might fail to recognise AI ads if the language differs from the training data. Again, a possible solution would be to apply a more complex encoding method: one that encodes based on the overall use of text in ad texts, rather than individual words alone.

Concept drift

One issue related to the point described above is known as ‘concept drift’. AI technologies are developing rapidly and the number of possible applications is expanding constantly. That also means that future AI ads will look different compared to those of the period 2018-2024. This can be observed in the terms used in the job posting, the type of jobs advertised, and the type of employers. It is therefore important that, if the model is to be used over the coming years, the training data should be supplemented with relevant new (AI) ads. Additionally, it may be necessary to remove older, no longer relevant ads from the training data. The dataset used in this project has already demonstrated some limited degree of concept drift. Data scientist vacancies in recent years often have machine learning as a requirement, which was much less common in previous years (2018/2019). The relationship between the concepts of data scientist and AI is changing over time.

6.2.6 Conclusion

The best performing ML classification model in this study is good enough at identifying AI vacancies to produce reliable figures on the number of AI vacancies in the Netherlands. However, the model does still produce errors: false positives as well as false negatives. It is expected that the model can be improved in the future to further reduce the percentage of false positive and false negative results. Once more computational power becomes available, it is expected that the biggest improvement can be made by employing a more complex encoding method; one that takes context into account and looks at broader use of language, rather than individual words. If other encoding methods are tested in a follow-up study, it is important to retest which ML classification model best fits these new methods, too.

6.3 Statistical description of AI vacancies

This section provides a description of the AI vacancies identified using the ML classification model. The figures provide a picture of some possible applications of the AI vacancy identification explored in this chapter. These are provisional figures, as the modelling process could still be improved. Figures are rounded to the nearest five. The numbers presented relate to vacancies in the Netherlands, and were obtained by weighting the labelled results of online vacancies using the aforementioned weighting method. After applying the weighting model, the 9,430 online job ads found correspond to a total of 8,725 vacancies.

6.3.1 Number of AI vacancies over time

Figure 6.3.1.1 shows the number of AI vacancies from the first quarter of 2018 to the second quarter of 2024. An increase in AI vacancies can be seen during the period 2018 to 2022, peaking at the beginning of 2022. After this, the number drops before hovering around a value of 430. Over the time period considered, the proportion of identified AI vacancies ranged from 0.05 to 0.13 percent of the total number of vacancies in the Netherlands.

6.3.2 Characteristics of AI vacancies

Figure 6.3.2.1 reveals that, from the first quarter of 2018 to the second quarter of 2024, the total number of AI vacancies was greatest in Noord-Holland (2770), Zuid-Holland (1435) and Noord-Brabant (1205). Zeeland, Flevoland, Drenthe and Friesland had the lowest number of AI vacancies (less than 100).

Figure 6.3.2.2 shows the total number of AI vacancies per occupational group according to the International Standard Classification of Occupations (ISCO). Most AI vacancies fall within a limited number of occupational groups. The five most common groups are: 2511 Systems Analysts; 3314 Statistical, Mathematical and Related Associate Professionals; 2512 Software Developers; 2310 University and Higher Education Teachers; and 1330 Information and Communications Technology Service Managers. The top five occupational groups account for more than 70 percent of all AI vacancies found, with the remaining 30 percent distributed among 192 other ISCO codes.

Roughly 75 percent of AI vacancies were written in English, while the remaining 25 percent were written in Dutch. This distribution differs significantly from the language distribution across all vacancies, where 84 percent were Dutch and only 16 percent were English. Figure 6.3.2.3 shows that more than 72 percent of AI vacancies fall under occupational skill level 4, which is defined as ‘highly complex, specialised tasks requiring a higher or academic education level.’ About 23 percent of AI vacancies fall under skill level 3, which is defined as ‘complex tasks, requiring secondary education or higher.’ Thus, AI vacancies are almost entirely geared toward individuals with higher levels of education.

6.3.3 Characteristics of companies with AI vacancies

Figure 6.3.3.1 shows the number of AI vacancies, organised by sector of the companies posting those vacancies. The sectors with the most AI vacancies are P Education, J Information and communications, M Specialised business services, G Trade and C Manufacturing. Together, these five groups account for 71 percent of the total number of AI vacancies. In Education, the largest sector, 91 percent of AI vacancies fall into the subcategory Tertiary Education.

Comparing figures 6.3.3.1 and 5.3.2 (Chapter 5) shows that the number of students leaving AI education and the number of AI vacancies are similarly distributed across sectors. It is notable that sector P Education is the largest sector when it comes to AI vacancies, while it is smaller in terms of the number of students leaving AI education. One possible explanation is that this category includes predominately positions for PhDs, postdocs and professors. These positions are regularly filled by international candidates and not only by students leaving Dutch AI education. Chapter 5 showed that a relatively high number of AI students end up in public administration or public services (figure 5.3.2), but the number of AI vacancies in this industry is relatively small (figure 6.3.3.1). This could be explained by the fact that, while AI students are desired for jobs in the public sector, the government does not specifically ask for AI skills.

Of all AI vacancies, 86 percent could be assigned to a company or organisation. The ten organisations with the most AI vacancies to their name collectively sent out 2,000 AI vacancies over the period studied. Of these top ten organisations, seven are Dutch universities. As could be seen in figure 6.3.3.1, many AI vacancies are associated with university research and education.

Figure 6.3.3.2 shows the distribution of AI vacancies among the corresponding companies and organisations. The x-axis corresponds to the percentage of the total enterprise population and the y-axis to the percentage of the total number of AI vacancies. The figure shows that 20 percent of companies send out 80 percent of AI vacancies. Thus, the number of AI vacancies is not distributed evenly among companies and organisations. There is a long tail of companies with two or fewer AI vacancies, starting at around 40 percent.