Data Analysis of Employment Market Using Text Mining & Job Finder System

Abstract

This research report analyzes the application of text mining and job finder systems in data analysis of the employment market. A literature and source review provides an insight into how these information technologies can be used for getting an advantage in recruitment, job search, analysis of employees’ moods, and studying the employment market. In addition, this report contains the breakdown of strengths and limitations associated with the use of text mining and job finders.

Introduction

Text mining and job finder systems have become important assets for employers and candidates. Analyzing labor market data is extremely important for identifying and understanding issues, the decision-making process, and evaluating the costs and benefits of the solutions related to employment [1]. The modern era created a tendency for fast changes in required skills, which might make unexpected turns [2]. Therefore, both parties use these information technologies to gain an advantage in the labor market. On the one hand, employers can utilize text mining to identify the relevant skills directly from the existing job advertisements [2]. In addition, the companies can use that Labor Market Information (LMI) for learning what sectors have high growth potential or what training the employees need [3]. On the other hand, the candidates can utilize filters in job finder systems to find the desired job among the many offers [4]. At a glance, text mining and job finders provide multiple benefits for employment market analysis, including:

• Cost-efficiency;
• Speed;
• Audience reach;
• Easy access;
• Clarity and user-friendliness.

Therefore, the adaptation of text mining and job finders in employment market analysis seems to be a reasonable choice for both candidates and the companies. However, additional scholarly evidence and practical application examples should be provided to confirm the usefulness of text mining and job finders. Finding examples of how information technologies facilitate the employment market analysis is the primary goal of this research report.

Scope

The scope of this research report encompasses two major technologies which can be used for the analysis of the modern employment market. The first part contains a review of scholarly sources on the practical application of text mining and job finders. In particular, this section covers the use of text mining and job finders for job recommendations and matching, studying the contemporary labor market demands, and evaluating the employees’ job satisfaction. The second part covers job finder systems, particularly their use for employment market analysis from the candidate’s point of view. Finally, the strengths and limitations of text mining and job finders will be analyzed in the last part of the report.

Literature Survey

Text mining and job finders for a job recommendation and matching quality

The role of special staffing companies that connect potential candidates with employers has become increasingly significant in recent years [5]. Since every candidate has their respective strengths and weaknesses, the staffing company must carefully evaluate them before giving a recommendation to the employer. As a result, the staffing companies gather a massive amount of data related to the jobs, such as commute time, hourly rates, and relevant skillset. Kino et al. (2017) used the KH Coder tool to analyze freely available text to extract words, which affect job matching results in either positive or negative ways [5]. As a first step, the researchers analyzed the job matching datasets to extract the most frequently appearing keywords. After that, a hierarchical cluster analysis had been conducted, and a co-occurrence network had been created. Finally, a table with the most useful keywords for job matching had been formed.

Table 1. Useful keywords

Shalaby et al. (2017) proposed methods for creating a scalable job recommendation system that utilizes available implicit and explicit data to entail a multigraph of jobs [6]. The job finding system fills the gap in existing job recommendation systems by leveraging multiple contextual and behavioral signals in one unified salable graph-based architecture [6]. The system consists of three main steps: building scalable homogenous graphs of jobs, creating a weighted directed edge for each job pair, and generating recommendations [6].

The average accuracy of job recommendations based on the graph method has reached around 90% [6]. Therefore, text mining proved to be an effective instrument for data analysis and, consequently, job recommendations. The addition of graphs allowed to overcome the possible challenges of semantic relations sparsity and scalability. As a result, that synthesis provides the applicants with highly accurate and personalized job recommendations.

Jung et al. (2019) have developed a job matching system based on Korean National Competency Standard (NCS) [7]. The algorithm was designed to solve the problem of matching graduated students with their first employers. Therefore, the algorithm creators considered several aspects during its development. First of all, the students were enabled to express their knowledge and past activities at the university in their self-introduction and resume. Secondly, the students were given an opportunity to express qualifications obtained at the university as their basic competencies. Thirdly, the system provided students with customized employment information and services since they have different competencies and job expectations. Lastly, the algorithm preferred soft matching, which showed associated results if some conditions were met, to hard matching, which showed results only in the case of perfect conditions matching [7].

The researchers provided a framework of their job matching search system. According to Figure 2, it consists of three main components: job seeker database, search engine, and valuation rule [7].

A user interface was designed to facilitate the understanding of the search system. According to Figure 3, the interface showed the details of matching factors and evaluation criteria for job matching [7].

Overall, the job matching system was considered a valuable asset for employment market analysis and job search. However, the researchers created a list of policies that should be followed to apply the job matching system to the employment market successfully. First, the government was suggested to develop a personal data protection policy to provide the ground for the safe placement of quantitative and measurable data [7]. Second, the use of a job-matching system was recommended by third-party professionals, which connects the applicants with employers. Finally, the developers recommended showing ranking information based on the score achieved by the candidate, so the companies and third parties could identify suitable employees more efficiently. As a result, the promising candidates would be given a priority in getting the employment opportunity.

Text mining for skills analysis

The demand for skilled workers in the modern age is growing, and the desired skillsets for jobs are changing rapidly [8]. Therefore, both workers and employers should be aware of the required skills for obtaining an occupation and performing within it. In that regard, text mining technology can serve as a valuable asset for analyzing changing skills. To prove that notion, Darabi et al. (2018) introduced a Skill Miner System (SMS), designed for producing a picture of what skills must be possessed by a job seeker in Industrial Engineering [8]. The researchers collected over 5000 unique jobs in Industrial Engineering and created a dataset for the study. After that, keywords and phrases were extracted from job descriptions, and a more detailed analysis of skills was conducted, as shown in Figure 4 [8].

Consequently, the researchers categorized 25 top technical skills and 12 top professional skills by the SDI (Skill Demand Index), based on the frequency of mentioning in job descriptions. The technical skills can be understood as competencies taught in the university, while the professional category represents skills learned and developed through working experience. The researchers noted that the information produced by the SMS could be updated and serve well for various stakeholders, such as universities, job seekers, employers, and staffing companies [8]. For example, the awareness of currently popular technical skills can be used to incorporate relevant courses in the learning programs.

Table 2. Example of top priority skills extracted via text mining

The modern era created a need for an urgent understanding of the impact on the workforce caused by technology. This urgency stems from the so-called Industry 4.0 paradigm — a sociotechnical revolution that influences governance, society, and even human identity [9]. Fareri et al. (2020) considered that the exact impact of Industry 4.0 on the labor market should be properly studied and used text mining technology to evaluate it. The researchers focused on estimating the influence of Industry 4.0 on the case of the Italian plant of the multinational company Whirlpool. Using the system based on the technologies mapping tools, the authors examined three main categories: job profiles, frequency of 4.0 technologies, and transversality of skills 4.0 [9].

The researchers validated their text-mining results through the information on the O*NET (The Occupational Information Network), receiving 21 out of 23 matches (91,3%) [9]. Such a high percentage of matching data means that text mining technology allows obtaining information about existing 4.0 skill gaps in Whirlpool. In addition, the text mining tool could be used to automatize the revision and updates of job profiles. The HR managers could use text mining to reveal missing skills and update the company’s job descriptions on time, which is quite useful considering the pace of technological development.

The true value of text mining technology lies within its cost-efficiency — with the help of modern digital technologies, a small team or even a single specialist can process large amounts of employment market data. For example, Wowczko (2015) single-handedly conducted a skill and vacancy analysis of the Irish IT sphere. Using the publicly available data from the IrishJobs website, the researcher examined 4846 IT vacancies with RapidMiner software [2]. Text mining technology allowed to examine the job advertisements based on job titles and descriptions. Initially, 30 top terms from job titles and descriptions were extracted, as shown in Figure 6 [2].

Through the creation of various groups based on information from job descriptions, the researchers managed to visualize specific job requirements for particular occupations, as shown in Figure 7 [2].

Overall, this research proved that the text mining method could support skills and vacancy research of the employment market by providing an alternative to resource-expensive methods, such as employer surveying [2]. In addition, dynamic identification of vacancies and popular skills helps make job advertisements more accurate. Therefore, text mining technology allows smaller employers to conduct their market analysis of a particular sphere and improve recruitment without spending unnecessary large funding.

Another case that proves the importance of text mining in employment market analysis can be found in the work by Pejić-Bach et al. (2020) [10]. These researchers utilized data extraction and machine learning analysis methods to examine 1460 job advertisements from LinkedIn in relation to Industry 4.0 and Smart Factory trends. Text mining was conducted via WordStat Provalis software in order to determine the most prevalent topics connected with Industry 4.0 and Smart Factory [10]. After that, the researcher identified the association between extracted phrases and terms, as shown in Figure 8, and divided them into clusters [10].

Furthermore, the clusters were mapped for a more convenient representation of the text mining analysis. The size of the circles reflected the frequency of the phrase, and the distance between them represented the strength of the relationship between the phrases included in the cluster [10]. Overall, text miming allowed us to identify and visualize key topics, vacancies, and competencies of the Industry 4.0 professional paradigm.

Another example of the successful use of text mining for revealing the new trends in the employment market comes from Finland. Bäck et al. (2021) studied more than 480 000 job advertisements to trace the emergence of AI-related occupations during the 2013-2020 period [11]. The researchers conducted an analysis of jobs based on three tiers of AI-relatedness:

• Tier 1: main generic terms referring to AI (i.e., artificial intelligence, machine learning);
• Tier 2: core technologies associated with AI (i.e., NLTK, Decision tree);
• Tier 3: technologies that support or enhance AI solutions but not direct AI core technologies (i.e., Cloud, Database, Matlab) [11].

Each of these tiers was represented in two languages: Finnish and English, or in English only in lack of a Finnish equivalent. In the end, the researchers obtained a comprehensive breakdown of AI-related jobs by tiers of relatedness and respective industries. Once again, text mining allowed us to draw a clear picture of the particular segment of the labor market. For instance, separation by tier and total share of AI-related jobs was represented by a graph below [11].

Finally, Pejić-Bach et al. (2019) summarized the application of text mining in general research and the financial sector in particular [12]. The authors outlined three main directions for the future application of text mining technology:

• Studying of “grey” literature sources, such as case studies and corporate reports;
• Supporting of more effective decision-making on tactical, operational, and strategic levels;
• Investigating the organizational characteristics of the companies that implemented text mining in their business processes [12].

Overall, text mining proved to be a handy analytical tool for studying the evolution of the employment market in various industries. Text mining yields results regardless of national or cultural specifics and does not require significant financial or human resources to be effective, making it a lucrative technology for smaller companies.

Text mining for analyzing job satisfaction

Another useful application of text mining technology is the analysis of user-generated content (UGC), voluntarily created on social media or other websites. Jung and Suh (2019) utilized text mining to evaluate the employees’ opinions on various factors related to job satisfaction [13]. Using the tens of thousands of IT job reviews left on Jobplanet.co.kr, the largest online company review resource in South Korea, the researchers obtained the following first-hand information:

• Key factors of job satisfaction;
• The sentiment and relative importance of these factors;
• The most important factors for overall satisfaction;
• The variance of factors across job ratings [13].

The results were portrayed in a graphical format, easy for general comprehension, as shown in Figure 12 [13]. In addition, the analysis grasped dynamics in employees’ stances over time.

Overall, the nature of obtained textual data poses a significant value for employers and their HR managers. Text mining technology proved the capability of revealing the general considerations on job satisfaction factors, so companies can utilize it for improving the working experience of their employees.

Analysis

Strengths

According to Phillpott (2021), online recruitment has become a preferred means of talent scouting [14]. Therefore, both companies and potential employees have an incentive to use digital technologies such as text mining and job finder systems extensively. According to the reviewed literature, these solutions provide the following advantages:

• Variety of tools. The candidate can rely on multiple job finder systems to analyze the employment market and find a preferred job. For instance, CareerSidekick (n.d.) provided a list of 10 job search engines that can be helpful for the employee [15]. As for text mining, plenty of methods and applications have been presented in the previous sections of this review;
• Cost-efficiency. Text mining-based employment market analyses were often conducted by one or two researchers with limited resources. In the case of job finder systems, the users can apply filters to remove unwanted or mismatched jobs [16]. As a result, job finder systems minimize search time and maximize its efficiency;
• Accessibility. Text mining mostly utilizes data, freely available on job finder websites. Likewise, job finder systems provide this advantage since the candidates can easily obtain the information on posted jobs and apply directly [17]. The companies can also reach potential employees or analyze the descriptions posted by the competitors.
• User-friendliness. Text mining-based research reviewed in this report has a convenient visual representation of results, allowing non-academic readers to understand them. Likewise, job finders present vacancy and candidate data in an easy-to-read format, as can be seen in [18].

Limitations

Despite the multiple strengths and empiric results obtained with text mining and job finder systems, these technologies also have certain limitations. These issues should be considered by the researchers, employers, and potential candidates.

• Technical issues. According to Joseph (2019), some candidates may be reluctant to use job finders due to disbelief and insecurity of online applications [19]. As a result, their data would be unavailable for the employment market analysis. From the text mining perspective, a problem may arise with the length of processed strings [20]. The longer strings would yield fewer data points that meet the parameters, while the shorter ones would allow for processing larger volumes of textual queries at the cost of accuracy [20];
• Lack of generalization. This issue is especially relevant for text mining analysis of local employment markets. For instance, Jung and Suh (2019) pointed out that their research was based on a single job finder portal, one country, and a particular industry [13]. Likewise, multiple pieces of research from the reviewed selection were focused on studying global trends on a national scope. Therefore, their results should not be extrapolated to the worldwide employment market or even the different industries of the national markets.

Conclusion

During this research report, I tried to study the use of text mining technology and job finder systems in data market analysis. Specifically, I tried to reveal the practical applications of text mining and job finders, their strengths, and the limitations of their use.

I found that text mining and job finders proved their usefulness for multiple tasks. First of all, these technologies enabled the companies to improve the job recommendation and matching results. Secondly, the employers benefited from text mining analysis of job descriptions since it allowed them to identify trending skills in various professional spheres. Lastly, text mining can be used for understanding the employees’ needs and job satisfaction.

Text mining and job finder systems bring several significant advantages, such as a variety of research tools, cost-efficiency, accessibility, and user-friendliness. However, these technologies also have specific technical limitations and tend to provide localized results.

Although this research report has certain limitations of scope, its results provided a general understanding of how text mining and job finder systems help analyze the employment market. It is possible to expand the scope and alleviate the limitations by continuing research.

Reference List

International Labour Office (2015). Gathering and analyzing labor market data. Web.

Wowczko, I. A. (2015) ‘Skills and vacancy analysis with data mining techniques’, Informatics, 2(4), pp. 31–49.

Local Workforce System Guide (n.d.). Labor market information analysis. Web.

Neill, A. (2018). How to use filters on a job board, part 1. Web.

Kino, Y., Kuroki, H., Machida, T., Furuya, N., and Takano, K. (2017) ‘Text analysis for job matching quality improvement’, Procedia Computer Science, 112, pp. 1523–1530.

Shalaby, W., AlAila, B., Korayem, M., Pournajaf, L., AlJadda, K., Quinn, S., and Zadrozny, W. (2017) ‘Help me find a job: A graph-based approach for job recommendation at scale’. in 2017 IEEE International Conference on Big Data, pp. 1544–1553.

Jung, S., Cho, J. H., and Kim, I. W. (2019). ‘Corporations’ and job seekers’ using intention and WOM (word-of-mouth) of NCS-based job matching system’, Advances in Economics and Business, 7(5), pp. 194–201.

Darabi, H., Karim, F. S. M., Harford, S. T., Douzali, E., and Nelson, P (2018) ‘Detecting current job market skills and requirements through text mining’, in 2018 ASEE Annual Conference & Exposition, pp. 1–16.

Fareri, S., Fantoni, G., Chiarello, F., Coli, E., and Binda, A. (2020) ‘Estimating Industry 4.0 impact on job profiles and skills using text mining’, Computers in Industry, 118, pp. 1–19.

Pejić-Bach, M., Bertoncel, T., Meško, M., and Krstić, Ž. (2020) ‘Text mining of industry 4.0 job advertisements’, International Journal of Information Management, 50, pp. 416–431.

Bäck, A., Hajikhani, A., and Suominen, A. (2021) ‘Text mining on job advertisement data: Systematic process for detecting artificial intelligence-related jobs’, in CEUR Workshop Proceedings, 2871, pp. 111–124.

Pejić-Bach, M., Krstić, Ž., Seljan, S., and Turulja, L. (2019) ‘Text mining for big data analysis in the financial sector: A literature review’, Sustainability, 11(5), 1277, pp. 1–27.

Jung, Y. and Suh, Y. (2019) ‘Mining the voice of employees: A text mining approach to identifying and analyzing job satisfaction factors from online employee reviews’, Decision Support Systems, 123, 113074.

Phillpott, S. (2021). The advantages and disadvantages of online recruitment. Web.

CareerSidekick (n.d.). The top 10 job search engines for 2021. Web.

Haiyan, L. (2019). Jobs filter: Improving the job seeker experience. Web.

Top Echelon (2021). Advantages and disadvantages of online recruitment. Web.

Romanovskiy, A. (2019). Designing a responsive web app for a job search platform: a UX/UI case study. Web.

Joseph, C. (2019). Disadvantages of online recruitment. Web.

Girijala, R. (2018). Text mining strategies and limitations. Web.