BIAS: Mitigating Diversity Biases of AI in the Labor Market

The EU Horizon project brings together an interdisciplinary consortium of nine partner institutions to develop a deep understanding of the use of AI in the employment sector and to detect & mitigate unfairness in AI-driven recruitment tools

Fiche signalétique

  • Département responsable Technique et informatique
  • Institut Institute for Data Applications and Security (IDAS)
  • Unité de recherche IDAS / Applied Machine Intelligence
  • Organisation d'encouragement Europäische Union
  • Durée (prévue) 01.11.2022 - 31.10.2026
  • Responsable du projet Prof. Dr. Mascha Kurpicz-Briki
  • Direction du projet Prof. Dr. Mascha Kurpicz-Briki
  • Équipe du projet Dr. Alexandre Riemann Puttick
  • Partenaire European Commission
    Norges Teknisk-Naturvitenskapelige Universitet
    Háskóli Íslands
    Globaz, S.A. (LOBA)
    CROWDHELIX LIMITED
    SMART VENICE SRL
    UNIVERSITÄT LEIDEN
    DIGIOTOUCH OU
    FARPLAS OTOMOTIV ANONIM SIRKETI
  • Mots-clés bias, human resources, artificial intelligence, augmented intelligence, natural language processing

Situation

Artificial Intelligence (AI) is increasingly used in the employment sector. A recent Sage study found that 24% of companies use AI for recruitment purposes. This often involves Natural Language Processing (NLP) based AI-models to analyze text created by a job candidate. High profile cases have shown that such systems can reproduce social prejudices and unfairly discriminate against underrepresented minorities. This form of algorithmic bias is exacerbated by the fact that AI decision-making processes usually occur in a black-box, opaque even to the engineers who designed them. This results in systems capable of rendering unjust and unjustified decisions with low accountability, decisions that are often not subject to appeal on behalf of adversely affected human stakeholders. In practice, machine learning (ML) and NLP-based applications typically consist of off-the-shelf large-language models (LLMs) such as BERT, GPT models etc., which are then fine-tuned on a task-specific dataset; for example, an archive of job applications labeled according to whether the corresponding candidate was successful. Both the general language models employed, and the task-specific training data are potential sources of bias.

Approche

We aim to provide a scheme for bias detection and mitigation in AI-based recruitment, with an emphasis on building transparent, trustworthy systems that support human decision-making rather than replacing it (augmented intelligence). Existing methods provide a framework for investigating bias encoded within LLMs. This requires expert knowledge on language-, culture- and domain-specific prejudices in order to compile lists of sensitive words and concepts that can be used to identify encoded bias. These lists will be compiled by partners in our interdisciplinary consortium. Together with partner institutions, we will investigate methods to encourage fairer and more transparent AI-recruitment tools. We seek to build a foundation for tools to detect and mitigate both encoded bias in underlying language models, as well as the bias-sensitive aspects of input data, aiming to ensure that both AI-tools and employers use relevant information for hiring, rather than basing decisions on existing bias and irrelevant sensitive features (e.g., race, gender, sexual orientation...).

Perspectives

Using a transdisciplinary approach, the BIAS project will develop a deep understanding of the use of AI in the HR-sector and the ripple effects such tools have for all stakeholders. Our contribution will focus on laying the research foundation for bias detection and mitigation following EU guidelines for Trustworthy AI (ALTAI) and the principle of augmented intelligence. Beyond this project, our methods will be further developed into new deployment-ready recruitment tools. In addition, the work of other consortium members will culminate in policy proposals and training programs to promote the fair use of AI in Human Resources Management (HRM).

Ce projet contribue aux objectifs de développement durable suivants

  • 3: Accès à la santé
  • 5: Égalité entre les sexes
  • 8: Accès à des emplois décents
  • 10: Réduction des inégalités
  • 16: Justice et paix