WorkRB: A Community-Driven Evaluation Framework for AI in the Work Domain

ACM Conference on Recommender Systems (RecSys), pp. 933–938

Abstract

Today’s evolving labor markets rely increasingly on recommender systems for hiring, talent management, and workforce analytics, with natural language processing (NLP) capabilities at the core. Yet, research in this area remains highly fragmented. Studies employ divergent ontologies (ESCO, O*NET, national taxonomies), heterogeneous task formulations, and diverse model families, making cross-study comparison and reproducibility exceedingly difficult. General-purpose benchmarks lack coverage of work-specific tasks, and the inherent sensitivity of employment data further limits open evaluation. We present WorkRB (Work Research Benchmark), the first open-source, community-driven benchmark tailored to work-domain AI. WorkRB organizes 13 diverse tasks from 7 task groups as unified recommendation and NLP tasks, including job / skill recommendation, candidate recommendation, similar item recommendation, and skill extraction and normalization. WorkRB enables both monolingual and cross-lingual evaluation settings through dynamic loading of multilingual ontologies. Developed within a multi-stakeholder ecosystem of academia, industry, and public institutions, WorkRB has a modular design for seamless contributions and enables integration of proprietary tasks without disclosing sensitive data. WorkRB is available under the Apache 2.0 license at https://github.com/techwolf-ai/WorkRB.

Authors 19

1. Matthias De Lange
   Affiliation as printed

   Techwolf, Ghent, Belgium

2. Warre Veys
   Affiliation as printed

   Techwolf, Ghent, Belgium

3. Federico Retyk
   Affiliation as printed

   Avature, Barcelona, Spain

4. Daniel Deniz
   Affiliation as printed

   Avature, Barcelona, Spain

5. Warren Jouanneau
   Affiliation as printed

   Malt, Paris, France

6. Mike Zhang

   University of Copenhagen

   Affiliation as printed

   University of Copenhagen, Copenhagen, Denmark

7. Aleksander Bielinski

   Edinburgh Napier University

   Affiliation as printed

   Edinburgh Napier University, Edinburgh, United Kingdom

8. Emma Jouffroy
   Affiliation as printed

   Malt, Paris, France

9. Nicole Clobes
   Affiliation as printed

   WAPES, Brussels, Belgium

10. Nina Baranowska Aachen

    Leiden University

    Affiliation as printed

    Leiden University, Leiden, Netherlands

11. David Graus

    University of Amsterdam

    Affiliation as printed

    University of Amsterdam, Diemen, Netherlands

12. Marc Palyart
    Affiliation as printed

    Malt, Paris, France

13. Rabih Zbib
    Affiliation as printed

    Avature, Barcelona, Spain

14. Dimitra Gkatzia

    Edinburgh Napier University

    Affiliation as printed

    Edinburgh Napier University, Edinburgh, United Kingdom

15. Thomas Demeester

    Ghent University

    Affiliation as printed

    Ghent University - imec, Ghent, Belgium

16. Tijl De Bie

    Ghent University

    Affiliation as printed

    AIDA-IDLab, Ghent University, Ghent, Belgium

17. Toine Bogers

    IT University of Copenhagen

    Affiliation as printed

    IT University of Copenhagen, Copenhagen, Denmark

18. Jens-Joris Decorte
    Affiliation as printed

    Techwolf, Ghent, Belgium

19. Jeroen Van Hautte
    Affiliation as printed

    Techwolf, Ghent, Belgium