FAIR Data Assessment Using LLMs: The Fair-Way

ACM International Conference on Information and Knowledge Management (CIKM), pp. 5228–5232

Abstract

As part of modern research practices, the FAIR data principles have become essential for data discoverability, usability, and sharing.Existing implementations for automatically assessing FAIR adherence (FAIRness) often suffer from limited usability, inconsistent accuracy, and difficult-to-interpret results, as they require explicit rules to cover for specific FAIR assessment frameworks, which are not easy to generalize.This paper introduces Fair-Way, an open source tool that leverages Large Language Models (LLMs) to automate FAIRness assessment.Fair-Way applies a divide-and-conquer approach to decompose the assessment process into fine-grained tasks, as well as to split the metadata into manageable chunks.Evaluation demonstrates that Fair-Way achieves performance comparable to existing tools, while outperforming them in several key metrics.Moreover, Fair-Way generalizes across FAIR assessment indicators without requiring explicitly programmed logic and supports both structured and unstructured metadata in diverse formats.Finally, it enables user-defined, domain-specific tests, which are typically not supported by other systems.Overall, Fair-Way represents a scalable and flexible solution to accelerate FAIR data practices across research domains.

Authors 7

1. Sulayman K. Sowe

   Fraunhofer Institute for Applied Information Technology

   Affiliation as printed

   Fraunhofer FIT, Aachen, Germany

2. Soo-Yon Kim Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Aachen, Germany

3. Sayed Hoseini

   Hochschule Niederrhein

   Affiliation as printed

   Hochschule Niederrhein, Krefeld, Germany

4. Fidan Limani

   ZBW – Leibniz-Informationszentrum Wirtschaft

   Affiliation as printed

   ZBW - Leibniz Information Centre for Economics, Kiel, Germany

5. Zeyd Boukhers

   Fraunhofer Institute for Applied Information Technology

   Affiliation as printed

   Fraunhofer FIT, Sankt Augustin, Germany

6. Christoph Lange

   Fraunhofer Institute for Applied Information Technology

   Affiliation as printed

   Fraunhofer FIT, Sankt Augustin, Germany

7. Stefan Decker Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Aachen, Germany