Software Quality Indicators: extraction, categorisation and recommendations from canonical sources

Abstract

Research software plays a central role in modern science, and its quality is increasingly recognized as essential for reproducibility, sustainability, and trust. Numerous initiatives have proposed indicators to guide quality assessment, yet these indicators are dispersed across domains and vary in scope, terminology, and practical use. This work presents a curated catalogue of software quality indicators tailored to the needs of research software. Developed during BioHackathon Europe 2024 and refined in collaboration with the ELIXIR Tools Platform and EVERSE project, the catalogue consolidates and structures indicators from a range of authoritative sources.Over 300 indicators were gathered and systematically reviewed for relevance, clarity, and implementation feasibility. Each was classified into thematic categories—such as Documentation, Security, Usability, and Sustainability—and annotated with target applicability, ease of evaluation, and recommended actions. Redundant, overly abstract, or narrowly scoped indicators were excluded or flagged, while additional tags highlighted cross-cutting concerns such as licensing, testing, and community practices.The resulting open dataset, available as a structured spreadsheet, includes detailed metadata and decision criteria to support reuse, adaptation, and extension. The catalogue offers a foundation for context-specific assessment frameworks. Intended users include research software developers and maintainers, evaluators, and developers of quality-focused tools and guidelines

Authors 17

1. Eva Martín del Pico

   Barcelona Supercomputing Center

   Affiliation as printed

   Barcelona Supercomputing Center

2. Fotis E Psomopoulos

   Aristotle University of Thessaloniki

   Affiliation as printed

   Aristotle University of Thessaloniki

3. Laura Portell-Silva
4. Renato Alves

   European Molecular Biology Laboratory · European Molecular Biology Laboratory · European Molecular Biology Laboratory · European Molecular Biology Laboratory · European Molecular Biology Laboratory

   Affiliation as printed

   European Molecular Biology Laboratory (EMBL)

5. Sébastien Moretti

   SIB Swiss Institute of Bioinformatics

   Affiliation as printed

   SIB Swiss Institute of Bioinformatics

6. Shoaib Sufi

   University of Manchester

   Affiliation as printed

   University of Manchester

7. Adel Kamel Eddine Bouhraoua
8. David Steinberg

   University of California, Santa Cruz

   Affiliation as printed

   University of California , Santa Cruz

9. Gavin Farrell

   European Bioinformatics Institute

   Affiliation as printed

   European Bioinformatics Institute

10. Maria Tsontaki

    Vlaams Instituut voor Biotechnologie

    Affiliation as printed

    Vlaams Instituut voor Biotechnologie

11. Rafael Andrade Buono

    Vlaams Instituut voor Biotechnologie

    Affiliation as printed

    Vlaams Instituut voor Biotechnologie

12. Sebastian Beier

    Forschungszentrum Jülich

    Affiliation as printed

    Forschungszentrum Jülich

13. Daniel Garijo

    Universidad Politécnica de Madrid

    Affiliation as printed

    Universidad Politécnica de Madrid

14. José M. Fernández

    Barcelona Supercomputing Center

    Affiliation as printed

    Barcelona Supercomputing Center

15. Mariia Steeghs-Turchina Aachen

    Leiden University Medical Center

    Affiliation as printed

    Leiden University Medical Center (LUMC)

16. Mihail Anton

    ELIXIR

    Affiliation as printed

    ELIXIR Hub , Chalmers University of Technology BioHackathon series: BioHackathon Europe 2024 Barcelona , Spain, 2024

17. Magnus Palmblad Aachen

    Leiden University Medical Center

    Affiliation as printed

    Leiden University Medical Center (LUMC)