Developing a holistic AI literacy assessment matrix – Bridging generic, domain-specific, and ethical competencies

Computers and Education Open, vol. 6, pp. 100177

Abstract

Motivated by a holistic understanding of AI literacy, this work presents an interdisciplinary effort to make AI literacy measurable in a comprehensive way, considering generic and domain-specific AI literacy as well as AI ethics. While many AI literacy assessment tools have been developed in the last 2-3 years, mostly in the form of self-assessment scales and less frequently as knowledge-based assessments, previous approaches only accounted for one specific area of a comprehensive understanding of AI competence, namely cognitive aspects within generic AI literacy. Considering the demand for AI literacy development for different professional domains and reflecting on the concept of competence in a way that goes beyond mere cognitive aspects of conceptual knowledge, there is an urgent need for assessment methods that capture domain-specific AI literacy on each of the three competence dimensions of cognition, behavior, and attitude. In addition, competencies for AI ethics are becoming more apparent, which further calls for a comprehensive assessment of AI literacy for this very matter. This conceptual paper aims to provide a foundation upon which future AI literacy assessment instruments can be built and provides insights into what a framework for item development might look like that addresses both generic and domain-specific aspects of AI literacy as well as AI ethics literacy, and measures more than just knowledge-related aspects based on a holistic approach.

Authors 7

1. Nils Knoth corresponding

   University of Kassel

   Affiliation as printed

   Institute for Psychology (IfP), University of Kassel, Holländische Straße 36-38, 34127, Kassel, Germany

2. Marie Mirsch Aachen Gender and Diversity in Engineering (GDI)

   RWTH Aachen University

   Affiliation as printed

   Gender and Diversity in Engineering (GDI), RWTH Aachen University, Kackertstraße 9, 52072, Aachen, Germany

3. Matthias Carl Laupichler

   University of Bonn · University Hospital Bonn

   Affiliation as printed

   Institute of Medical Education, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany

4. Marc Pinski

   Technische Universität Darmstadt

   Affiliation as printed

   Information System & Electronic Services, Technische Universität Darmstadt, Hochschulstraße 1, 64289, Darmstadt, Germany

5. Nils Frederik Buchholtz

   Universität Hamburg

   Affiliation as printed

   Faculty of Education, University of Hamburg, Von-Melle-Park 8, 20146, Hamburg

6. Katharina Bata

   TH Köln - University of Applied Sciences

   Affiliation as printed

   Faculty of Process Engineering, Energy and Mechanical Systems, TH Köln, Claudiusstraße 1, 50678, Köln, Germany

7. Ben Schultz Aachen Gender and Diversity in Engineering (GDI)

   RWTH Aachen University

   Affiliation as printed

   Gender and Diversity in Engineering (GDI), RWTH Aachen University, Kackertstraße 9, 52072, Aachen, Germany