Multi-actor policy dynamics in research evaluation: Experts, databases, and academics

Abstract

This mixed-methods study examines the complex interplay of actors and factors shaping research assessment policies in Lithuania in the period 1996–2008, a period of transition towards Western practices. Analysing policy documents, bibliometric data, interviews, and grey literature, we focus on the interactions between policymakers, international experts, providers of publication data, and researchers. Our findings reveal the significant influence of international networks on Lithuanian policymakers, leading to the adoption of quantitative assessment and reliance on Web of Science data. However, this narrow focus, coupled with existing power dynamics and top-down decision-making, led to unintended consequences and conflicts among disciplines, culminating in a Constitutional Court challenge. This research also highlights the substantial role of journal publishers and data providers in influencing policy shifts. We underscore the need for international organizations to adopt open data sources that encompass a wider array of scholarly outputs. Additionally, national policymakers must consider disciplinary differences and research capacity when designing assessment policies. The Lithuanian experience serves as a cautionary tale, emphasizing the importance of context-specific, inclusive, and flexible research assessment systems, particularly relevant for ongoing reforms such as the Coalition for Advancing Research Assessment (CoARA).

Authors 3

1. Eleonora Dagienė Aachen Centre for Science and Technology Studies

   Leiden University · University of Twente · Mykolas Romeris University

   Affiliation as printed

   Centre for Science and Technology Studies (CWTS), Leiden University, The Netherlands

   Centre for Science and Technology Studies (CWTS), Leiden University, The Netherlands;

   Institute of Communication, Mykolas Romeris University, Lithuania

   University of Twente, The Netherlands

2. Ludo Waltman Aachen Centre for Science and Technology Studies

   Leiden University · University of Twente · Mykolas Romeris University

   Affiliation as printed

   Centre for Science and Technology Studies (CWTS), Leiden University, The Netherlands

   Centre for Science and Technology Studies (CWTS), Leiden University, The Netherlands;

   Institute of Communication, Mykolas Romeris University, Lithuania

   University of Twente, The Netherlands

3. Guus Dix Aachen Centre for Science and Technology Studies

   Leiden University · University of Twente · Mykolas Romeris University

   Affiliation as printed

   Centre for Science and Technology Studies (CWTS), Leiden University, The Netherlands

   Centre for Science and Technology Studies (CWTS), Leiden University, The Netherlands;

   Institute of Communication, Mykolas Romeris University, Lithuania

   University of Twente, The Netherlands