A comprehensive quality assessment framework for scientific events

Scientometrics, vol. 126, pp. 641–682

Abstract

Systematic assessment of scientific events has become increasingly important for research communities. A range of metrics (e.g., citations, h-index) have been developed by different research communities to make such assessments effectual. However, most of the metrics for assessing the quality of less formal publication venues and events have not yet deeply investigated. It is also rather challenging to develop respective metrics because each research community has its own formal and informal rules of communication and quality standards. In this article, we develop a comprehensive framework of assessment metrics for evaluating scientific events and involved stakeholders. The resulting quality metrics are determined with respect to three general categories-events, persons, and bibliometrics. Our assessment methodology is empirically applied to several series of computer science events, such as conferences and workshops, using publicly available data for determining quality metrics. We show that the metrics' values coincide with the intuitive agreement of the community on its "top conferences". Our results demonstrate that highly-ranked events share similar profiles, including the provision of outstanding reviews, visiting diverse locations, having reputed people involved, and renowned sponsors.

Authors 7

1. Sahar Vahdati corresponding

   University of Oxford

   Affiliation as printed

   Department of Computer Science, University of Oxford, Oxford, UK

   Institute for Applied Informatics (InfAI), Dresden, Germany

2. Said Fathalla corresponding

   University of Bonn · Alexandria University

   Affiliation as printed

   Faculty of Science, University of Alexandria, Alexandria, Egypt

   University of Bonn, Bonn, Germany

3. Christoph Lange corresponding Aachen

   Fraunhofer Institute for Applied Information Technology · RWTH Aachen University

   Affiliation as printed

   Fraunhofer FIT, Sankt Augustin, Germany

   RWTH Aachen University, Aachen, Germany

4. Andreas Behrend

   TH Köln - University of Applied Sciences

   Affiliation as printed

   Institute for Telecommunications (INT), TH Köln, Germany

   Institute for Telecommunications (INT), TH, Köln, Germany

5. Aysegul Say

   University of Bonn

   Affiliation as printed

   University of Bonn, Bonn, Germany

6. Zeynep Say

   University of Bonn

   Affiliation as printed

   University of Bonn, Bonn, Germany

7. Sören Auer

   Leibniz University Hannover · Technische Informationsbibliothek (TIB) · L3S Research Center

   Affiliation as printed

   L3S Research Center, Leibniz University of Hannover, Hannover, Germany

   TIB Leibniz Information Centre for Science and Technology, Hannover, Germany