Academic information retrieval using citation clusters: In-depth evaluation based on systematic reviews

arXiv (Cornell University)

Abstract

The field of scientometrics has shown the power of citation-based clusters for literature analysis, yet this technique has barely been used for information retrieval tasks. This work evaluates the performance of citation based-clusters for information retrieval tasks. We simulated a search process using these clusters with a tree hierarchy of clusters and a cluster selection algorithm. We evaluated the task of finding the relevant documents for 25 systematic reviews. Our evaluation considered several trade-offs between recall and precision for the cluster selection, and we also replicated the Boolean queries self-reported by the systematic review to serve as a reference. We found that citation-based clusters search performance is highly variable and unpredictable, that it works best for users that prefer recall over precision at a ratio between 2 and 8, and that when used along with query-based search they complement each other, including finding new relevant documents.

Authors 4

1. Juan Pablo Bascur Aachen Centre for Science and Technology Studies Leiden Institute for Advanced Computer Science

   Leiden University

   Affiliation as printed

   Centre for Science and Technology Studies , Leiden University , The Netherlands

   Leiden Institute for Advanced Computer Science , Leiden University , The Netherlands

2. Suzan Verberne
3. Nees Jan van Eck
4. Ludo Waltman