Evaluating ReLink for Traceability Link Recovery in Practice

IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 80–90

Abstract

Traceability is important in the software development life cycle for managing the connections among various software artifacts, particularly pull requests (PRs) and issues. Developers often neglect to link these manually, reducing traceability. Existing algorithms to recover these links have limited application in closed-source projects. In this study, we share the experience of using ReLink, a predictive PR-issue linking tool with visualization capabilities, in a closed-source project environment. ReLink stands out due to its availability as a fully functional web application and its semi-automated nature, which enhances usability. ReLink determines missing links between PRs and issues based on a confidence score normalized between 0 and 100, calculated using text similarity and heuristic rules. The tool's effectiveness was evaluated in both an open-source project and an industry-based case study. In the open-source project, ReLink achieved a top-5 accuracy of 0.80 and a precision of 0.77. In the industrial case study, ReLink's effectiveness was validated by practitioners selecting the correct link from five issue suggestions, resulting in a top-5 accuracy of 0.86 and a mean reciprocal rank (MRR) of 0.84. Through this experience, we offer insights into both the benefits and challenges of implementing traceability link recovery in the industry and provide recommendations for practitioners seeking to bridge traceability gaps efficiently.

Authors 7

1. Ayberk Yaşa
   Affiliation as printed

   Stash,Ankara,Turkey

2. Cemhan Kaan Özaltan Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Aachen,Germany

3. Görkem Ayten

   Bilkent University

   Affiliation as printed

   Bilkent University,Ankara,Turkey

4. Fatih Kaplama

   Bilkent University

   Affiliation as printed

   Bilkent University,Ankara,Turkey

5. Ömercan Devran
   Affiliation as printed

   Beko,İstanbul,Turkey

6. Baykal Mehmet Uçar
   Affiliation as printed

   Beko,İstanbul,Turkey

7. Eray Tüzün

   Bilkent University

   Affiliation as printed

   Bilkent University,Ankara,Turkey