Combining Retrieval and Classification: Balancing Efficiency and Accuracy in Duplicate Bug Report Detection

arXiv (Cornell University)

Abstract

In the realm of Duplicate Bug Report Detection (DBRD), conventional methods primarily focus on statically analyzing bug databases, often disregarding the running time of the model. In this context, complex models, despite their high accuracy potential, can be time-consuming, while more efficient models may compromise on accuracy. To address this issue, we propose a transformer-based system designed to strike a balance between time efficiency and accuracy performance. The existing methods primarily address it as either a retrieval or classification task. However, our hybrid approach leverages the strengths of both models. By utilizing the retrieval model, we can perform initial sorting to reduce the candidate set, while the classification model allows for more precise and accurate classification. In our assessment of commonly used models for retrieval and classification tasks, sentence BERT and RoBERTa outperform other baseline models in retrieval and classification, respectively. To provide a comprehensive evaluation of performance and efficiency, we conduct rigorous experimentation on five public datasets. The results reveal that our system maintains accuracy comparable to a classification model, significantly outperforming it in time efficiency and only slightly behind a retrieval model in time, thereby achieving an effective trade-off between accuracy and efficiency.

Authors 4

1. Qianru Meng
2. Xiao Zhang

   University of Groningen · Dialyse Centrum Groningen

   Affiliation as printed

   CLCG University of Groningen Groningen , Netherlands

3. Guus J. Ramackers Aachen

   Leiden University · University of Applied Sciences Leiden

   Affiliation as printed

   LIACS Leiden University Leiden , Netherlands

4. Visser Joost