Expand, Highlight, Generate: RL-driven Document Generation for Passage Reranking

Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 10087–10099

Abstract

Generating synthetic training data based on large language models (LLMs) for ranking models has gained attention recently.Prior studies use LLMs to build pseudo querydocument pairs by generating synthetic queries from documents in a corpus.In this paper, we propose a new perspective of data augmentation: generating synthetic documents from queries.To achieve this, we propose DocGen, that consists of a three-step pipeline that utilizes the few-shot capabilities of LLMs.Doc-Gen pipeline performs synthetic document generation by (i) expanding, (ii) highlighting the original query, and then (iii) generating a synthetic document that is likely to be relevant to the query.To further improve the relevance between generated synthetic documents and their corresponding queries, we propose DocGen-RL, which regards the estimated relevance of the document as a reward and leverages reinforcement learning (RL) to optimize Doc-Gen pipeline.Extensive experiments demonstrate that DocGen and DocGen-RL significantly outperform existing state-of-the-art data augmentation methods, such as InPars, indicating that our new perspective of generating documents leverages the capacity of LLMs in generating synthetic data more effectively.We release the code, generated data, and model checkpoints to foster research in this area 1 .

Authors 5

1. Arian Askari Aachen

   Leiden University

   Affiliation as printed

   Leiden University {a

   Leiden University

2. Mohammad Aliannejadi

   University of Amsterdam

   Affiliation as printed

   University of Amsterdam {m

   University of Amsterdam

3. Chuan Meng

   University of Amsterdam

   Affiliation as printed

   University of Amsterdam {m

   University of Amsterdam

4. Evangelos Kanoulas

   University of Amsterdam

   Affiliation as printed

   University of Amsterdam {m

   University of Amsterdam

5. Suzan Verberne Aachen

   Leiden University

   Affiliation as printed

   Leiden University {a

   Leiden University