Task-Oriented Document-Grounded Dialog Systems by HLTPR@RWTH for DSTC9 and DSTC10

IEEE/ACM Transactions on Audio Speech and Language Processing, vol. 32, pp. 733–741

Abstract

This paper summarizes our contributions to the document-grounded dialog tasks at the 9th and 10th Dialog System Technology Challenges (DSTC9 and DSTC10). In both iterations the task consists of three subtasks: first detect whether the current turn is knowledge seeking, second select a relevant knowledge document, and third generate a response grounded on the selected document. For DSTC9 we proposed different approaches to make the selection task more efficient. The best method, Hierarchical Selection, actually improves the results compared to the original baseline and gives a speedup of 24x. In the DSTC10 iteration of the task, the challenge was to adapt systems trained on written dialogs to perform well on noisy automatic speech recognition transcripts. Therefore, we proposed data augmentation techniques to increase the robustness of the models as well as methods to adapt the style of generated responses to fit well into the proceeding dialog. Additionally, we proposed a noisy channel model that allows for increasing the factuality of the generated responses. In addition to summarizing our previous contributions, in this work, we also report on a few small improvements and reconsider the automatic evaluation metrics for the generation task which have shown a low correlation to human judgments.

Authors 4

1. David Thulke Aachen Machine Learning and Human Language Technology

   RWTH Aachen University

   Affiliation as printed

   Machine Learning and Human Language Technology, RWTH Aachen University, Aachen, Germany

   AppTek GmbH, Aachen, Germany

2. Nico Daheim Aachen Machine Learning and Human Language Technology

   Technische Universität Darmstadt · RWTH Aachen University

   Affiliation as printed

   Machine Learning and Human Language Technology, RWTH Aachen University, Aachen, Germany

   Ubiquitous Knowledge Processing Lab, Technical University, Darmstadt, Germany

3. Christian Dugast
   Affiliation as printed

   AppTek GmbH, Aachen, Germany

4. Hermann Ney Aachen Machine Learning and Human Language Technology

   RWTH Aachen University

   Affiliation as printed

   Machine Learning and Human Language Technology, RWTH Aachen University, Aachen, Germany

   AppTek GmbH, Aachen, Germany