Challenges in applying large language models to requirements engineering tasks

Design Science, vol. 10

Abstract

Abstract Growth in the complexity of advanced systems is mirrored by a growth in the number of engineering requirements and related upstream and downstream tasks. These requirements are typically expressed in natural language and require human expertise to manage. Natural language processing (NLP) technology has long been seen as promising to increase requirements engineering (RE) productivity but has yet to demonstrate substantive benefits. The recent addition of large language models (LLMs) to the NLP toolbox is now generating renewed enthusiasm in the hope that it will overcome past shortcomings. This article scrutinizes this claim by reviewing the application of LLMs for engineering requirements tasks. We survey the success of applying LLMs and the scale to which they have been used. We also identify groups of challenges shared across different engineering requirement tasks. These challenges show how this technology has been applied to RE tasks that need reassessment. We finalize by drawing a parallel to other engineering fields with similar challenges and how they have been overcome in the past – and suggest these as future directions to be investigated.

Authors 6

1. Johannes J. Norheim corresponding

   Massachusetts Institute of Technology

   Affiliation as printed

   Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA, USA

2. Eric Rebentisch

   Massachusetts Institute of Technology

   Affiliation as printed

   Sociotechnical Systems Research Center, Massachusetts Institute of Technology, Cambridge, MA, USA

3. D Q Xiao Aachen Laboratory for Machine Tools and Production Engineering (WZL)

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools and Production Engineering (WZL), RWTH Aachen University, Aachen, Germany

4. Lorenz Draeger Aachen Laboratory for Machine Tools and Production Engineering (WZL)

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools and Production Engineering (WZL), RWTH Aachen University, Aachen, Germany

5. Alain Kerbrat

   Airbus (France)

   Affiliation as printed

   Airbus, Toulouse, France

6. Olivier L. De Weck

   Massachusetts Institute of Technology

   Affiliation as printed

   Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA, USA