Example ADR Review Prompt and corresponding LLM-based ADR Reviews

Zenodo (CERN European Organization for Nuclear Research)

Abstract

This dataset contains a review prompt for an exemplary architectural decision record (ADR) and the corresponding reviews/evaluations done by different LLMs. This dataset accompanies the short paper "AI Assistance for Architectural Decision Making: Domain-Driven Context and Prompt Engineering" submitted to the IDEA-ARCH workshop @ ECSA 2026. The *.txt file contains the specific review prompt including the specific ADR to review. The *.yaml files contain the structured reviews based on this prompt made by different Large Language Models. Their filenames indicate the model name, version, and if they were run locally, including the number of parameters.

Authors 2

1. Olaf Zimmermann

   University of St.Gallen

   Affiliation as printed

   University of St.Gallen

2. Ronny Seiger Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University