Example ADR Review Prompt and corresponding LLM-based ADR Reviews

Zenodo (CERN European Organization for Nuclear Research)

Abstract

This dataset contains a review prompt template and an instance for an exemplary architectural decision record (ADR) and the corresponding reviews/evaluations done by different LLMs. The LLMs were asked to review a given ADR based on given criteria. This dataset accompanies the short paper "AI Assistance for Architectural Decision Making: Domain-Driven Context and Prompt Engineering" accepted at the IDEA-ARCH workshop @ ECSA 2026. The *.md file contains the prompt template. The *.txt file contains the specific review prompt instance including the specific ADR to review as input for the LLMs. The *.yaml files contain the structured reviews based on this prompt made by different Large Language Models. Their filenames indicate the model name, version, and if they were run locally, including the number of parameters.

Authors 2

1. Olaf Zimmermann

   University of St.Gallen

   Affiliation as printed

   University of St.Gallen

2. Ronny Seiger Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University