RefSite3D: A Multi-Phase and Multi-Scenario 3D Point Cloud Dataset from the Reference Construction Site Aachen for Target Extraction, Scan-to-BIM Verification and Construction Monitoring

Zenodo (CERN European Organization for Nuclear Research)

Abstract

RefSite3D is a multi-phase and multi-scenario construction progress monitoring dataset created within the TARGET-X project and further processed within the Cluster of Excellence CARE and the CARE Future Technologies Experimental Construction Site. The dataset documents the assembly, disassembly, transport, and reassembly of the modular, multi-material TARGET-X ReStage demonstrator on the Reference Construction Site in Aachen and at the bauma 2025 trade fair. It contains: Planned 3D building models in OBJ format Synthetic point clouds generated from the planning models Real-world point cloud scans from multiple sensing technologies Target-extracted subsets Object verification labels Semantic segmentation labels Metadata for each scene Utility scripts and documentation The dataset is designed to support two sequential use cases: Target Extraction Construction Progress Monitoring via Object Verification The TARGET-X ReStage demonstrator, including the planning models, construction activities, and data acquisition, was developed within TARGET-X and funded by the Smart Networks and Services Joint Undertaking (SNS JU) under the European Union’s Horizon Europe research and innovation programme (Grant Agreement No. 101096614). The transformation of the acquired and simulated data into a structured benchmark dataset, including annotation generation, utility script development, and dataset documentation, was carried out within the Cluster of Excellence CARE. CARE - Climate-Neutral and Resource-Efficient Construction is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy – EXC 3115 – 533767731. 1. TARGET-X ReStage Demonstrator The TARGET-X ReStage demonstrator was developed as part of the research project TARGET-X. The demonstrator represents a modular building system composed of: Precast concrete elements Timber elements Steel elements Additional modular units The assembly process was documented across multiple construction phases. During each phase, high-resolution 3D point cloud scans were captured using terrestrial laser scanning, photogrammetry, and NeRF-based reconstruction. The resulting data reflects realistic construction scenarios and enables the development and benchmarking of algorithms for digital construction and spatial AI. 1.1 Construction Phases Phase 1: Initial Assembly at the Reference Construction Site Aachen In Phase 1, the first building elements were assembled at the Reference Construction Site in Aachen. This phase contains the initial construction state of the demonstrator. Phase 2: bauma 2025 Extension Module In Phase 2, an additional modular unit was fabricated for exhibition at the bauma 2025 trade fair. The module was transported and assembled independently. Phase 3: Reuse Scenario In Phase 3, the bauma 2025 module was deconstructed, transported back to Aachen, and reconstructed as extension to the original demonstrator. This phase demonstrates: Reuse of building components Modular construction principles Circular construction workflows 1.2 Progress Monitoring States Several construction progress monitoring states were recorded. These states represent incremental construction stages. Progress State Description PM1 Concrete elements only PM2 Concrete + timber elements PM3 Concrete + timber + steel elements PM4 Standalone bauma 2025 extension module PM5 Complete demonstrator including bauma 2025 module 2. Benchmark Tasks Benchmark Task 1: Target Extraction The first task is to identify and extract the Object of Interest (OOI) from a full scene point cloud. The Object of Interest may represent: The TARGET-X ReStage demonstrator A machine An assembly of steel or timber components A storage area containing multiple construction components Accordingly, the dataset includes both construction progress monitoring scenes and additional target extraction benchmark scenes. These additional scenes were acquired between the progress monitoring states, when TARGET-X ReStage components were temporarily stored on the Reference Construction Site Aachen. They also include standalone scenes of steel assemblies, timber assemblies, and industrial machinery. Scan Acquisition Strategy All target extraction scenes were acquired using an object-oriented scanning strategy. In this approach, the scanner is moved closely around the predefined region of interest to obtain high point density and geometric completeness. This strategy ensures that all scenes are captured under consistent conditions and are directly comparable. Label Definition The target extraction task uses a three-class semantic labeling scheme: Label ID Class Name Description 0 Environment All points outside the region of interest and its supporting ground, such as surrounding structures, equipment, and vegetation 1 Ground The ground surface directly underlying the region of interest 2 Target All points belonging to the Object of Interest The target class may consist of a single object, multiple related objects, or a spatially distributed storage area. Only the relevant objects are labeled as target, while the floor or terrain beneath them is always labeled as ground. Applications Semantic segmentation Target extraction Object detection Construction object localization Building extraction from complex scenes Extraction of storage areas and component assemblies Benchmark Task 2: Progress Monitoring via Object Verification After extracting the target object, the extracted point cloud is registered with the corresponding planning model. Each planned building component is then compared to the scanned data to determine whether sufficient evidence exists to confirm its presence. Label Definition The object verification task uses a binary labeling scheme: Label ID Class Name Description 0 Object not verified The component could not be confirmed in the scan data 1 Object verified The component was successfully identified in the scan data These labels are stored in JSON files in the annotations/progress_tracking/ directories. Applications Automated construction progress monitoring Scan-to-BIM comparison Scan-vs-BIM verification Object-level installation status assessment Digital twin updating 3. Processing Workflow The dataset supports a two-stage workflow for automated construction progress monitoring. In the first stage, semantic segmentation is used to extract the relevant target structure from the full scene point cloud. In the second stage, the extracted target point cloud is aligned with the corresponding planning model and analyzed at the component level to determine which planned elements are present in the scan data. Full Scene Point Cloud ↓Semantic Target Extraction ↓Target Point Cloud ↓Registration with Planning Model ↓Component-Level Verification ↓Construction Progress Assessment Synthetic point clouds generated from the planning models serve as idealized reference geometries for the component-wise comparison. The additional benchmark scenes included in the dataset are intended exclusively for evaluating the semantic target extraction task. They provide annotated point clouds for diverse object categories, but do not include planning models or object verification labels. This separation allows the dataset to be used both as: a general benchmark for semantic target extraction, and a complete benchmark for Scan-to-BIM-based construction progress monitoring. 4. Planning Models and Synthetic Point Clouds The planning models were originally created in IFC4 format. They were converted into OBJ meshes for each construction phase. For every mesh, synthetic point clouds were generated. These synthetic point clouds provide idealized, noise-free representations of each building element and serve as references for object verification. Advantages Known geometry No sensor noise Complete coverage Direct component correspondence 5. Real Scan Data The dataset includes real-world point clouds generated using multiple sensing technologies. Acquisition Methods Terrestrial laser scanning (RIEGL VZ-400i, Leica RTC360 3D-Laserscanner) Image-based photogrammetry using Pix4D with images captured by an Apple iPhone X NeRF-based reconstruction using Luma AI with video captured by an Apple iPhone X For each scene The dataset contains: Full scene point cloud Building-extracted point cloud Scene metadata 6. Annotation Data The dataset contains annotation files for two benchmark tasks: Semantic target extraction Object-level verification for construction progress monitoring Semantic Segmentation Labels Semantic segmentation labels are stored as plain text files in the annotations/scene_segmentation/ directory. Each file contains one semantic class label per line. Each semantic segmentation label file corresponds to the full scene point cloud stored in the `scans/` directory (e.g., `scene_001_pm1_..._scene_... .ply`). The i-th line in the label file contains the semantic class label of the i-th point in the corresponding full scene point cloud. The labels are therefore defined for the complete scene, including the target structure, the supporting ground surface, and all surrounding objects. The labels are not associated with the building-extracted point cloud. Label Definition Label ID Class Name Description 0 Environment All points outside the region of interest and its supporting ground, such as surrounding structures, equipment, vegetation, and temporary objects 1 Ground The ground surface directly underlying the region of interest 2 Target All points belonging to the Object of Interest Example TargetTargetTargetGroundGroundEnvironmentEnv

Authors 6

1. Jan Luca Fahrendholz Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University

2. Sigrid Brell‐Çokcan Aachen

   Association for Robots in Architecture · RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University

   Association for Robots in Architecture

3. RWTH Aachen University
4. Construction Robotics GmbH
5. Chu Han Wu Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University

6. Marit Zöcklein
   Affiliation as printed

   Construction Robotics GmbH