Human-in-the-loop control strategy for IoT-based smart thermostats with Deep Reinforcement Learning

Energy and AI, vol. 20, pp. 100490

Abstract

Thermostatic Radiator Valves (TRVs) are a widely used technology for regulating room heating in Europe countries. Smart TRVs can provide significant energy savings, often ranging from 20–40% compared to conventional heating systems. They use sensors and algorithms to learn user behavior and optimize heating schedules accordingly. They can often be easily retrofitted to existing heating systems, making them a practical option for enhancing energy efficiency in present buildings, especially in office buildings due to their highly dynamic operational patterns. This work presents a novel human-in-the-loop control strategy for Internet of Things (IoT)-based TRVs using Deep Reinforcement Learning (DRL). A key focus of this research is enhancing the adaptability of agents’ behavior by implementing a more generic and flexible Markov Decision Process (MDP) to promote policy generalization across diverse scenarios. The study explores the challenges of transferring control behaviors from simulation environments to real-world settings, examining the performance across different thermal zones and evaluating the integration flexibility of the control strategy within building systems. Real-world occupant behavior is incorporated, including dynamic comfort preferences and occupancy predictions, to better align thermostat operation with user preferences. Furthermore, this paper discusses the practical challenges encountered during implementation, including battery consumption of IoT devices, integration of occupancy detection and prediction systems, and maintenance requirements. By addressing these issues, the proposed control strategy seeks to improve the scalability and feasibility of IoT-based TRVs, thereby providing a viable solution for their widespread deployment in buildings. • Human-in-the-loop control strategy for IoT-based smart TRVs. • MDP formulation to improve scalability and adaptability of the control strategy. • Boosting learning efficiency and agent generalizability with a novel training strategy. • Real-life implementation in a building environment using an agent-based IoT framework. • Evaluating control effectiveness using a state-of-the-art DQN agent.

Authors 4

1. Payam Fatehi Karjou corresponding Aachen Institute for Energy Efficient Buildings and Indoor Climate

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Institute for Energy Efficient Buildings and Indoor Climate, Aachen, Germany

2. Fabian Stupperich Aachen Institute for Energy Efficient Buildings and Indoor Climate

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Institute for Energy Efficient Buildings and Indoor Climate, Aachen, Germany

3. Phillip Stoffel Aachen Institute for Energy Efficient Buildings and Indoor Climate

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Institute for Energy Efficient Buildings and Indoor Climate, Aachen, Germany

4. Dirk Müller Aachen Institute for Energy Efficient Buildings and Indoor Climate

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Institute for Energy Efficient Buildings and Indoor Climate, Aachen, Germany