Modeling Cognitive-Affective Processes With Appraisal and Reinforcement Learning

IEEE Transactions on Affective Computing, vol. 16, pp. 771–782

Abstract

Computational models can advance affective science by shedding light onto the interplay between cognition and emotion from an information processing point of view. We propose a computational model of emotion that integrates reinforcement learning (RL) and appraisal theory, establishing a formal relationship between reward processing, goal-directed task learning, cognitive appraisal, and emotional experiences. The model achieves this by formalizing four evaluative checks from the component process model (CPM) in terms of temporal difference learning updates: suddenness, goal relevance, goal conduciveness, and power. The formalism is task independent and can be applied to any task that is represented as a Markov decision problem (MDP) and solved using RL. We evaluate the model by predicting a range of human emotions based on a series of vignette studies, highlighting its potential to improve our understanding of the role of reward processing in affective experiences.

Authors 2

1. Joost Broekens Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science, Leiden University, Leiden, CA, The Netherlands

2. Jussi Jokinen

   University of Jyväskylä

   Affiliation as printed

   Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland