A new model of decision processing in instrumental learning tasks

bioRxiv (Cold Spring Harbor Laboratory)

Abstract

Abstract Learning and decision making are interactive processes, yet cognitive modelling of error-driven learning and decision making have largely evolved separately. Recently, evidence accumulation models (EAMs) of decision making and reinforcement learning (RL) models of error-driven learning have been combined into joint RL-EAMs that can in principle address these interactions. However, we show that the most commonly used combination, based on the diffusion decision model (DDM) for binary choice, consistently fails to capture crucial aspects of response times observed during reinforcement learning. We propose a new RL-EAM based on an advantage racing diffusion (ARD) framework for choices among two or more options that not only addresses this problem but captures stimulus difficulty, speed-accuracy trade-off, and stimulus-response-mapping reversal effects. The RL-ARD avoids fundamental limitations imposed by the DDM on addressing effects of absolute values of choices, as well as extensions beyond binary choice, and provides a computationally tractable basis for wider applications.

Authors 5

1. Steven Miletić corresponding

   University of Amsterdam

   Affiliation as printed

   University of Amsterdam, Department of Psychology, Nieuwe Achtergracht 129B, Amsterdam, The Netherlands

   University of Amsterdm, Deprtment of Psychology, Nieuwe Achtergrcht 129B, Amsterdm, The Netherlnds

2. Russell James Boag

   University of Amsterdam

   Affiliation as printed

   University of Amsterdam, Department of Psychology, Nieuwe Achtergracht 129B, Amsterdam, The Netherlands

   University of Amsterdm, Deprtment of Psychology, Nieuwe Achtergrcht 129B, Amsterdm, The Netherlnds

3. Anne C. Trutti Aachen Department of Psychology

   Leiden University · University of Amsterdam

   Affiliation as printed

   Leiden University, Department of Psychology, Wassenaarseweg 52, Leiden, The Netherlands

   University of Amsterdam, Department of Psychology, Nieuwe Achtergracht 129B, Amsterdam, The Netherlands

   University of Amsterdm, Deprtment of Psychology, Nieuwe Achtergrcht 129B, Amsterdm, The Netherlnds

4. Birte U. Forstmann

   University of Amsterdam

   Affiliation as printed

   University of Amsterdam, Department of Psychology, Nieuwe Achtergracht 129B, Amsterdam, The Netherlands

   University of Amsterdm, Deprtment of Psychology, Nieuwe Achtergrcht 129B, Amsterdm, The Netherlnds

5. Andrew Heathcote

   University of Newcastle Australia · University of Amsterdam

   Affiliation as printed

   University of Amsterdam, Department of Psychology, Nieuwe Achtergracht 129B, Amsterdam, The Netherlands

   University of Newcastle, School of Psychology, Newcastle, Australia

   University of Amsterdm, Deprtment of Psychology, Nieuwe Achtergrcht 129B, Amsterdm, The Netherlnds

   University of Newastle, Shool of Psyhology, Newastle, Australia