Predictive Coding Networks and Inference Learning: Tutorial and Survey

ACM Computing Surveys, vol. 58, pp. 1–47

Abstract

Recent years have witnessed a growing call for renewed emphasis on neuroscience-inspired approaches in artificial intelligence research, under the banner of NeuroAI . A prime example of this is predictive coding networks (PCNs), based on the neuroscientific framework of predictive coding. This framework views the brain as a hierarchical Bayesian inference model that minimizes prediction errors through feedback connections. Unlike traditional neural networks trained with backpropagation (BP), PCNs utilize inference learning (IL), a more biologically plausible algorithm that explains patterns of neural activity that BP cannot. Historically, IL has been more computationally intensive, but recent advancements have demonstrated that it can achieve higher efficiency than BP with sufficient parallelization. Furthermore, PCNs can be mathematically considered a superset of traditional feedforward neural networks (FNNs), significantly extending the range of trainable architectures. As inherently probabilistic (graphical) latent variable models, PCNs provide a versatile framework for both supervised learning and unsupervised (generative) modeling that goes beyond traditional artificial neural networks. This work provides a comprehensive review and detailed formal specification of PCNs, particularly situating them within the context of modern ML methods. This positions PC as a promising framework for future ML innovations.

Authors 3

1. Björn van Zwol Aachen

   Leiden University · Utrecht University

   Affiliation as printed

   Department of Information and Computing Sciences, Utrecht University

   Leiden University

2. Ro Jefferson

   Utrecht University

   Affiliation as printed

   Institute for Theoretical Physics, and Department of Information and Computing Sciences, Utrecht University

3. Egon L. van den Broek

   Utrecht University

   Affiliation as printed

   Department of Information and Computing Sciences, Utrecht University