The principles of adaptation in organisms and machines II: Thermodynamics of the Bayesian brain

by   Hideaki Shimazaki, et al.

This article reviews how organisms learn and recognize the world through the dynamics of neural networks from the perspective of Bayesian inference, and introduces a view on how such dynamics is described by the laws for the entropy of neural activity, a paradigm that we call thermodynamics of the Bayesian brain. The Bayesian brain hypothesis sees the stimulus-evoked activity of neurons as an act of constructing the Bayesian posterior distribution based on the generative model of the external world that an organism possesses. A closer look at the stimulus-evoked activity at early sensory cortices reveals that feedforward connections initially mediate the stimulus-response, which is later modulated by input from recurrent connections. Importantly, not the initial response, but the delayed modulation expresses animals' cognitive states such as awareness and attention regarding the stimulus. Using a simple generative model made of a spiking neural population, we reproduce the stimulus-evoked dynamics with the delayed feedback modulation as the process of the Bayesian inference that integrates the stimulus evidence and a prior knowledge with time-delay. We then introduce a thermodynamic view on this process based on the laws for the entropy of neural activity. This view elucidates that the process of the Bayesian inference works as the recently-proposed information-theoretic engine (neural engine, an analogue of a heat engine in thermodynamics), which allows us to quantify the perceptual capacity expressed in the delayed modulation in terms of entropy.


page 3

page 5

page 8

page 14

page 16

page 17

page 19

page 23


Neural Networks with Recurrent Generative Feedback

Neural networks are vulnerable to input perturbations such as additive n...

Generalization of generative model for neuronal ensemble inference method

Various brain functions that are necessary to maintain life activities m...

Bayesian Brain: Computation with Perception to Recognize 3D Objects

We mimic the cognitive ability of Human perception, based on Bayesian hy...

Improved Neuronal Ensemble Inference with Generative Model and MCMC

Neuronal ensemble inference is a significant problem in the study of bio...

Stochasticity from function - why the Bayesian brain may need no noise

An increasing body of evidence suggests that the trial-to-trial variabil...

The principles of adaptation in organisms and machines I: machine learning, information theory, and thermodynamics

How do organisms recognize their environment by acquiring knowledge abou...

Cortical Microcircuits from a Generative Vision Model

Understanding the information processing roles of cortical circuits is a...

Please sign up or login with your details

Forgot password? Click here to reset