Free Energy Principle

Chapter 1 - Worldview

---

Previous page: Chaos - Free Energy Principle - Next page: Markov blanket

---

Back to Book content or directly to Main Page or Worldview

---

.

Connections create energy

Welcome to the Free Energy Principle page (the third building block)

.

In the physical world, everything tends towards total entropy. Surprisingly, life, on the other hand, goes in the opposite direction. The 'Free Energy Principle' can provide an explanation for this.

.

Core idea

.

Your road away from chaos

You read in the entropy part:

• High entropy means a low potential energy (towards chaos)
• Low entropy means a high potential energy (towards life)

How do we deal with this as humans?

Your brain is an energy guzzler. Energy needed for survival. Your brain will constantly try to minimise energy consumption by minimising its entropy, thus maintaining a high potential free energy.

It tries to do this by evaluating - (through a Bayesian method) - the difference between:

• the reactions to the actions of your body
• and the already existing model of the world in your brain.

If (and only if) this evaluation passes a certain threshold, the brain will change its internal model to rebalance its homeostasis.

.

Deep dive

.

Key take-aways from the deep dive

• Your brain will tell your body (how) to act the 'next moment' in the expectation of minimising the gap from your beliefs

• Therefore, action comes first, sensory input second
• Sensory input that does not confirm the action causes surprise
• No surprise, or one below a certain threshold, strengthens the internal attractors
• Surprise above a certain threshold can adapt the internal attractors

.

---

.

Biology fighting entropy

.

Self-organisation

Schrödinger famously observed that living systems are unique among natural systems because they appear to resist the second law of thermodynamics by persisting as bounded, self-organising systems over time.

Originating from biology, general selection entails three interacting principles of change: variation, selection, and retention . This Darwinian process not only applies to organisms (i.e., natural selection), but acts on all dynamically coupled systems (e.g., molecules, neural synapses, behaviours, theories, and technologies, and is a universal principle that cuts across both statistical and quantum mechanics. On the other hand, self-organisation stems from dynamic systems theory in physics, and refers to the emergence of functional, higher-order patterns resulting from recursive interactions among the simpler components of coupled dynamical systems over time. (1)

	Content source
(1)	Answering Schrödinger’s question: A free-energy formulation - Karl Friston - Physics of Life Review - 2018

.

Biological systems

All biological systems exhibit this specific form of self-organisation, sculpted by natural selection.

The self-organisation allows them to actively maintain their integrity by revisiting characteristic states within well-defined bounds.

How do living systems perform this feat?

Being alive is a continuous process of acting to minimise the gap between one's expectations and sensory inputs, a profound concept known as the 'Free Energy Principle'.

This principle is more straightforward than it might seem. It rests on the fact that all living systems repeatedly revisit a random set of dynamical attractors.

Understanding the concept of attractors is key, and it begins with distinguishing between the system and its environment.

1. On the one hand, those states that constitute or are intrinsic to the system and,
2. On the other hand, those states that are not.

The existence of any system that can be distinguished from its external milieu mandates a Markov blanket (see next page) of dynamical attractors.

.

---

.

Free Energy (Principle)

The difference between our current position and the initial belief is called 'Free Energy'. We will always try to minimise this gap, this 'Free Energy'.

.

---

.

Minimising free energy

Active inference

Every 'moment' we act (doing something as subtle as whispering a word, to as violent as shopping up a tree), we sense the 'position' our body is in and match this with our prior belief about ourselves in the world. We constantly monitor our changing beliefs and perceptions and the motivations behind our bodies' actions. It goes like this:

1. First, the brain signals the body to act in a way so that the body is in a new state
2. Secondly, new sensory input (tries to) match(es) the pre-existing belief.

This process is called active inference.

.

Expectations causing surprise

The classical view of perception is that the brain processes sensory information in an ‘outside-in’ direction: sensory signals enter through receptors (for example, the retina) and then progress deeper into the brain, with each stage recruiting increasingly sophisticated and abstract processing. In the 19th century, the German polymath Hermann von Helmholtz proposed that the brain is a prediction machine and that we see, hear and feel nothing more than our brain’s best guesses about the causes of its sensory inputs.

This framework proposes that signals flowing into the brain from the outside world convey only prediction errors – the differences between what the brain expects and receives. The most recent view is that perceptual content is carried by perceptual predictions flowing in a ‘top-down’ direction from deep inside the brain out towards the sensory surfaces.

Perception involves minimising prediction error - which causes surprise - simultaneously across many levels of processing within the brain’s sensory systems by continuously updating the brain’s predictions.

In this view, often called ‘predictive coding’ or ‘predictive processing’, perception is a controlled hallucination in which sensory signals from the world and the body continually rein in the brain’s hypotheses.

.

Attractors

In humans, to minimise surprise, our body and brain excitations can be described as moving towards our attractors, that is, towards our most likely circumstances. All our thoughts and behaviours push us towards more and more probable states. How do we do that? By using two functions in doing so:

1. On the one hand, surprise – that is, the improbability of being in a specific state
2. On the other hand, evidence is the probability that a given explanation or model for that condition is correct.

If we exist, we need to increase our model proof or self-evidence to minimise surprises.

A rebounding state has a low surprise and high proof. Therefore, complex systems fall into known, reliable cycles because these processes are necessarily concerned with validating the principle underlying their existence.

• Attractors force systems to fall into predictable states, reinforcing the system's model of its world.

If this surprise-minimizing, self-evident, inferential behaviour fails, the system will fall into surprising, unknown states – until it no longer exists in any meaningful way. Attractors are the product of processes that deal with inferences to bring themselves into existence.

• In other words, attractors are the foundation of what it means to be alive.

.

Coherence

Your brain will tell your body (how) to act the 'next moment' in the expectation of minimising the gap from your beliefs.

As it is a minimising principle in living systems, it is opposite to entropy in non-living systems as entropy maximises constantly.

.

---

.

In conclusion

The answer we entertain is based upon a variational free energy minimisation principle that has proved useful in accounting for many aspects of brain structure and function. In brief, biological systems can distil structural regularities from environmental fluctuations (like changing concentrations of chemical attractants or sensory signals) and embody them in their form and internal dynamics. In essence, they become models of causal structure in their local environment, enabling them to predict what will happen next and counter surprising violations of those predictions. In other words, by modelling their environment they acquire a homoeostasis and can limit the number of states they find themselves in. This perspective on self-organisation is interesting because it connects probabilistic descriptions of the states occupied by biological systems to probabilistic modelling or inference as described by Bayesian probability and information theory. Minimising variational free energy (maximising Bayesian model evidence) not only provides a principled explanation for perceptual (Bayesian) inference in the brain but can also explain action and behaviour. The principle of variational free energy minimization has therefore been proposed to explain the ability of complex systems like the brain to resist a natural tendency to disorder and maintain a sustained and homoeostatic exchange with its environment. (2)

	Content source
(2)	A Free Energy Principle for Biological Systems - Karl Friston - Entropy - 2012

.

Do you want to know more?

.

Predictive processing

.

Predictive processing (PP) is a framework involving a general principle which can be applied to describe perception, action, cognition, and their relationships in a single, conceptually unified manner. It is not directly a theory about the underlying neural processes (it is computational, not neurophysiological), but there are more or less specific proposals of how predictive processing can be implemented by the brain (see, e.g., Engel et al. 2001; Friston 2005; Wacongne et al. 2011; Bastos et al. 2012; Brodski et al. 2015). Moreover, it seems that at least some of the principles which can be applied to descriptions on subpersonal (e.g., computational or neurobiological) levels of analysis can also be applied to descriptions on the personal level (e.g., to agentive phenomena, the structure of reasoning, or phenomenological reports which describe the contents of consciousness). This is one reason why PP is philosophically interesting and relevant. If the theory is on the right track, then: it may provide the means to build new conceptual bridges between theoretical and empirical work on cognition and consciousness, it may reveal unexpected relationships between seemingly disparate phenomena, and it may unify to some extent different theoretical approaches.

Overview and brief explanation of some central concepts involved in predictive processing (PP). Top-down Processing: Computation in the brain crucially involves an interplay between top-down and bottom-up processing, and PP emphasizes the relative weighting of top-down and bottom-up signals in both perception and action. Statistical Estimation: PP involves computing estimates of random variables. Estimates can be regarded as statistical hypotheses which can serve to explain sensory signals. Hierarchical Processing: PP deploys hierarchically organized estimators (which track features at different spatial and temporal scales). Prediction: PP exploits the fact that many of the relevant random variables in the hierarchy are predictive of each other. Prediction Error Minimization (PEM): PP involves computing prediction errors; these prediction error terms have to be weighted by precision estimates, and a central goal of PP is to minimize precision-weighted prediction errors. Bayesian Inference: PP accords with the norms of Bayesian inference: over the long term, prediction error minimization in the hierarchical model will approximate exact Bayesian inference. Predictive Control: PP is action-oriented in the sense that the organism can act to change its sensory input to fit with its predictions and thereby minimize prediction error; among other benefits, this enables the organism to regulate its vital parameters (like levels of blood oxygenation, blood sugar, etc.). Environmental Seclusion: The organism does not have direct access to the states of its environment and body (for a conceptual analysis of “direct perception”, see Snowdon 1992), but infers them (by inferring the hidden causes of interoceptive and exteroceptive sensory signals). Although this is a basic feature of some philosophical accounts of PP (cf. Hohwy 2016; Hohwy 2017), it is controversial (cf. Anderson 2017; Clark 2017; Fabry 2017a; Fabry 2017b). The Ideomotor Principle: There are “ideomotor” estimates; computing them underpins both perception and action, because they encode changes in the world which are registered by perception and can be brought about by action. Attention and Precision: Attention can be described as the process of optimizing precision estimates. Hypothesis-Testing: The computational processes underlying perception, cognition, and action can usefully be described as hypothesis-testing (or the process of accumulating evidence for the internal model). Conceptually, we can distinguish between passive and active hypothesis-testing (and one might try to match active hypothesis-testing with action, and passive hypothesis-testing with perception). It may however turn out that all hypothesis-testing in the brain (if it makes sense to say that) is active hypothesis-testing. Free Energy Principle: Fundamentally, PP is just a way of minimizing free energy, which on most PP accounts would amount to the long-term average of prediction error. (3)

	Content source
(3)	Philosophy and Predictive Processing - Thomas Metzinger & Wanja Wiese (Eds.) - MIND Group, Frankfurt am Main - 2017 - This collection is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License. www.predictive-mind.net

.