Free Energy Principle

Chapter 1 - Worldview

---

Previous page: Chaos - Free Energy Principle - Next page: Markov blanket

---

Back to Book content or directly to Main Page or Worldview

---

Connections create energy

Welcome to the Free Energy Principle page

.

Your road away from chaos

.

You read in the entropy part:

• High entropy means a low potential energy (towards chaos)
• Low entropy means a high potential energy (towards life)

How do we deal with this as humans?

Your brain is an energy guzzler. Energy needed for survival. Your brain will constantly try to minimise energy consumption by minimising its entropy.

It tries to do this by evaluating - (through a Bayesian method) - the difference between:

• the reactions to the actions of your body
• and the already existing model of the world in your brain.

If (and only if) this evaluation passes a certain threshold, the brain will change its internal model to rebalance its homeostasis.

.

Key take-aways from the deep dive

• Your brain will tell your body (how) to act the 'next moment' in the expectation of minimising the gap from your beliefs

• Therefore action comes first, sensory input second
• Sensory input that does not confirm the action causes surprise
• No surprise, or one below a certain threshold, strengthens the internal attractors
• Surprise above a certain threshold can adapt the internal attractors

.

Deep dive

Biology fighting entropy

Self-organisation

Schrödinger famously observed that living systems are unique among natural systems because they appear to resist the second law of thermodynamics by persisting as bounded, self-organising systems over time.

Originating from biology, general selection entails three interacting principles of change: variation, selection, and retention . This Darwinian process not only applies to organisms (i.e., natural selection), but acts on all dynamically coupled systems (e.g., molecules, neural synapses, behaviours, theories, and technologies, and is a universal principle that cuts across both statistical and quantum mechanics. On the other hand, self-organisation stems from dynamic systems theory in physics, and refers to the emergence of functional, higher-order patterns resulting from recursive interactions among the simpler components of coupled dynamical systems over time. (1)

	Content source
(1)	Answering Schrödinger’s question: A free-energy formulation - Karl Friston - Physics of Life Review - 2018

.

Biological systems

All biological systems exhibit this specific form of self-organisation, sculpted by natural selection.

The self-organisation allows them to actively maintain their integrity by revisiting characteristic states within well-defined bounds.

How do living systems perform this feat?

To be alive is to act in ways that reduce the difference between one's expectations and one's sensory inputs. This idea is called the 'Free Energy Principle'.

This principle is simpler than it might seem. It rests on the fact that all living systems repeatedly revisit a random set of dynamical attractors.

To understand the concept of attractors, one needs to differentiate between the system and its environment.

1. On the one hand, those states that constitute or are intrinsic to the system and,
2. on the other hand, those that are not.

The existence of any system that can be distinguished from its external milieu mandates a Markov blanket (see next page) of dynamical attractors.

.

---

.

Minimising free energy

Active inference

Every 'moment' we act (doing something as subtle as whispering a word, to as violent as shopping up a tree), we sense the 'position' our body is in and match this with our prior belief about ourselves in the world. We constantly monitor our changing beliefs and perceptions and the motivations behind our bodies' actions. It goes like this:

1. First, the brain signals the body to act in a way so that the body is in a new state
2. Secondly, new sensory input (tries to) match(es) the pre-existing belief.

This process is called active inference.

.

Expectations causing surprise

The classical view of perception is that the brain processes sensory information in an ‘outside-in’ direction: sensory signals enter through receptors (for example, the retina) and then progress deeper into the brain, with each stage recruiting increasingly sophisticated and abstract processing. In the 19th century, the German polymath Hermann von Helmholtz proposed that the brain is a prediction machine and that we see, hear and feel nothing more than our brain’s best guesses about the causes of its sensory inputs.

This framework proposes that signals flowing into the brain from the outside world convey only prediction errors – the differences between what the brain expects and receives. The most recent view is that perceptual content is carried by perceptual predictions flowing in a ‘top-down’ direction from deep inside the brain out towards the sensory surfaces.

Perception involves minimising prediction error - which causes surprise - simultaneously across many levels of processing within the brain’s sensory systems by continuously updating the brain’s predictions.

In this view, often called ‘predictive coding’ or ‘predictive processing’, perception is a controlled hallucination in which sensory signals from the world and the body continually rein in the brain’s hypotheses.

.

Attractors

In humans, to minimise surprise, our body and brain excitations can be described as moving towards our attractors, that is, towards our most likely circumstances. All our thoughts and behaviours push us towards more and more probable states. How do we do that? By using two functions in doing so:

1. On the one hand, surprise – that is, the improbability of being in a specific state
2. On the other hand, evidence is the probability that a given explanation or model for that condition is correct.

If we exist, we need to increase our model proof or self-evidence to minimise surprises.

A rebounding state has a low surprise and high proof. Therefore, complex systems fall into known, reliable cycles because these processes are necessarily concerned with validating the principle underlying their existence.

• Attractors force systems to fall into predictable states, reinforcing the system's model of its world.

If this surprise-minimizing, self-evident, inferential behaviour fails, the system will fall into surprising, unknown states – until it no longer exists in any meaningful way. Attractors are the product of processes that deal with inferences to bring themselves into existence.

• In other words, attractors are the foundation of what it means to be alive.

.

Free Energy Principle

The difference between our current position and the initial belief is called 'Free Energy'. We will always try to minimise this gap, this 'Free Energy'.

.

Coherence

Your brain will tell your body (how) to act the 'next moment' in the expectation of minimising the gap from your beliefs.

As it is a minimising principle in living systems, it is opposite to entropy in non-living systems as entropy maximises constantly.

.

In conclusion

The answer we entertain is based upon a variational free energy minimisation principle that has proved useful in accounting for many aspects of brain structure and function. In brief, biological systems can distil structural regularities from environmental fluctuations (like changing concentrations of chemical attractants or sensory signals) and embody them in their form and internal dynamics. In essence, they become models of causal structure in their local environment, enabling them to predict what will happen next and counter surprising violations of those predictions. In other words, by modelling their environment they acquire a homoeostasis and can limit the number of states they find themselves in. This perspective on self-organisation is interesting because it connects probabilistic descriptions of the states occupied by biological systems to probabilistic modelling or inference as described by Bayesian probability and information theory. Minimising variational free energy (maximising Bayesian model evidence) not only provides a principled explanation for perceptual (Bayesian) inference in the brain but can also explain action and behaviour. The principle of variational free energy minimization has therefore been proposed to explain the ability of complex systems like the brain to resist a natural tendency to disorder and maintain a sustained and homoeostatic exchange with its environment. (2)

	Content source
(2)	A Free Energy Principle for Biological Systems - Karl Friston - Entropy - 2012

.

---

Next page: Markov blanket

---

.