The Free Energy Principle and Active Inference

I. The Ambition of a Unified Theory

In neuroscience and artificial intelligence, perception, action, and learning are typically studied as separate problems. But Karl Friston made an extraordinarily bold claim:

All biological systems — from single cells to the human brain — are doing the same thing: minimizing variational free energy.

This is the Free Energy Principle (FEP). It attempts to explain all adaptive behavior of living systems within a single, unified mathematical framework.

II. What Is Free Energy?

The "free energy" here is not the Helmholtz free energy from thermodynamics, but rather variational free energy from variational Bayesian inference.

An intuitive understanding:

Free energy ≈ prediction error + model complexity

More formally, for a system with internal states (beliefs):

It maintains a generative model of the external world
It receives sensory observations
Free energy measures the discrepancy between "current beliefs" and "the true state implied by sensory data"

Free energy is an upper bound on surprise. Biological systems cannot compute surprise directly (since that would require knowledge of the true world state), but they can compute free energy as an approximation. Therefore:

Minimizing free energy ≈ indirectly minimizing surprise ≈ making one's predictions as accurate and parsimonious as possible

III. Two Pathways: Perception and Action

The most critical insight of the free energy principle is that biological systems have two ways to reduce free energy.

Pathway One: Updating Beliefs (Perception)

Change one's internal model to better fit the world.

This is exactly what Predictive Coding does. When sensory input conflicts with predictions, the system updates its beliefs to reduce the error.

Example: You hear an ambiguous sound and initially assume it is the wind, but as information accumulates, you update your belief — it turns out someone is speaking.

Pathway Two: Acting on the World (Action)

Change the external world to match one's predictions.

This is Active Inference. The system does not merely adjust beliefs passively; it actively takes action to make the world conform to its expectations.

Example: You predict that your hand should be above the table. If you actually feel your hand beneath the table, you can either update your belief ("Oh, my hand is below"), or move your hand (making the world match your expectation).

Pathway	Strategy	Corresponding Theory	What Changes
Perception	Update beliefs to fit the world	Predictive Coding	Internal model
Action	Change the world to fit beliefs	Active Inference	External environment

The unification of these two pathways is the most elegant aspect of the free energy principle. Perception and action are no longer two independent systems, but rather two implementations of the same objective: minimizing free energy.

IV. Comparison with Reinforcement Learning

In artificial intelligence, action is typically handled by Reinforcement Learning (RL): an agent maximizes cumulative reward. Active inference offers a fundamentally different framework.

Dimension	Reinforcement Learning	Active Inference
Core objective	Maximize external reward	Minimize expected free energy (expected surprise)
Source of reward	Defined by the environment or humans	Derived naturally from the generative model
Exploration mechanism	Requires additional design (e.g., epsilon-greedy)	Information gain emerges naturally as an exploration drive
Requirements on the world	Requires an explicit reward function	Requires only a generative model
Theoretical status	Engineering method	Claims to be a fundamental principle of biological intelligence

A key theoretical result:

Active inference can subsume reinforcement learning as a special case — when the generative model includes "preference priors" (i.e., prior beliefs about preferred states), minimizing expected free energy becomes equivalent to maximizing expected reward.

This means active inference does not require an external reward signal. The "rewards" of a biological organism are actually prior preferences written into the generative model by evolution: maintaining body temperature, sustaining blood sugar levels, avoiding tissue damage, and so on.

V. Mathematical Intuition

The mathematical expression of free energy can be written as:

F = prediction error + complexity cost (KL divergence)

Where:

Prediction error measures how well the model fits the observed data — the more accurate the model's predictions, the smaller this term
Complexity cost measures how much the posterior beliefs deviate from the prior beliefs — the more parsimonious the model (the closer to the prior), the smaller this term

There is a tension between these two terms:

Pursuing only accurate prediction can lead to overly complex models (overfitting)
Pursuing only parsimony can lead to ignoring important sensory data

The brain's task is to find the optimal balance between accuracy and parsimony — making predictions that are as accurate as possible with a model that is as simple as possible.

This is highly consistent with principles from machine learning such as regularization, Occam's razor, and minimum description length.

VI. The 2025 Frontier: VERSES AI and Engineering Breakthroughs

The free energy principle was long considered an elegant but difficult-to-engineer theory. In 2025, VERSES AI began to change this landscape.

The Genius Platform (April 2025)

VERSES AI launched the Genius platform, the first attempt to engineer active inference at scale. According to published benchmarks:

Genius matched or exceeded the performance of state-of-the-art deep reinforcement learning and Transformer models on multiple tasks, while using only about 10% of the training data.

This data efficiency advantage is not surprising — active inference inherently possesses strong priors and efficient inference capabilities.

The AXIOM Architecture

The underlying architecture of Genius is called AXIOM, with the following core features:

Uses probabilistic beliefs rather than deterministic weights
Updates beliefs through message passing rather than backpropagation
Natively supports uncertainty representation and multi-scale reasoning
Built on the mathematical framework of active inference

A Pre-Training-Free Robotics Architecture (August 2025)

In August 2025, VERSES demonstrated a robotics control architecture based on active inference, with a particularly striking property:

No pre-training required. The robot learns online through active inference, in real-time interaction with its environment.

This stands in sharp contrast to the prevailing paradigm of "large-scale pre-training first, then fine-tuning for deployment."

VII. Friston's Insight: LLMs and Free Energy

Karl Friston offered a thought-provoking reinterpretation:

Large Language Models (LLMs) can be understood as approximate inference engines, where next-token prediction is essentially a form of free energy minimization.

The logic behind this analogy:

An LLM possesses an implicit "generative model" (parameters shaped by training data)
Given context (observations), the LLM predicts the next token (inference)
The training process minimizes cross-entropy loss, which is formally similar to minimizing variational free energy

However, Friston also pointed out key differences:

LLM inference is feedforward and single-pass, lacking the iterative belief updating found in active inference
LLMs have no action channel — they cannot reduce surprise by changing the world
The "generative model" of an LLM is implicit and does not possess explicit world dynamics

This perspective both provides a theoretical explanation for the success of LLMs and clearly identifies their limitations.

VIII. Why Active Inference Matters for General Intelligence

Current AI systems typically design perception, decision-making, and learning as independent modules. But a defining feature of biological intelligence is:

Perception, action, and learning are an inseparable, unified process.

Active inference provides this unity. An active inference agent:

During perception, it minimizes free energy with respect to the current state
During action, it minimizes expected free energy with respect to future states
During learning, it updates the parameters of the generative model to reduce long-term free energy
During exploration, it seeks actions that maximize information gain (reduce uncertainty)

All of this stems from the same objective function. There is no separate reward function to design, no separate exploration strategy to tune, and no interface between perception and action to engineer.

Consider how an infant learns to grasp objects: the infant has a prior about hand position, seeing a toy generates free energy, moving the arm reduces it, the feedback updates beliefs about arm dynamics, and uncertainty drives new attempts. The entire process requires no external reward — only a generative model and the drive to minimize free energy.

IX. Criticisms and Open Questions

Falsifiability: Is the FEP so general that it cannot be falsified?
Computational challenges: Exact variational inference is difficult in high-dimensional spaces; whether VERSES's engineering approach can scale to truly complex real-world tasks remains to be seen
The gap with deep learning: Active inference has not yet demonstrated overwhelming advantages on standard benchmarks
Where does the generative model come from: How to obtain a sufficiently good generative model is itself a core challenge

X. Summary and the Complete Logical Chain

The free energy principle asserts: life is inference. All adaptive behavior — perception, action, learning, exploration — is the minimization of variational free energy.

The complete logical chain:

Biological systems maintain a generative model of their environment
Free energy measures the discrepancy between model predictions and actual observations
There are two pathways to minimize free energy: updating beliefs (perception) and acting on the world (action)
This unifies perception and action — predictive coding handles the former, active inference handles the latter
Active inference can subsume reinforcement learning as a special case, requiring no external reward
Free energy = prediction error + complexity cost; the brain seeks a balance between accuracy and parsimony
VERSES AI's engineering efforts demonstrate that active inference has significant advantages in data efficiency
The next-token prediction of LLMs can be reinterpreted as a form of free energy minimization
Active inference provides the theoretical foundation for building unified perception-action-learning systems