The Master Algorithm
In the final part of The Master Algorithm, Domingos puts forward the central thesis of the book: does there exist a universal learning algorithm that, given enough data, can learn anything that is learnable? He believes the answer is yes, and that such an algorithm must necessarily fuse the strengths of the existing five tribes. This page lays out that unification hypothesis, Domingos's own candidate (Markov Logic Networks), other fusion routes, the counterthesis (the No Free Lunch theorem), and the engineering reality as of 2026.
1. The Master Algorithm hypothesis
Domingos's central thesis (paraphrased from the book):
All knowledge — past, present, and future — can be derived from data by a single universal learning algorithm.
If this thesis holds, it would mean: - ML is no longer a collection of disparate techniques but different surfaces of a single underlying principle. - Engineering would no longer require hand-picking algorithms per problem — a single learner plus data would suffice. - Philosophically, it would also answer the "unified intelligence" question of cognitive science.
This is a strong assumption. Domingos himself acknowledges it may not be true, but its value lies in drawing a map of the relationships among the five tribes that researchers can use to look for the greatest common factor of all five.
Why no single tribe is enough
| Tribe | Strength | Fatal weakness |
|---|---|---|
| Symbolists | Logical reasoning, interpretability | Knowledge-acquisition bottleneck, brittleness |
| Connectionists | Perception, automatic representation learning | Black box, data-hungry, lacks causality |
| Evolutionaries | Black-box optimization, no gradients required | Extremely sample-inefficient |
| Bayesians | Uncertainty, small data | Computationally expensive, subjective priors |
| Analogizers | Simple, instant learning | Curse of dimensionality, choice of metric |
Domingos's argument: these weaknesses are typically the strengths of another tribe, so the right direction for unification is not replacement but combination.
2. Domingos's own candidate: Markov Logic Networks
Markov Logic Networks (MLN) (Richardson & Domingos, 2006) have been the main line of work in Domingos's group for almost two decades, with the explicit goal of fusing Symbolists + Bayesians into a single mathematical framework.
2.1 Intuition
- First-order logic: highly expressive, but Boolean — a formula is either true or false.
- Probabilistic graphical models: handle uncertainty, but struggle to express universal quantifiers like "for all \(x\)".
The MLN idea: assign real-valued weights to first-order logic formulas, where higher weights mean a "harder" rule. Violating a rule no longer drives the entire world's probability to zero, only lowers it.
2.2 Formalization
Given a set of first-order formulas \(\{F_i\}\) with corresponding weights \(\{w_i\}\), define a probability distribution over the set of ground atoms:
where: - \(n_i(x)\): the number of true groundings of formula \(F_i\) in world \(x\); - \(Z\): the normalization constant.
This is a Markov Random Field (the Bayesians' graphical model) — parameterized by first-order logic formulas (the Symbolists' language).
2.3 Inference and learning
- Inference: MAP inference → SAT problems / weighted MaxSAT; marginal inference → MCMC (Gibbs sampling on the grounded MLN).
- Learning: weight learning (maximizing pseudo-likelihood / voted-perceptron updates); structure learning (searching the space of first-order clauses).
graph LR
A[First-order formulas + weights] --> B[Grounding]
B --> C[Markov Random Field]
C --> D{Inference}
C --> E{Learning}
D --> D1[MaxSAT/Gibbs]
E --> E1[Weights: pseudo-likelihood]
E --> E2[Structure: ILP search]
style A fill:#ffe4b5
style C fill:#bbdefb
2.4 Assessment
MLNs have been deeply influential in the statistical relational learning (SRL) community, but as a candidate for the "Master Algorithm" they have limitations: - They only fuse Symbolists + Bayesians, leaving out the representation learning of the Connectionists. - Inference is expensive. - On large-scale perception tasks (images, speech), they cannot compete with deep learning.
MLNs are better understood as a schematic prototype of the unification hypothesis than as an industrial-grade answer.
Related symbolic-probabilistic bridges on this site: 概率推理与贝叶斯网络, 贝叶斯派.
3. Other fusion routes
Beyond MLN, there are many pairwise or multi-way fusion routes between the five tribes, each an active research area.
| Fusion | Name / representative work | Idea |
|---|---|---|
| Symbolists + Connectionists | Neuro-Symbolic AI (Garcez & Lamb 2020) | Neural networks for perception, symbolic systems for reasoning; DeepProbLog, neural theorem provers |
| Symbolists + Bayesians | Markov Logic Networks (Richardson & Domingos 2006); ProbLog | First-order logic + probability distribution |
| Symbolists + Evolutionaries | Genetic Programming (Koza 1992) | Tree-structured programs (symbolic) + genetic search |
| Connectionists + Bayesians | Bayesian Deep Learning; VAE (Kingma 2014) | Neural-network weights / latent variables are distributions |
| Connectionists + Evolutionaries | Neuroevolution (NEAT, OpenAI ES) | Evolutionary search over weights / topology of neural networks |
| Connectionists + Analogizers | Deep Kernel Learning (Wilson 2016); Contrastive Learning (SimCLR/CLIP) | Neural networks produce embeddings, downstream uses similarity |
| Bayesians + Analogizers | Gaussian Processes | The kernel function is similarity under a prior |
| Bayesians + Evolutionaries | Bayesian Optimization | Bayesian model as surrogate, evolutionary strategy for candidates |
| Multiple tribes | Probabilistic Programs | Programs (symbolic) + distributions (Bayesian) + embeddings (connectionist) |
graph TB
Sym[Symbolists]
Con[Connectionists]
Bay[Bayesians]
Ana[Analogizers]
Evo[Evolutionaries]
Sym ---|MLN/ProbLog| Bay
Sym ---|Neuro-Symbolic<br/>DeepProbLog| Con
Sym ---|GP| Evo
Con ---|VAE/BNN| Bay
Con ---|NEAT/OpenAI ES| Evo
Con ---|SimCLR/CLIP/DKL| Ana
Bay ---|Gaussian Processes| Ana
Bay ---|Bayesian Optimization| Evo
classDef tribe fill:#ffeaa7,stroke:#d4a017
class Sym,Con,Bay,Ana,Evo tribe
Each edge corresponds to one or more pages on this site; the 贝叶斯派, 进化派, and 类比派 tribe entries in this notebook all contain a "fusion with other tribes" subsection.
4. The 2026 perspective: Transformer + RLHF + tool use
Domingos's book was written in 2015. A decade later, the de facto answer from industry is: no one has yet written down a Master Algorithm, but the Transformer pathway has become a "partial unification in practice" on many tasks.
4.1 Large models as a product of multi-tribe fusion
Transformers + post-training + tool use in fact blend all five tribes:
| Stage | Dominant tribe | Manifestation |
|---|---|---|
| Pretraining (next-token prediction on the web) | Connectionists | Backpropagation, massive data, scaling laws |
| In-context / few-shot prompting | Analogizers | Reasoning by analogy from demonstrations |
| Chain-of-Thought / tool use | Symbolists | Explicit symbolic reasoning, logical chains |
| RLHF (reinforcement learning from human feedback) | Evolutionaries (behavioral trial and error) + Bayesians (the policy-stabilization part of PPO admits a Bayesian reading) | Behavior optimization |
| Uncertainty expression ("I'm not sure") / ensembling | Bayesians (weak form) | Still an engineering gap |
| RAG (retrieval-augmented generation) | Analogizers | Dense retrieval = the modern incarnation of the Analogizer tribe |
graph LR
Pre[Pretraining] -->|Connectionists| LLM[LLM]
LLM -->|Analogizers| ICL[In-context Learning]
LLM -->|Symbolists| CoT[Chain-of-Thought + Tools]
LLM -->|Evolutionaries+Bayesians| RLHF[RLHF/DPO]
RAG[RAG: Dense Retrieval] -->|Analogizers| LLM
style LLM fill:#ffeaa7
4.2 Is this the Master Algorithm?
No — but it is the closest engineering practice known so far. Issues remain:
- It is not a single algorithm but an engineered pipeline (pretrain + SFT + RLHF + RAG + tools).
- The five tribes are chained rather than truly fused: each stage is dominated by a single tribe.
- It lacks rigorous uncertainty quantification (the Bayesian coverage is thin).
- The compute cost is enormous, far from the "universal and efficient" algorithm Domingos envisioned.
- The mechanisms behind emergent capabilities are unclear (more like the black-box price tag of the Connectionists).
A closer unification will probably have to wait for a next-generation architecture — one that brings the world model, explicit reasoning, probabilistic uncertainty, and retrieval-generation under a single objective function.
5. The counterthesis: the No Free Lunch theorem
The strongest objection to the Master Algorithm hypothesis comes from the No Free Lunch (NFL) theorem of Wolpert & Macready (1997).
5.1 Formal statement
Over all possible target functions \(f: X \to Y\) and any two learning/optimization algorithms \(A_1, A_2\), the expected performance averaged over all target functions is identical:
Intuitively: no algorithm can beat all others on all problems. Whatever advantage an algorithm gains on one class of problems must be paid for with a disadvantage on another class.
5.2 Implications for the Master Algorithm hypothesis
NFL appears to refute the Master Algorithm directly — a universally optimal algorithm cannot exist. But care is needed:
- NFL assumes target functions are uniformly distributed over all possible functions. Real-world functions are not — they exhibit strong inductive biases (locality, smoothness, compositionality).
- Domingos's claim is weaker: there exists an algorithm that can learn anything that is learnable, not one that is optimal on all problems.
- "Learnable" itself requires structure in the data. NFL is concerned with worst case; the Master Algorithm is concerned with the real world.
5.3 The engineering response
The mainstream view in the practical community:
"Combination > silver bullet" — real ML practice is about choosing the right algorithm for a problem (or combining several), not about believing in a master key.
This pragmatic line has long dominated industry. It does not deny the value of fusing the five tribes, but treats the "Master Algorithm" as a research north star rather than a deliverable.
6. The engineering state of five-tribe fusion
Setting theory aside, real ML systems are already multi-tribe hybrids.
6.1 The MLOps view
| Component | Tribes involved |
|---|---|
| Data pipelines, feature engineering | Symbolists + Bayesians |
| Model training (DL) | Connectionists |
| Hyperparameter optimization | Bayesians + Evolutionaries |
| Retrieval / recommendation | Analogizers |
| A/B testing | Bayesians (Bayesian A/B) + Frequentist |
| Model monitoring (drift, calibration) | Bayesians |
6.2 Model ensembling
Ensemble methods are themselves the engineering version of "many tribes united":
- Bagging (Random Forest): multiple Symbolist models + Bayesian averaging.
- Boosting (XGBoost): sequential Symbolists fitting residuals.
- Stacking: a meta-learner (which can be from any tribe) combines base learners.
- Mixture-of-Experts (MoE): the sparse activation in modern LLMs is essentially Analogizer-style routing where "each expert specializes on one type of input".
6.3 The pragmatic conclusion
The Master Algorithm has not yet appeared; but the Master Toolbox already exists: MLOps platforms + foundation models + RAG + tool use + RLHF. The engineer's job is to know what each tribe can and cannot do, and to wire them together correctly.
7. Takeaways for learners
Reading this book and this notebook, the core points to remember:
- No silver bullet: every tribe has structural flaws; choosing a model is fundamentally choosing an inductive bias.
- Each tribe has a "master algorithm" hypothesis: Bayes' theorem, inverse deduction, backpropagation, genetic search, similarity queries.
- Modern ML is multi-tribe fusion: Transformer + RLHF + RAG + tool use is the de facto engineering pathway.
- The NFL theorem ≠ five-tribe fusion is meaningless: real-world inductive biases mean that a universal algorithm over the "set of learnable problems" may still exist.
- Best learning route: master each tribe's "master algorithm" first, then look at fusion work. The 贝叶斯派 / 进化派 / 类比派 sections of this notebook are organized around exactly this goal.
Back to the notebook entry: The Master Algorithm
References
- Pedro Domingos. The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. Basic Books, 2015 (Paperback 2018-02-13).
- Matthew Richardson, Pedro Domingos. "Markov Logic Networks". Machine Learning, 62(1-2), 2006.
- David H. Wolpert, William G. Macready. "No Free Lunch Theorems for Optimization". IEEE Transactions on Evolutionary Computation, 1(1):67-82, 1997.
- Artur d'Avila Garcez, Luis C. Lamb. "Neurosymbolic AI: The 3rd Wave". arXiv:2012.05876, 2020.
- Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, Eric P. Xing. "Deep Kernel Learning". AISTATS, 2016.
- Sebastian Borgeaud et al. "Improving Language Models by Retrieving from Trillions of Tokens" (RETRO). ICML, 2022.
- Pedro Domingos. "A Few Useful Things to Know About Machine Learning". Communications of the ACM, 55(10):78-87, 2012.