Deriving language models from mathematical theory — The first engineering validation of core primitives in AGI theory: BriLLM-MetaPred

When language models start to "grow out of mathematics": BriLLM-MetaPred makes the first core primitive derived from AGI theory finally materialize in practice

Is the design paradigm of language models only possible by humans continuously "stacking modules and adjusting structures"? As the entire industry makes increasingly expensive incremental improvements along the line of Transformer, Professor Zhao Hai's team from Shanghai Jiao Tong University has provided an counterintuitive answer: first strictly define AGI mathematically, then let machine learning models "grow" out of that definition.Their newly uploaded BriLLM-MetaPred on arXiv is the first practical implementation of this approach. It is not just another "better language model," but rather the first runnable instance of the machine learning primitive MetaPred derived from the mathematical theory of AGI (A Mathematical Theory of AGI).

Paper: BriLLM-MetaPred: A Representation-Prediction Unity Language Model with Adaptive Self-MeasurementarXiv: https://arxiv.org/abs/2610.34018Team: Institute for General Artificial Intelligence, Shanghai Jiao Tong University (Professor Zhao Hai's team)

The convergence of two independent lines of evidence: Isomorphism between mathematical derivation and brain principles

If we only look at the "deriving MetaPred from AGI definition" line, readers might reasonably ask: isn't it just a self-consistent formal system? The mathematical theory of AGI precisely provides a crucial, often overlooked supplement here: This mathematical approach is highly isomorphic with the mainstream predictive coding/free energy principle in neuroscience.

The last chapter of the theoretical paper specifically discusses this issue. The "natural balance principle" in the mathematical theory of AGI — a purely functional ratio involving energy (system cost) and entropy (system information gain) — also derives MetaPred as the optimal solution. This means that MetaPred is not only derived from the definition of "the limit of infinite self-awareness," but also from "the principle that any self-organizing system must achieve an optimal balance between energy cost and information gain," which is closer to physical and biological constraints. A mathematical primitive that simultaneously satisfies "the limit of cognition" and "natural economy" is unlikely to be just a pure engineering coincidence.

The Related Work section of BriLLM-MetaPred also explicitly lists Friston's free energy principle and Bastos et al.'s work on predictive coding microcircuits as theoretical references. This is not just decorative citation. The core proposition of predictive coding is that the brain maintains alignment between its internal model and the world by minimizing prediction error; while the core mechanism of MetaPred — a single dynamic state that simultaneously carries representation and prediction, and maintains internal consistency at a fixed point through continuous self-measurement — is precisely a mathematical restatement of predictive coding at an abstract level. Representation is not externally added, and prediction is not "read out" from representation; they are two aspects of the same dynamic process. Self-supervised signals are not external proxy tasks, but rather intrinsic constraints of the state approaching its fixed point. This constitutes an indirect interdisciplinary argument rather than a circular one. Neuroscience does not "prove" that MetaPred is correct, and the mathematical theory does not "predict" that the brain must adopt MetaPred. However, when a primitive derived from a pure mathematical definition meets a unified principle inferred from experimental neuroscience on the same abstract structure, the credibility of this theoretical path is no longer solely dependent on "it claims to be necessary and sufficient by itself," but gains an external anchor from an independent discipline. In other words, the theory behind BriLLM-MetaPred is not a completely untested formal system; it has already converged with the principled description of brain mechanisms on the level of "what the core computation of intelligence should look like." Starting Point: A self-governing and rigorous mathematical definition of AGI

Most AGI approaches either draw inspiration from biological brains or assemble engineering modules externally. BriLLM-MetaPred's foundation lies in a completely different path: A Mathematical Theory of AGI, which does not assume any specific network topology or cognitive module, but instead starts from the concept of 'universal cognition' itself, offering an autonomous mathematical definition and solving it.

This theory defines AGI as 'the limit of infinite self-cognition'. From this definition, the theory transforms it into a strict mathematical variational problem. In an isomorphic sense, the unique solution to this variational problem is the machine learning primitive named MetaPred (meta-prediction).

In other words, MetaPred is not designed by humans, but derived mathematically. The theory further argues that MetaPred is both necessary and sufficient for universal cognition. The paper states this clearly: 'MetaPred is the first machine learning paradigm entirely derived from a mathematical theory, rather than constructed through artificial design and incremental improvements—perhaps the last such paradigm.'

Core of MetaPred: A state that is both representation and prediction

So, what does this primitive 'grown' from the mathematical optimal solution look like?

MetaPred's defining characteristics are extremely concise yet fundamentally different from contemporary mainstream architectures: the 'representation' formed by the model and the 'prediction' it produces are two aspects of the same continuously evolving dynamics state.

Traditional language models separate representation and prediction into two distinct structures: encoders or hidden layers are responsible for forming representations, while output heads read predictions from these representations. MetaPred eliminates this separation. It maintains a single, continuously evolving dynamics state: context information is written into this state, the state itself evolves over time, and the prediction of the next token is directly read from the same state.



This leads to another key mechanism: continuous self-measurement. The model continuously measures its own dynamics state, which brings three effects: reducing inconsistency caused by finite depth, maintaining internal consistency, and providing endogenous self-supervised signals in the form of fixed points.

This point is worth pausing to examine. Current mainstream self-supervision relies on externally constructed proxy tasks—mask prediction, contrastive learning, next token prediction—where the signal source lies outside the model's dynamics. MetaPred's self-supervision signal originates from the state's measurement of itself, and its training objective formally equates to requiring internal representations to converge toward stable self-consistent points during iteration. This is an 'endogenous' rather than 'exogenous' supervisory signal.



Because training and inference share the same dynamics state, MetaPred exhibits symmetry in almost all key aspects of machine learning—described by the paper as 'an intuitive and observable property of the mathematical optimal solution within the paradigm space.'

BriLLM-MetaPred: Putting the primitive into an executable carrier

The theory provides the primitive, but the primitive needs a carrier to run on real hardware.

BriLLM-MetaPred's engineering path clearly connects two prior contributions: the AGI theory provides the construction and motivation of the MetaPred primitive; the SiFu node-signal flow paradigm introduced by BriLLM offers the carrier through which the primitive can be executed on existing software and hardware accelerators.

The paper clearly defines its boundaries: BriLLM-MetaPred is not another cyclic language model, nor a simple extension of BriLLM. It is a bridge—connecting the primitives derived from theory with the executable carriers in engineering.

At the implementation level, it is a node-state language model: token projection vectors are written into fixed interface rows, a recurrent operator updates the entire tensor through dynamic evolution, and the distribution of the next token is read out from the specified output row and a finite auxiliary pool. The same continuously evolving tensor simultaneously carries input-conditioned representations and the state used for reading out.

The key empirical results are: the predictive performance of BriLLM-MetaPred is comparable to that of the GPT baseline with similar scale (approximately 13.2M parameters).

For a new paradigm derived from scratch, the significance of this result does not lie in 'outperforming GPT,' but in demonstrating that this primitive derived from mathematics is engineering feasible for the core cognitive task of language modeling. The paper positions it as 'preliminary evidence of the engineering feasibility of the MetaPred framework.'

Compared to its predecessor BriLLM, this represents a major performance leap: the latter required about ten times the model size of GPT-1 to barely achieve comparable generation performance; whereas BriLLM-MetaPred achieves the same effect with about one-tenth the scale.

Currently, the maximum validation size has reached 1.5B, matching the scale of GPT-2.



Unique Position in Parallel Approaches:The only AGI theoretical path that has completed LLM sequence engineering validation

. Placing BriLLM-MetaPred back into the lineage of AGI theoretical approaches clarifies its position. There are not lacking attempts that derive intelligence from some mathematical or physical principle, but most of these approaches have not achieved engineering alignment on contemporary LLM sequences. The unique advantage of BriLLM-MetaPred does not lie in its theory being 'more correct' than the free energy principle, nor in it being 'more elegant' than AIXI, but rather in: it is the only approach that simultaneously realizes both 'deriving from mathematical primitives' and 'achieving contemporary baseline performance on LLM sequences' . This is a qualification proof: this theoretical path has the qualification to enter the same empirical dialogue as mainstream paradigms, rather than remaining confined to philosophy or small-scale toy tasks.

Precisely for this reason, the seemingly restrained result of 'being comparable to the GPT baseline of the same scale' holds significant positioning meaning within a broader competitive landscape of approaches. It does not proclaim that AGI has arrived, but it brings an approach that previously only existed in mathematical derivations for the first time onto the contemporary engineering testing ground of language models.

Alignment with GPT's Frontline Challenges: Not a larger encyclopedia, but a dynamically core capable of continuous self-correction

. Understanding BriLLM-MetaPred merely as 'another language model' would miss the actual problem it is aligned with. The paper repeatedly emphasizes at the paradigm level: The difference between MetaPred and GPT is not a difference between architectural variants, but a difference at the level of 'machine learning paradigms.' GPT is built upon Transformer, with its representation and prediction as two separate computational stages; whereas the defining characteristic of MetaPred is that representation and prediction are two aspects of the same continuously evolving dynamics state, with training and inference sharing the same state evolution path, and there being no independent "training forward" and "inference forward." This paradigm-level difference precisely aligns with several of the most attention-grabbing structural pain points in the current GPT approach.

First, continual learning and catastrophic forgetting. In mainstream GPT paradigms, training and inference are separated: after training, model parameters are frozen, and learning during deployment can only be achieved indirectly through context windows or external retrieval. MetaPred's single dynamics state inherently couples training and inference: during training, self-measurement provides gradients for parameter updates; during inference, the same self-measurement offers direction for state correction. The "continuous-execution training regime" implemented in the paper retains node states and fast parameters between sequence segments, allowing the model to maintain state continuity within sequences rather than reinitializing at each segment. This is not yet a complete lifelong learning system, but it provides an architectural foundation where training and inference are not split—continual learning is no longer an additional module to be appended, but a natural consequence of paradigm symmetry.

Second, the structural prerequisite for recursive self-improvement (RSI). The core requirement of RSI is that the system can measure its own performance and improve itself accordingly. MetaPred's continuous self-measurement mechanism—where the model constantly measures its own dynamics state, generating endogenous self-supervised signals in a fixed-point form—structurally already contains a rudimentary "self-assessment → self-correction" closed-loop. The paper describes this as an instantiation of the Lambek fixed-point lemma in machine learning: the difference between finite-step simulations and deeper-step simulations forms a consistency signal that does not require external labels. This is still far from complete RSI, but it transforms "self-measurement" from an external monitoring tool into an endogenous component of the model's dynamics. In a roadmap oriented toward RSI, this represents a more fundamental starting point than "wrapping a self-assessment module around GPT."

Third, scalability challenges, and the notion that "AGI does not have to be an encyclopedia." Discussions on scalability in the GPT approach have long focused on the synchronized expansion of parameter size, data size, and computational scale. MetaPred's theoretical derivation offers a different direction: since the model carrier and dynamics state are weakly coupled, the dynamics state only needs to align with a subset of the carrier, rather than being strictly isomorphic to the entire architecture. The paper explicitly states that this relaxation of constraints is the fundamental mechanism enabling BriLLM to support arbitrarily long input and output while keeping the model size constant. In other words, handling longer contexts and more complex sequences does not necessarily require the model itself to expand infinitely. This directly resonates with the judgment that "AGI does not have to be an encyclopedia, nor an infinitely large model": if the core of intelligence is a continuously evolving, self-measuring, and self-correcting dynamics process that maintains internal consistency through interaction with the environment, then knowledge storage does not need to be fully compressed into parameters, and capability expansion does not have to equate to linear or even superlinear growth in parameter count. MetaPred's theory even predicts that performance can be decoupled from model scale, and BriLLM-MetaPred achieving the same baseline performance as GPT at roughly a tenth of GPT-1's scale is the first engineering milestone in this direction.

It must be noted with restraint: these alignments are currently "structural pointers," not "completed solutions". The continuous-execution training mechanism is not yet lifelong learning in an open world; the fixed-point self-measurement closed-loop is not yet recursive self-improvement; and the constant model size supporting arbitrary-length input does not equate to the scalability problem being resolved. However, they collectively point to a default setting different from the GPT paradigm: the core of intelligence is not "storing as much knowledge as possible in frozen parameters," but rather a dynamics process capable of continuous evolution, self-measurement, and maintaining internal consistency through interaction with the environment. Under this setting, "a larger encyclopedia" is not an inevitable path to AGI—a sufficiently fundamental dynamics core may be closer to the definition of general cognition than a larger static knowledge repository.

Breakthroughs at the core component level, rather than AGI's final endpoint

Understanding BriLLM-MetaPred hinges on recognizing what it has achieved and what it does not claim.

It has instantiated one of the most difficult and important components in the core learning primitives derived from the AGI mathematical theory—the language model—and has demonstrated that it can function, reaching performance levels comparable to those of contemporary mainstream architectures. This is the first engineering validation anchor between mathematical definitions and executable systems.

It does not claim that this language model instance itself is a complete AGI system. The title and abstract of the paper consistently position it as the first engineering implementation of a language model, rather than a full realization of general cognition. System-level requirements such as agency, embodied loops, and lifelong learning from the theory belong to subsequent parts of the theoretical blueprint, not the engineering contribution of this paper.

The value of BriLLM-MetaPred lies precisely in its refusal to bypass the most challenging step. A AGI theory path that cannot be implemented on an open-domain learning task of this scale and complexity will always remain in doubt. BriLLM-MetaPred provides the first engineering anchor for the path of "deriving machine learning paradigms from mathematical theory" with a modest yet testable result of "performance comparable to that of GPT of the same scale."

BriLLM-MetaPred is a brain-like model, but unlike other brain-like models: the "brain-like" characteristics here are a natural and inevitable result of theory and engineering implementation, rather than a design motivation.

It is not a faster engine, but rather the first prototype engine built according to a new, autonomous mathematical blueprint—and this prototype has already started running on the test bench.