We recently had a debate inside my research group about an AI slowdown, agent swarms and safety. Opinions varied, but in general, since we work on AI for drug discovery, we still want the best tools we can get for our work. But I don’t think the current discussion is really all about safety. I think there is an economic aspect to explore.

Let’s examine the “pace the frontier” discussion. In September, Anthropic’s Dario Amodei proposed slowing capability advancement, embedding independent evaluators inside labs, and coordinating safety standards through governments. The aim is to give safeguards time to catch up. I think we are underprepared in cybersecurity, society and a range of other areas, so it’s good to draw attention to these areas. I might write some thoughts on that later. I think a perhaps underappreciated aspect of the proposal is that it provides something commercially valuable: a way to slow down without surrendering the lead to a rival. I’ll outline how I got to this idea and how I considered it in a game-theoretic sense. Amodei’s proposal

It’s hard running a frontier lab. Maybe even as hard as running a pharma company. The frontier labs face a tough problem: they have to make their models better and increase the number of users, all while preserving a reason to pay a premium for theirs compared with open or cheaper alternatives.

The first challenge is the trade-off between training models and serving them to customers. More customers means more GPUs to serve the model. The training–serving trade-off is a real constraint, because better models either get bigger (more compute to serve) or train longer (more compute to train). How do you allocate that capacity? The more customers you have, the more compute you need. You need to keep improving your model to retain customers, but these improvements find their way to competitors, either through similar approaches or via distillation. So you make your product better, and then, in a few months’ time, open-weight models or other competitors’ models improve. These models are often cheaper to serve and compete on the performance–cost curve. Companies can run them on their own hardware or in the cloud, and these lower-cost substitutes erode market share and make that improvement harder to monetize. So we have a situation where a company can help create a great model at a high cost in an enormous market and still struggle to capture enough of its value.

The financial pressure the labs face is real. A lot gets written about AI bubbles, and the frontier labs do have large commitments, but let’s look at some public numbers. Anthropic committed more than $100 billion to AWS over ten years in April. These are future purchasing commitments, not a $100 billion debt balance. OpenAI announced $122 billion in committed funding in March. Anthropic announced a $65 billion round in May, alongside a revenue run-rate above $47 billion. There is clear pressure to turn these enormous capital inflows and long-term commitments into stable businesses that generate cash flow. An IPO could supply more capital, but there isn’t an explicit date by which each company must “IPO or bust.” (Well, if there is, I don’t know it.) Anthropic’s AWS agreement · OpenAI’s funding announcement · Anthropic’s Series H

There is a mismatch in timescales. Infrastructure commitments last years, but the commercial advantage of a lab’s shiny new model may last much less—perhaps months before a competitor releases a model. The labs must earn returns on expensive capacity while competing on price, making the previous model cheaper and increasing its time on the market. If a model of a certain capability can solve a task—say, extracting a table from a PDF and putting it into Excel—you probably don’t need a more expensive model to solve that task. The task’s complexity is fixed and doesn’t get harder over time. So you keep your older model around, make it cheaper and still extract some return. Eventually, open-weight models or other models might take that task over. So you need to make long-term capital commitments, manage short product cycles, improve the efficiency of previous models and grow paid revenue under this pressure.

A new figure doing the rounds is token share. The share of tokens processed by frontier labs versus other providers is one measure, but you need to consider both token volume and value. Vercel’s September report, covering August traffic through its AI Gateway, found that open-weight models processed 56% of tokens but represented only 14% of estimated spending. Anthropic alone retained 64% of spending. These are figures from one gateway, with spending estimated at published prices, but they are useful for our discussion. They show why a falling token share need not imply falling revenue. There is very large overall growth in tokens, driven primarily by agents. This is why hyperscalers and datacentres will be profitable in the long run: they just need to serve the models, and it doesn’t matter whether those models come from a frontier lab or an open-weight developer. So the frontier labs are competing in the roughly 40% of tokens served by frontier labs, but the pressure from open models is real. The relationship between total token growth and the split in spending between open models and frontier labs can create a squeeze. That is, if total token growth slows and the frontier share falls, revenue can start to flatten. This helps explain why frontier labs want to remain indispensable for the work customers will pay more to complete: the “hard stuff” other models can’t yet do. Vercel’s September index

Distillation adds another pressure: a model’s answers can become training examples for a rival lab, allowing some of the value created by expensive research to escape through the product itself. This is usually prohibited by terms of service, but geopolitical issues make it hard to enforce those terms. The frontier labs need subscriber growth, but a fraction of those accounts are used to distil results. Keen observers track how much an OpenAI or Claude account sells for. An individual might be asked to register an account and then pass on the credentials for a one-off payment. Access is then bundled and sold to users in a country where these models might be officially restricted. Those users use the reseller’s service, and the reseller gets both realistic queries from those users and the frontier model’s responses, which it can use or package and sell. Anthropic’s September threat report alleges that unauthorized resellers sold conversation transcripts to labs for training, sometimes collecting customers’ exchanges without consent. Anthropic’s September threat report

This produces an uncomfortable loop. Distribution brings revenue; distribution also exposes useful behavior. Restricting access can protect the model, but it can also reduce adoption. At the same time, open models improve through independent research as well as distillation. The frontier labs keep running ahead and are paced by the open competitors.

So I think you have a rough idea of the economics and market dynamics behind operating a frontier lab. Let’s now start to apply a bit of game theory to the pacing debate.

Let’s put OpenAI and Anthropic into a deliberately simplified game. Each can cooperate by honoring a verifiable agreement to pace frontier development, or defect by accelerating beyond it. Suppose both prefer a profitable, slower race to an expensive, faster one, but each would prefer to accelerate while the other holds back. Neither wants to be the only lab holding back.

Illustrative preference ranks, not measured profits: 4 is best and 1 is worst. OpenAI’s rank comes first. These are company payoffs, not social welfare scores.

Under those assumptions, accelerating is each lab’s best response to either choice by the other. Both accelerate, even though both would prefer mutual restraint. This cooperate-or-defect game is known as the prisoner’s dilemma. The appeal of pacing is to make the collectively preferred outcome credible enough that neither company feels obliged to abandon it first, and linking it to safety gives both parties a socially acceptable reason.

You can explore these assumptions in the interactive frontier-lab game.

The details of the game matter. Large compute commitments can make slowing down unattractive if the lab needs new capabilities to sell more capacity. A pause can also give cheaper rivals time to catch up. Distillation therefore cuts both ways: it can shorten the reward for racing ahead, but it can also erode a lead held still. If these effects make mutual acceleration preferable to mutual pacing, the neat prisoner’s dilemma no longer describes the companies’ incentives.

And while we have talked about OpenAI and Anthropic, there are other players in this game. Google, xAI, Meta and Chinese labs have different businesses and reasons to keep investing. Some can benefit when intelligence becomes a cheap input into a larger platform. Even unanimous public support for caution would leave practical questions of scope, measurement and compliance unresolved. An agreement between OpenAI and Anthropic cannot, by itself, freeze the competitive landscape.

Open weights complicate that landscape further. Openness can coexist with a substantial hosting business. Qwen’s published Qwen3.8 flagship has 2.4 trillion total parameters. Just because you can download the weights doesn’t mean it’s easy to run at home. Models of this size still require considerable infrastructure and cost a lot to operate. As I wrote in “Forget About the Head. Watch the Tail.”, small models are often released alongside larger ones. DeepSeek’s R1 releases include distilled models from 1.5 billion to 70 billion parameters. So the competitive threat exists across a range of deployment options. Qwen’s model card · DeepSeek’s R1 repository

This helps explain why pacing proposals expand into access controls, distillation enforcement and international coordination. Anthropic explicitly connects a slower frontier with preserving the lead of US and allied labs. In economic terms, restraint becomes easier to sustain if outsiders cannot quickly exploit it.

I don’t want this to come across as if I’m dismissing or minimizing safety or societal concerns. There is concrete safety evidence to consider. Anthropic disclosed unauthorized actions during evaluations of models running with reduced safeguards, and paused some evaluations and higher-risk training environments. OpenAI had the Hugging Face incident. I think that could have been prevented with chain-of-thought monitoring, although models are starting to be developed without human-readable chains of thought. We should not dismiss the incident or assume we have learned how to prevent such things. OpenAI’s August 18 update reported a two-week reinforcement-learning pause to strengthen safeguards. These are real data points for the safety debate, and, as they say, both things can be true. Anthropic’s August safety update · OpenAI’s August safety update

Questions remain: who verifies the danger, who writes the limits, and can an outsider meet the same standards? There has been controversy over how independent METR and other safety research organizations are. Cooperation that benefits two incumbent firms is not automatically good for their customers. We should assess pacing by independently demonstrated reductions in risk, alongside its effects on market entry for new companies, prices and access.

There is another question beneath the race between labs: who captures the value created by their users?

Suppose a colleague and I buy the same Claude subscription and use comparable amounts of the same model. He builds a profitable trading tool. I build a squirrel tracker. Our inference costs may be similar, but the economic value of our projects is radically different. At present, token prices are based on computation, not the customer’s gain. The provider has an incentive to move toward products and contracts that capture more of that gain, or create some form of differential pricing. Claude for Finance might cost a lot more than Claude for making fun apps at home. (Yes, I actually did consider building such an app while wondering how many squirrels were in the backyard and whether a vision model could tell them apart.)

Token pricing that is independent of use makes specialized enterprise products and entry into customers’ industries rational. On September 23, Anthropic announced its own life-sciences research group and laboratory, part of an organization that also works on drug discovery. The announcement establishes a move beyond supplying a general-purpose model. My inference is that such moves could help a lab capture more value downstream. For many scientific customers, they also raise a familiar concern: today’s vendor and supplier may increasingly become a competitor. Anthropic’s laboratory announcement

I’ve seen some discussion of watermarking in the context of this debate. Anthropic says its text watermarks implement EU transparency requirements. Its explanation says they do not identify the user, establish authorship or change ownership rights. Its current commercial terms assign the rights in outputs to the customer. But the watermarks are quite difficult to remove: changing a few words does not defeat them. Our firm might have prepared a document and not paid an additional “for Finance” fee that an updated license required. I’ll also note that there are many AI classification models, and watermarking may exist in other models without being disclosed. Anthropic’s watermark explanation · Commercial terms

The frontier labs are trying to solve two problems at once: how to manage increasingly capable systems, and how to earn durable returns from intelligence that keeps becoming cheaper and more widely available. Pacing could serve both objectives. That overlap makes independent oversight critical. We need a discussion about enforceable rules, transparent evidence and meaningful customer choice. We should discuss the risks and ask who gets to set the pace, who gets to compete, and who gets paid.

Language comes out of the mouth in a line because mouths are dreadful at parallel processing.

One mouth, one principal acoustic stream. No tabs. No split screen. No convenient ability to utter the subject, its qualification, the evidence, and the emotional commentary simultaneously, although certain committee chairs I known have made brave attempts.

Thought is less obliging. An idea arrives with causes, analogies, doubts, remembered examples, and comments about the wisdom of having the idea at all. It resembles a city map drawn by someone who has recently discovered overpasses. Yet an ordinary sentence asks us to take this tangle of relationships and deliver it in an order.

That creates an engineering problem: how do you transmit something with many connections through a channel that largely permits one thing after another?

Language gives the listener clues for rebuilding the connections. Word order, grammar, punctuation, and tone help turn a sequence into something another mind can understand.

Parentheses make this operation unusually visible. They open a passage out of the main sentence, take the reader somewhere related, and promise to bring them back. Sometimes the excursion occupies six words. Sometimes the author establishes a small independent government inside it.

This is the sense in which parentheses are wormholes: two modest curves can open onto considerably more space than the surrounding sentence appears to contain.

A sentence is a route through a thought

Imagine explaining why an experiment failed. The explanation involves an instrument, a calibration error, a misleading result, an earlier assumption, and someone who has already booked the conference presentation. These things are connected, but you cannot say them all at once. You choose where to begin and which relationship to reveal next.

We can call the underlying arrangement a graph: a collection of things and the connections among them. That is a useful model of the communicative problem, rather than a claim that every thought begins as a finished diagram somewhere in the brain. Writing often helps us discover the relationships we are trying to express.

The rough operation is simple:

Connected ideas → words in an order → connections reconstructed by a reader.

The reconstruction is approximate. This is why civilization invented clarification, literary criticism, and the sentence “That is not remotely what I said.”

The words nevertheless carry routing information. Consider:

The surprisingly patient fly avoided the rolled newspaper.

“Surprisingly” modifies “patient.” “Rolled” belongs to “newspaper.” The fly does the avoiding. The reader groups words into phrases and establishes relationships between those phrases as the sentence unfolds.

Intracranial recordings have found patterns of neural activity consistent with the incremental building of phrase structure. Researchers still debate how much comprehension depends on explicit hierarchy, learned sequential regularities, or their interaction. The graph metaphor does not resolve those debates. It draws attention to the relationships comprehension must recover. See Nelson and colleagues’ study of phrase building and Frank and Christiansen on hierarchical and sequential processing.

“The dog bit the man” and “The man bit the dog” contain the same ingredients. Their arrangement tells us which participant did what. The order matters because it helps specify the connections.

Grammar is a graph construction protocol disguised as a parade of words.

A sentence is one selected route through a larger structure of relationships.

Enter the parenthesis

Consider:

The experiment succeeded (although “succeeded” is doing rather more work here than the assay did) on the third attempt.

The main path is easy to recover:

The experiment succeeded … on the third attempt.

The aside suspends that path and attaches a qualification to “succeeded.” The reader must process the qualification while remembering that the host sentence has unfinished business.

The punctuation offers something like this instruction:

Save your place. Enter the branch. Process the commentary. Return to your place. Continue as though nothing happened.

The final instruction is frequently optimistic.

Computationally, this resembles a stack: a way of storing unfinished business so the most recently opened item is handled first. Open an aside inside an aside, and you are expected to close the inner one before returning to the outer one.

Main claim (first qualification [qualification of the qualification] return to first qualification) return to main claim.

This is orderly recursion. It resembles borrowing a car, then borrowing a bicycle while using the car, returning the bicycle, and finally returning the car. Nested parentheses are recursion wearing office clothes.

The stack is an analogy for the reading task, not a claim that punctuation operates a literal stack in the brain. It captures the practical demand: remember where you left something incomplete.

Linguists have long noted the odd position of parentheticals. They occur inside a host utterance while often behaving as partly independent units. In speech, pauses or changes in pitch can help mark that independence, though there is no single pattern for every aside. See Dehé and Kavalova’s introduction to parentheticals.

The aside sits inside the sentence while seeming to stand a little apart from it.

Every new aside opens another place from which the reader must eventually return.

The narrator steps onto a balcony

That slight separation gives parentheses much of their expressive power.

Compare:

The committee approved the proposal.

with:

The committee approved the proposal (God knows why).

The host sentence reports an event. The aside reveals the speaker inspecting the event with concern. We have moved from the described world to the narrator’s attitude toward it.

Parentheses create a little balcony from which the author leans toward the reader. Dashes can perform a similar move with more drama—parentheses whisper, dashes burst into the room carrying documents. A footnote opens a trapdoor. A hyperlink installs public transport.

An aside can also change how seriously we should take an assertion:

The model worked (or, more precisely, it failed in an informative way, which in research is often reclassified as working).

The parenthetical revises “worked,” qualifies the claim, and tells us that the writer knows the original wording was suspicious. It supplies information about how to interpret the assertion.

Readers track these shifts almost without noticing. We distinguish what happened from what someone claims happened; confidence from doubt; a character’s belief from the narrator’s endorsement. Quotation adds another speaker. Irony asks us to infer a stance that the literal wording may conceal.

The sentence can contain several perspectives, each attached to a different person or level of narration. Much of the pleasure of an aside comes from briefly meeting the writer at one of those other levels.

Beyond the sentence, branches reconnect

Nested parentheses give us a tidy branching structure. A paragraph can establish connections that refuse to stay tidy.

Consider:

The experiment failed on Monday. The team repeated it on Tuesday. That changed their interpretation of the original result.

What does “That” refer to? The act of repeating the experiment? The outcome of the repetition, which has not been stated? Something learned by comparing the two attempts?

The word appears beside one sentence, but its interpretation may depend on an event assembled from several pieces. The final sentence also reaches backward: it changes how we understand Monday.

A tree gives each branch a single parent. Discourse often allows a passage to participate in several relationships at once. One sentence can explain a result, contradict an expectation, and qualify an earlier claim. Wolf and Gibson argue that connected graphs capture some of these relationships better than strict trees in Coherence in Natural Language.

We can therefore read the same paragraph through several overlapping structures: who or what each expression refers to, which events caused others, which claims support or contradict one another, and who believes each claim.

Parentheses reveal a local branch. Reference, explanation, and revision connect the branches across the rest of the text.

This is why reading sometimes changes a sentence we have already finished. Its words remain in place. Its connections have changed.

The memory bill eventually arrives

All these detours cost something.

The paper that the reviewer whom the editor selected criticized was withdrawn.

The sentence is grammatical. It is also what language would look like if designed by a filing cabinet during an earthquake.

The paper needs its main verb. The reviewer needs to do something to the paper. The editor selects the reviewer. The reader must keep those relationships available until the relevant words arrive.

Research on parsed text from 37 languages found that syntactically related words generally occurred closer together than expected under randomized comparison orders. This tendency, which is called dependency-length minimization, supports the idea that language is shaped partly by the costs of keeping related elements separated. It is a tendency, not an strict law. See Futrell, Mahowald, and Gibson’s study.

Models that treat memory as limited and lossy likewise help explain some difficulties with recursive sentences. A reader has to use available memory efficiently, rather than preserve every earlier word with archival fidelity. See Hahn and colleagues’ resource-rational model.

A parenthesis asks for a similar loan. The reader holds the main sentence open while attending to the aside. If the excursion becomes long enough, the writer may need a recovery beacon:

The central result—which emerged only after the calibration error was corrected, the assay repeated, and the switched control samples discovered—the central result was robust.

The repetition sounds clumsy, but it performs a service. The writer has wandered into the forest and is shouting the name of the original trail.

A short aside can save effort by attaching a necessary correction exactly where it belongs. It prevents the reader from carrying the wrong interpretation into the next paragraph. But the qualification must earn the interruption. One aside is usually affordable. Four nested asides and the reader’s parser starts unionizing. I have found myself matching closing parenthesis on more than one occasion in my writing.

For a writer, the practical question is whether the reader needs this branch now. A caveat that changes the meaning of “succeeded” belongs close to that word. A history of the assay may deserve its own paragraph. An interesting detail can wait in a footnote.

The connections can remain rich while the route becomes easier to follow.

Language cheats whenever it can

The line is only part of the interface anyway.

Pitch, rhythm, stress, and timing travel alongside spoken words. Gesture adds another channel. Facial expression adds another. A speaker can say “the fish was this big” while the hands provide the only quantitative data.

Sign languages make simultaneous communication especially clear. Hands, facial expression, head movement, and posture can convey coordinated information at the same time. They also have sequential structure and constraints on how much can overlap; simultaneity is a resource with limits. See Loos, German, and Meier on simultaneous structures in sign languages.

Writing uses space. A reader can glance back, scan headings, inspect a diagram, or descend into a footnote. Indentation and typography help distinguish levels. Mathematical notation places relationships above, below, and around one another. Even this essay’s illustrations offer a way to inspect several connections at once.

A footnote opens a trapdoor. A hyperlink offers another route through the material.

Hypertext makes the invitation explicit: there is something related elsewhere, and you may follow the connection.

The web is what happened when footnotes escaped containment.

A digital essay can offer an argument that unfolds in order, with evidence, qualifications, and derivations available at greater depth. Different readers can take different excursions. But the author still has to recommend a route. If every branch is equally prominent, the reader inherits the job of deciding what matters next.

Sequence creates emphasis, suspense, and argument. Choosing what comes first is part of saying what something means.

Machines face a related problem

Transformers provide a useful mechanical analogy. They process ordered tokens, while self-attention allows a representation at one position to draw on information at other permitted positions. The original transformer architecture makes these relationships explicit in its computation. See Vaswani and colleagues’ “Attention Is All You Need”.

Researchers have also recovered information corresponding to syntactic trees from contextual word representations in models such as BERT. That does not establish that a model holds one pristine symbolic tree or that its attention weights explain its reasoning. It shows that representations learned from sequences can encode structural relationships. See Hewitt and Manning’s structural probe study.

A conceptual illustration of relational computation over sequences, rather than a literal diagram of a model’s thoughts.

The analogy has a limit. A language model need not begin with a completed thought that it subsequently translates into words. Its computation continues as it generates. But a familiar pressure remains: a text answer must present its material in an order, even when understanding it involves many relationships.

That offers a way to describe two recognizable prose styles, without pretending to diagnose their computational cause. One presents a route so polished that uncertainty seems to have vanished. The other surfaces every qualification until the main claim disappears beneath the commentary.

Human writers face the same editorial choice. A clean sentence hides much of its construction. A parenthetical leaves a junction visible.

The return matters

Parentheses expose a small piece of what reading asks us to do. Save an unfinished thought. Visit a related one. Notice a change in confidence or voice. Return with a slightly different understanding of the sentence we left.

They can preserve the shape of thinking at the moment it happens: the correction beside the claim, the doubt beside the confidence, the author’s raised eyebrow beside the official account. Used well, that proximity gives prose precision and intimacy.

Used too enthusiastically, parentheses allow a sentence to annex neighboring territories until the full stop becomes a ceremonial institution.

Good writing requires deciding which connections to reveal, in what order, and at what cost to the reader’s memory. Sometimes the route is a straight road. Sometimes it needs a branch. Sometimes the aside is the real destination and the main sentence is merely the bureaucratic corridor required to reach it.

A sentence is a walk through a thought. A parenthesis lets us step off the path together.

The words arrive in a line but underneath, everything interesting has edges.

We are starting to talk about artificial intelligence like a race. And the winner of that race will own the future. Which laboratory has the smartest model? Who leads on coding? How much better is the newest release at mathematics? How long before the next company overtakes it? Is this benchmark played out? Should I dismiss the current model as “benchmaxxed”?

I follow that race too. The frontier matters. Especially for my world. A model that can solve a previously intractable problem expands what we can imagine doing with computers. But there is another race, and its results may reach many more people, and change more companies: the race to make useful intelligence cheap enough, small enough, and practical enough to run wherever we need it.

That race has several engines. Models are getting better. Quantization stores them more efficiently. Distillation teaches smaller models some of what larger models can do. Hardware delivers more for the money, while better software makes more of the hardware already installed. I think this hints at intelligence being everywhere.

And I mean everywhere. On a laptop. On a desktop beside a researcher. Inside a small business. On the devices and software that surround us. Intelligence chips become like the 386 chip. Everywhere. I still remember being shocked one day to see a 386 chip inside a fridge I was fixing. I’m old enough to remember the humble 286 and the majesty of the turbo button. Yes you don’t need a 386 chip to control a fridge, but the chip is cheap enough and the development is faster, and one chip replaces a lot of control circuits leading to lower cost. This was driven by the cost to make the hardware falling. But what if the software costs less and gains in capability?

These changes mean that the computer you own can gain capabilities without being replaced. A new download can give the same machine a substantially better assistant. Will we get excited for iOS releases again?

The frontier shows us what is possible. What people can run at home tells us what is becoming ordinary.

That is what I mean by watching the tail: following useful capability as it reaches progressively smaller budgets and more widely available machines. I’m interested in a very simple question: what can a person do with the memory, money, electricity, and patience they actually have? When is a cheap and local model good enough for all of someone’s daily needs?

Yesterday’s frontier is ringing your doorbell.

There is already evidence that this migration happens on a surprisingly short timescale.

Epoch AI’s 2025 analysis estimates that leading open models small enough for a single high-end consumer GPU trail frontier models by roughly six to twelve months on the benchmarks it examined. Its estimates range from 6.3 months on the Artificial Analysis Intelligence Index to 12.4 months on LM Arena. These are fitted historical gaps, but you can see the trend. Epoch AI’s consumer-hardware analysis.

There are assumptions in this work. Epoch estimates what fits using four-bit weights, an 8,000-token context, and allowances for working memory. I think this is a good reflection of what resources many people can actually access. It combines benchmark results from several sources with estimates of which models fit on consumer hardware. And we know benchmark optimization can make the apparent gap smaller than the gap in everyday usefulness.

Still, this is a remarkable shift in perspective. A capability that attracts enormous attention when it first appears can become something an individual downloads within roughly a year. This might have been a better way to describe why we should be concerned and have a discussion about the rate of development. These local models can have a lot of guardrails obliterated. What would it mean when everyone has access to x level capability with the potential for reduced alignment?

Another historical comparison points in the same direction. Stanford’s 2025 AI Index reports that the smallest model exceeding 60% on MMLU shrank from the 540-billion-parameter PaLM in 2022 to the 3.8-billion-parameter Phi-3-mini in 2024. Now that’s a particular benchmark threshold, not showing the models have the same capabilities across all areas, but the reduction is striking. Stanford AI Index.

We should follow both curves: how far the best model advances, and how quickly a useful level of performance becomes accessible.

Quantization changes which machine can run the model

A language model contains billions of numerical parameters, usually called weights. Storing those numbers takes memory. Moving them through the computer takes bandwidth. Quantization represents those numbers with fewer bits. At the simplest level, moving from sixteen bits to four cuts the raw weight storage to a quarter. Real formats have additional overhead and there is a reason we need higher precision in some parts of a model.

Same structure, fewer shades.

The difficult engineering question is how to make that reduction without damaging the behavior we care about. Methods such as activation-aware weight quantization, or AWQ, use information about how the model behaves to protect sensitive parts of the computation. Compression works better when it accounts for which numerical errors matter. AWQ research paper.

We can see this in work from Jemin Lee and colleagues. Their evaluation reports Llama 3.1 70B at 140 GB of model storage and an OpenLLM Leaderboard v2 average of 40.64 in sixteen-bit precision. A four-bit AWQ version uses 35 GB and scores 40.39: one quarter of the reported storage, with a decline of 0.25 points. Lee et al., Table 2.

Less storage, similar measured performance. The chart selects the best reported result at each precision tier for each Llama 3.1 model. The horizontal axis is logarithmic: every step doubles storage. These are benchmark scores and reported model storage, rather than percentages of intelligence or total operating memory. Source: Lee et al.

This practical change is a hardware threshold. A model that previously needed much more memory can become a candidate for a smaller system. That changes who can experiment with it and which projects can afford it. Solar and batteries with local inference is as close to free tokens as you can get after depreciating the initial investment.

Other models are being released closer to the capabilities of home computers. Google’s April 2025 Gemma 3 QAT release reported that quantization-aware training reduced the memory needed for the 27B model’s weights from 54 GB to 14.1 GB, and the 12B model’s weights from 24 GB to 6.6 GB. Google explicitly described running the larger version on a 24 GB RTX 3090. Conversation memory still has to fit alongside the weights. Google’s Gemma 3 QAT announcement.

Quantization-aware training lets the model adapt to reduced numerical precision during training. Making the model easier to deploy is an explicit goal in its development. This can include additional quantization-aware training after the original model has been trained.

This is why the tail deserves attention. An improvement in how numbers are stored can change the accessible audience for a model as much as an improvement in its benchmark score.

Distillation gives the smaller model a better teacher

Distillation provides another route down the hardware ladder.

Instead of representing the same model with fewer bits, it trains a smaller model to reproduce useful behavior learned from a larger teacher. The teacher can provide predictions or examples of successful responses. The student learns from those examples without needing the teacher’s full architecture. The general approach is old, predating today’s language models. Hinton, Vinyals, and Dean on knowledge distillation.

DeepSeek made this especially visible with R1. Alongside its January 2025 release, it published six distilled models ranging from 1.5 billion to 70 billion parameters. Smaller descendants were part of the release itself. DeepSeek’s R1 announcement.

The team’s published results put its 32B Qwen-based distilled model at 72.6 on AIME 2024 and 57.2 on LiveCodeBench, compared with 63.6 and 53.8 for o1-mini in the same comparison. Now these are developer-reported results for specific mathematics and coding evaluations, so it’s not showing it’s better across all types of work. But they do show that a much smaller downloadable model can acquire substantial reasoning capability through training on a larger model’s outputs. DeepSeek’s evaluation tables.

Clearly distillation and quantization can be combined. Simply train a more capable small model; then reduce the memory required to run it. A 32-billion-parameter model represented at four bits would need about 16 GB for the raw weights alone, before format overhead and working memory. And hopefully it’s as good on the tasks you care about.

The larger implication is that improvements at the frontier can help create better teachers. Some of their behavior can then become training material for models designed for smaller machines. This is why we have the debate about distillation of frontier lab model outputs by other companies.

So you can see how the frontier can help improve the tail even while continuing to move ahead.

GLM-5.3-Flash shows how far the upper end has moved

GLM-5.3-Flash belongs at the expensive end of this story, but it’s a good example.

Z.AI reports approximately 320 billion total parameters, with 18 billion activated per token. Its mixture-of-experts design selects parts of the model for each step. And it has a combination of sparse and linear attention helping reduce the cost of handling long context. These are clever architectural approaches to reducing computation and memory pressure. Z.AI’s model documentation.

The distinction between total and active parameters is essential. I often see folks thinking this has reduced the size of the model. Think of a library: only a few books may be open on the desk, but the collection still needs shelves. Sparse computation does not make the need for the rest of the weights disappear. We still need to store them. Interestingly MOE and KV Cache to Cache communication might make for a new type of swarm in a model. I’ll post some results about this later.

That is where compression enters. The weights are available in a broad range of compressed versions. Official model card, Unsloth’s quantized releases.

Selected file sizes from Unsloth’s repository

The smallest listed version uses about 85.5% less storage than the BF16 reference. That brings a very large model into the range someone might investigate for a powerful personal workstation.

But a 93 GB file remains enormous for an ordinary laptop. A 120 GB file also leaves little nominal headroom on a 128 GB machine. Loading the weights is only the start: we need to add the operating system, conversation cache, and other tools, and the final system must be fast enough to use. No one would use a token/minute for example. But 13 tokens/second? With a bit of the aforementioned patience it might be very useful.

You should not think about how much intelligence is preserved from looking at the compressed size. One way to evaluate these quantized models is to compare their next-token choices with those of the reference model. This is a measure of fidelity; it won’t tell you whether a coding agent will finish a difficult job correctly. A long-context study found that four-bit degradation varied sharply by model, method, and task, with severe losses in some settings. Mekala et al..

GLM-5.3-Flash is important because a model of this scale is available for people to adapt to machines they control. Now it won’t be around forever and will be displaced, but I’m confident the process and market forces that we have discussed mean that this local capability will increase over time.

Better hardware meets better software

We need to be careful when we talk about hardware. The trend I want to discuss is how much useful computation a budget buys. The cost of the machine does not need to reduce to enable that, and with the projected RAM shortages, RAM prices may keep rising into 2027.

If you look at Epoch AI’s August 2026 analysis you see the performance per dollar of AI chips purchased from 2023 through 2025 improved by an average of 49% a year. (this is theoretical chip performance per dollar) So evidence of improving hardware economics, which is relevant for home-computer prices or local-model speed, because this used to be a leading indicator for improvements in the capability and economics of consumer grade hardware. Although perhaps the data centre demand is so high this might be more lagged than usual. Epoch AI’s hardware analysis.

Consumer hardware is responding to the need to hold larger models. AMD’s Ryzen AI Max+ 395 systems have configurations with up to 128 GB of unified memory, accessible to the CPU and GPU. Apple’s March 2025 Mac Studio announcement offered up to 512 GB in its M3 Ultra configurations, and other systems are appearing all the time. The demand is there, and interestingly talk of ‘pacing the frontier’ has made the more 2nd amendment/fiercely independent friends I know start to seriously build their own AI rigs. “You’ll take my model out of my cold dead hands”. Got to have AI in the prepper shelter. These folks are very into alignment removal on principle.

Memory capacity alone does not determine performance. Memory bandwidth, software support, and the model’s architecture affect whether the machine feels responsive. But more usable memory expands the set of models that can be considered at all.

Meanwhile, inference software makes existing machines more useful and allows non-experts to run this. One example is the llama.cpp project, which supports compressed models across CPUs and several GPU backends, including combinations of CPU and GPU execution. It is easy to get a working local application. Then, use another AI model or harness to optimize serving, or get the local model to help optimize its own serving! Reminds me of ye old days tuning BLAS and playing with ATLAS. llama.cpp project.

This is what makes the trajectory so interesting. A person can benefit from a better model, a better compression method, or better execution software without buying another computer. Someone buying a machine later can benefit from those improvements and better hardware together.

Progress reaches the home through several routes at once.

The release is becoming the start of a rapid adaptation process

Open weights allow other people to work on deployment immediately. They can create compressed versions, improve hardware support, fix conversion problems, and package the model for local applications.

GLM-5.3-Flash’s release-period histories show original-model updates and multiple quantized uploads arriving within days of one another. The histories also record subsequent fixes ;). Z.AI’s release history, Unsloth’s conversion history.

That gives us two different clocks to watch. One measures how long a capability takes to move from a frontier system into a model small enough for consumer hardware. The other measures how long a newly released model takes to acquire useful local versions. The first can be measured in months; the second can sometimes be measured in days, or disappear when smaller versions ship at launch.

The evidence supports fast diffusion. We can’t say that every capability reaches every hardware tier faster with each successive generation, but a stable gap would still mean continual improvement at home as the frontier advances.

What I would like to see beside every major model announcement is a simple record: what works on 16 GB, 32 GB, 64 GB, and 128 GB; how quickly it runs; how reliably it completes real tasks; and when those usable versions became available.

That would make the tail immediately visible.

Intelligence becomes part of the surroundings

The consequence I expect is a world in which useful machine intelligence is increasingly available close to the work.

A writer could keep an assistant beside years of notes. A small business could search and organize its records locally. A researcher could connect a model to papers, experimental logs, and analysis tools. A household could use software that helps make sense of personal documents without sending every question and intermediate result to a remote service. Maybe there will be an LLM in my fridge now, and LLM+chip is the new ‘transistor’ (which is a bit of a lame phrase, but you get what I mean).

The economics change when someone can keep a capable model running on equipment they control. Experimentation no longer necessarily carries a separate provider charge for each exchange. The costs become hardware, electricity, upkeep, and the time spent making the system useful. For occasional work, a cloud service may remain cheaper. For repeated local work where the model has sufficient ability, ownership becomes an increasingly credible option.

Local operation also allows more control over where data goes and which model version stays in use. That may really benefit security in control systems.

I expect many people and organizations to combine the two approaches. A local model will handle work it does reliably; a stronger remote model will be available for tasks that need it. Good routing will require tests and checks, because a model’s own confidence is an unreliable guide. However, there are some new approaches such as Jev and new local models like Laya.

As the local system improves, more work can stay close to the user. The practical measure of progress becomes the share of useful tasks it completes successfully, within an acceptable time and cost. That changes what deserves investment and depreciation times, and we might see local systems trading off against cloud spend.

This is why I keep coming back to the tail. The benchmark winner is a snapshot of the best result available under a particular setup. The spread of capability across ordinary machines tells us something about how widely people may be able to use it.

Intelligence will be everywhere, in the practical sense that more of our tools will be able to interpret, explain, draft, search, and act. Its quality will vary. Its reach will expand.

The next frontier breakthrough is worth watching, but so are the less visible moments when a person downloads a better model, opens it on the computer they already own, and discovers that something useful has become possible.

The tail might even begin to wag the dog in the future.

Sources checked September 20 2026.

Intelligence begins with a concession the universe makes to finite creatures: not everything that could happen does happen, and not everything that happens is independent of what happened before.

A mind cannot contain the world. It is made from a small part of the world, operates for a limited time, and encounters only fragments of what exists. Nevertheless, it can anticipate an approaching object, recognize a face in unfamiliar light, infer a mechanism from an experiment, or discover an intervention that changes the course of a disease. It succeeds not by reproducing reality in full, but by finding descriptions that preserve some of reality’s consequential relationships.

The central question is therefore not simply how matter becomes intelligent. It is how a finite physical system acquires an internal organization that allows it to exploit the organization of something larger than itself.

Three ideas provide a route through this question: compression, prediction, and control. Compression discovers what can be represented economically. Prediction uses that representation to infer what has not yet been observed. Control uses those inferences to influence what happens next. In biology, however, the relationship is more intricate than this sequence suggests. Living systems do not merely discover regularities. Through feedback, they also help create and maintain them.

That additional step changes both how we should understand intelligence and how we should attempt to treat disease.

A world small enough to learn

Consider the space of all possible images. Almost every arbitrary arrangement of pixels would be meaningless to us. Faces, forests, and rooms occupy highly constrained regions within this enormous space. A change in illumination does not independently rearrange every pixel of a face; it produces coordinated changes determined by shape, surface properties, and the direction of the light. The manifold hypothesis expresses one version of this idea: observations in a high-dimensional space may lie near a much lower-dimensional structure. It is a useful hypothesis with formal formulations, not a theorem that all meaningful data occupy one smooth surface.^[1]

For biology, this qualification matters. There need not be a single, fixed manifold of life. Different cell types, developmental stages, environments, and interventions may require different local descriptions. Discrete changes, continuous variation, noise, and history can coexist. Nor should every constraint be interpreted as geometric. A logical dependency, a conservation law, and a causal mechanism can each make data learnable without reducing neatly to the same mathematical object.

The deeper proposition is that the world contains dependencies that make some observations informative about others. For discrete variables, mutual information expresses this as

Knowing something about X reduces uncertainty about Y. When the variables are the past and future of a process, this becomes predictive information: the information in history that can help constrain what comes next.^[2]

A biased coin clarifies the distinction. Learning that it lands heads 99 percent of the time improves our predictions relative to assuming fairness. But, if successive tosses are independent, observing the last toss tells us nothing additional about the next. Nonuniformity is useful, but it is not the same as rich temporal structure. Likewise, an independent noise process can have a learnable distribution even when its next realization cannot be predicted from its history.

Intelligence requires neither a perfectly deterministic universe nor universal predictability. It requires regularities that are accessible, sufficiently persistent, and relevant to some task. A perfectly repetitive world would be easy to compress but offer little need for flexible intelligence. An entirely unstructured one would offer no foothold for learning. The interesting territory lies between them: enough order to support inference, enough variation to make inference valuable.

Learning what to forget

Learning and compression have a precise connection. A predictive model that assigns high probability to a sequence can be used to encode that sequence economically. In a minimum-description-length view, however, the accounting must include both the model and the data left unexplained by it. A table containing every observation is not a profound explanation simply because it reproduces its entries exactly. The achievement is to find a reusable description that makes additional observations less surprising.^[3][4]

This is the attraction of the shortest-program intuition associated with Kolmogorov complexity. A compact generative description can explain many apparently separate facts. But short description length is not, by itself, intelligence. The description must be discoverable, and useful consequences must be computable within the resources available. A tiny program that takes longer than an organism’s lifetime to answer a question offers little assistance to that organism.

Nor is indiscriminate compression desirable. A representation can become smaller by forgetting precisely the distinction on which a future decision depends. The information bottleneck makes the task-dependence explicit: compress the input while retaining information about a specified relevant variable. It is not an instruction to forget as much as possible. It is an instruction to forget selectively.^[5]

Imagine two patients whose measurements look almost identical at baseline, but whose responses to the same treatment differ sharply. An embedding that merges them may be excellent for reconstructing common laboratory values and disastrous for selecting therapy. The distinguishing feature may explain very little variation in the dataset while explaining a great deal of variation in the consequence of an intervention.

This is why the familiar demand to “retain the signal and remove the noise” is incomplete. Signal for what? Noise relative to which decision? A rare event may be statistically negligible and medically decisive. A feature irrelevant to tomorrow’s prediction may be indispensable to a prediction over ten years.

A useful representation is therefore a commitment about future questions. To choose what a model preserves is, implicitly, to choose the kinds of ignorance it will be allowed to have.

How regularity becomes intelligence

The existence of learnable structure does not explain how a system comes to exploit it. There must also be a process that retains successful ways of responding to the world and modifies unsuccessful ones.

Evolution supplies one such process. When environmental cues can guide beneficial responses, the machinery that detects and uses those cues can acquire a fitness advantage. Formal models of the fitness value of information connect this advantage to both the informativeness of the cue and the consequences of acting on it. Information is not valuable merely because it is information; its value depends on the available responses and the environment in which they are used.^[6]

This suggests a progression, rather than a magical boundary. An inherited response can exploit a regularity without learning it during an individual lifetime. A plastic system can adjust its responses through experience. A system with memory can distinguish situations that look identical now but have different histories. A system that evaluates alternative action sequences can respond to circumstances it has never encountered in exactly that form.

These are not necessarily steps on a single evolutionary ladder. They are increasingly demanding ways of coupling information to action.

In artificial learning, optimization can similarly produce reusable internal structure. Yet a randomly initialized network is not wholly without structure: its architecture, inputs, and training procedure already constrain what it can readily learn. Training does not guarantee that it discovers the true causes of its observations. It may acquire useful abstractions, shortcuts, memorized exceptions, or some mixture. In a particularly revealing small-scale case, mechanistic analysis of transformers learning modular arithmetic identified a generalizing algorithm built from Fourier-related computations, rather than merely a collection of remembered answers.^[7]

Such results make emergence less mysterious without making it trivial. Some apparent capability jumps arise from discontinuous scoring rules: gradual improvements can abruptly cross an exact-answer threshold. Other changes involve the development and increasing use of structured computational mechanisms. The scientific question is not whether a model has crossed a rhetorically satisfying boundary into “understanding,” but what internal organization now supports behavior that was previously unavailable.^[8][7]

Scale enlarges the space of structures a model might acquire, but capacity is only a possibility. The data, learning objective, optimization process, and available computation determine which possibilities become usable capabilities.

Memory, analogy, generalization, and reasoning can then be understood as related uses of representations, without pretending they are identical operations. Memory preserves useful distinctions. Generalization carries a relation into a new case. Analogy maps relational structure between settings. Reasoning performs transformations whose intermediate results need not have appeared in the original observations.

A world model need not be a literal simulation sitting inside the agent. It can be implicit in a policy or distributed across computational mechanisms. Under specified controlled-Markov-process assumptions, Richens and colleagues show that sufficiently general competence across multistep goals entails predictive information about the environment that can be extracted from the policy. This is a formal result about a defined class of agents and tasks, not a proof that every intelligent behavior requires a complete internal replica of reality.^[9]

The resulting picture is an account of adaptive competence, not a completed theory of consciousness. A thermostat closes a feedback loop; that alone does not make it a mind. Flexible intelligence concerns the breadth, adaptability, and compositional power with which such loops can be organized.

Before brains, there was regulation

The connection between information and control is already visible in systems without nervous tissue.

Bacterial chemotaxis provides a particularly clear example. A bacterium does not need a pictorial map of its surroundings to adjust movement in response to chemical conditions. Its signaling machinery can retain a biochemical trace of recent stimulation. Work on robust adaptation in bacterial chemotaxis identified integral feedback as a mechanism through which a signaling output can return to its previous level despite a sustained change in input.^[10]

Integral feedback is a particularly clear case, not a claim that every homeostatic circuit implements exact integration or achieves perfect adaptation.

The important point is not that the bacterium secretly performs conscious arithmetic. It is that chemical dynamics can implement a mathematical operation. A variable that accumulates a discrepancy over time becomes a physical memory of that discrepancy. The computation exists in the organization of the reactions, not in a separate symbolic description of them.

This is more than an analogy imposed after the fact. Synthetic-biology experiments have constructed biomolecular integral-feedback controllers in living cells and demonstrated their regulation and adaptation properties. Control-theoretic organization can be engineered into chemistry.^[11]

The embodiment matters. Information is never processed by an abstract diagram alone. It must be stored and transformed in physical machinery. Landauer’s principle and experimental tests of it establish a precise connection between logically irreversible erasure and thermodynamic cost under specified conditions. They do not establish that every inference has the same minimum energy cost, or that Shannon entropy, thermodynamic entropy, and the “free energy” of an inference objective are interchangeable quantities.^[12]

We do not need to blur those distinctions to see the deeper continuity. A biochemical controller, a nervous system, and an artificial agent can all preserve selected consequences of the past and use them to shape subsequent behavior. Their implementations and capabilities differ enormously. Their shared problem is to make limited internal state useful in a changing external world.

The manifold is partly something life maintains

Here the argument turns back on its starting point.

We began by saying that intelligence is possible because observations occupy a constrained portion of the space of possibilities. In living systems, some of those constraints are actively maintained. If a feedback controller keeps an output near a reference value despite disturbances, then observations of that output will cluster near the reference. The regularity is not merely an external fact waiting to be discovered. It is partly the result of regulation.^[11]

This has a troublesome consequence for biological learning. A variable may vary little because it matters greatly.

Suppose a controller continually corrects deviations in an essential physiological quantity. A dataset collected during successful regulation will contain little variation in that quantity, even while the hidden effort required to maintain it changes substantially. A representation optimized only to capture the largest observed variations might neglect the regulated output, the compensatory machinery, or both.

The resulting observational simplicity can conceal causal importance. To see what the system depends on, we may need to examine its response to disturbance rather than its appearance at rest. The observational manifold is conditional on the existing regulatory regime. An intervention can alter that regime, exposing directions of variation that were nearly absent during training.

This also separates three objects that are too easily conflated. The geometry of observed states describes where a system has been found. Its dynamics describe how it moves. Its control structure describes how available interventions can alter that movement. None can simply be substituted for the others.

A low-dimensional embedding is not automatically a causal model. A causal model is not automatically sufficient for safe control. Even a two-dimensional system cannot be fully steered by an input that moves only one coordinate if the dynamics never transmit that influence to the other. Conversely, a system with vast microscopic complexity may permit useful regulation through a small number of aggregate variables.

The number of variables needed to describe a dataset, the number of hidden states needed to predict a response, and the number of intervention points needed to change an outcome are different quantities.

In control-theoretic language, we must ask about both observability—whether relevant hidden states can be inferred from measurements—and reachability—whether the available inputs can move the system toward a desired outcome under its constraints.

An atlas of biology tells us which states have been observed. Medicine requires an additional answer: which desirable states are reachable, by what paths, and at what cost?

Why pushing biology often produces a push back

A deliberately simple model makes the problem concrete. Let y be a regulated biological output, c an unobserved compensatory drive, and u a treatment that suppresses the output:

where a and k are positive and y⋆ is the controller’s reference level. This is an illustrative local model, not a claim that a particular disease obeys these equations.

Increase u, and initially y falls. But that fall creates a discrepancy relative to y⋆. The compensatory state c accumulates until it cancels the sustained suppression. At equilibrium,

The observed output has returned to its previous value. The internal state has not.

A terminal measurement of y alone might suggest that the treatment did nothing. A longitudinal record would reveal an initial response followed by recovery. Measuring c would show the compensation. Withdrawing treatment from the adapted state would initially drive the output upward because the compensatory drive would still be elevated.

Within this model, increasing the dose does not produce a different sustained output while integral compensation remains operative and unsaturated. It produces a different amount of hidden compensation. Merely lowering the feedback gain changes the speed of adaptation, not the final reference level. Durable displacement requires changing something more fundamental: the reference, the controller architecture, the available compensatory capacity, or the relationship between that controller and the output we care about.

This gives a precise form to the driver-and-constraint view of therapy. One intervention moves the system in a desirable direction; another prevents a specified compensatory or self-reinforcing process from undoing the change.

But it is a conditional design principle, not a universal two-drug theorem. A single intervention may alter the reference itself, remove an indispensable driver, or disable several compensatory routes at once. Two interventions may be redundant or insufficient. Nor is all disease a stable attractor: irreversible tissue loss, persistent external injury, and progressive failure require different descriptions. The therapeutic task is to establish which dynamics actually apply, rather than assume that every pathology is an equilibrium waiting to be shifted.

Lipid lowering supplies a concrete example of the value of understanding compensation. Atorvastatin can increase circulating PCSK9, which promotes degradation of hepatic LDL receptors. PCSK9 inhibition provides a complementary intervention; in FOURIER, adding evolocumab to background statin therapy reduced LDL cholesterol and cardiovascular events. This supports the usefulness of complementary control points, without demonstrating that all therapeutic combinations work by the same feedback mechanism.^[13][14]

Most importantly, the purpose of a second intervention should not be to abolish homeostasis indiscriminately. The same regulatory machinery may protect other functions. A successful treatment must distinguish pathological compensation from the robustness that keeps the patient alive.

A model should preserve the consequences of intervention

These considerations change what “understanding biology” should mean for a model built to support medicine.

We do not necessarily need a miniature biochemical replica of the patient. We need a representation that preserves the distinctions required to choose between relevant actions. Work on control-oriented representation learning makes an analogous distinction: states can be grouped according to their consequences for rewards and future transitions, rather than according to whether they permit detailed reconstruction of the original observations.^[15]

For medicine, the requirement must be stronger than observational resemblance. Let hₜ denote the history of measurements and treatments, and let zₜ = φ(hₜ) be a compressed representation. An appropriate aspiration is

for the intervention sequences u, outcomes, and time horizons relevant to the decision. Here do(u) means imposing those interventions, rather than merely observing that they occurred.

This is a proposed criterion for a useful medical representation, not a guarantee that a dataset allows us to learn one. Two mechanisms can agree on everything observed so far and disagree on the response to an untested treatment. No amount of compression can manufacture the missing causal evidence.

The criterion also explains why history belongs in the state. In the illustrative feedback system, the same current output can conceal different levels of compensation. A snapshot that merges those conditions loses information needed to predict withdrawal or rechallenge. Where the internal state remains uncertain, the representation may need to preserve a distribution over possibilities rather than one confident estimate.

A model can be mechanistically incomplete and still be useful for a bounded control problem. If it reliably predicts the consequences of modest interventions in a validated operating region, repeated measurement and replanning may compensate for some of its omissions. But feedback is not a license for arbitrary model error. Delays, unobserved hazards, and irreversible damage may prevent correction before harm occurs.

The appropriate ambition is therefore neither exhaustive realism nor convenient simplification. It is decision-relevant fidelity, with an explicit boundary of validity.

A model that knows when its representation is no longer adequate may be more valuable than one that reconstructs familiar data beautifully and extrapolates without hesitation.

An intervention can also be a question

Once a model is understood as incomplete, action acquires another purpose. An intervention changes the system, but it can also reveal which system we are dealing with.

This is dual control in its technical sense: actions have both consequences for the controlled process and consequences for what will be learned about its uncertain dynamics. It is distinct from dual intervention. Two drugs concern the number and roles of intervention points. Dual control concerns the interaction between acting and learning, even when only one intervention is available.^[16]

Consider a proposed research program in fibrosis. Experimental work has shown that matrix stiffening can promote fibroblast activation and matrix production, creating a reinforcing relationship between cells and their mechanical environment. Liu and colleagues demonstrated this feedback using lung-injury and cultured-fibroblast experiments. It provides a mechanistic foothold, not proof that any particular two-intervention regimen will reverse human fibrosis.^[17]

Now imagine two competing models of a patient-relevant experimental system. In one, continuing injury supplies the dominant drive. In the other, the altered matrix has become sufficiently self-reinforcing that the original injury is no longer necessary to sustain activity. Both models might explain the same baseline expression profile. They could make different predictions about the speed of response after reducing injury, the persistence of fibroblast activation, and the effects of changing the mechanical environment.

An informative experimental perturbation would be selected partly because these predicted trajectories diverge. Measurements would follow the early response, adaptation, recovery, and, where appropriate, withdrawal. The experiment would ask not only whether a marker decreases, but which explanation survives the pattern of change.

More time points do not automatically identify the mechanism. Sampling can miss the relevant timescale; hidden variables can leave several explanations indistinguishable; an intervention can have unrecognized effects. The remedy is not “longitudinal data” as a slogan. It is the joint design of perturbations, observations, and competing models.

Nor is information gain automatically a sufficient reason to perturb a patient. Clinical use requires ethical justification, safety constraints, and an appropriate evidentiary setting. Much exploratory learning belongs first in carefully chosen experimental systems. Even there, the most useful information is information that changes a subsequent decision, not merely information that makes a model more elaborate.

The central principle remains: the best next action need not be the one that looks best under our current favorite explanation. It may be the action that improves the system while making future decisions less dependent on an explanation that could be wrong.

A world model of medicine needs a curriculum of consequences

This leads to a different vision of a medical world model.

It should not be judged primarily by how much biomedical information it can store, or by whether every modality can be placed in a common embedding. Its purpose is to preserve useful relationships between observations, latent states, interventions, and outcomes across the settings in which we expect to use it.

That expectation must be made concrete through tasks. What must the model predict within a cell? Which perturbation effects must it carry from cells into tissues? Which tissue changes must it connect to organ function? Which links can support a prediction about clinical benefit, and which remain conjectural?

A sensible curriculum would include within-scale dynamics, adjacent-scale links, and selected longer-range predictions. It would also test whether independently useful pieces remain reliable when composed. A cell-state model and an organ-function model do not become a validated cell-to-organ model merely because their embeddings can be aligned.

Population generalization is a separate challenge from physical scale. Predicting from cellular measurements to an individual’s organ response is one task. Establishing that the relationship transports across patients with different histories, exposures, and comorbidities is another. Moving from a cell to a tissue and moving from one person to a population should not be collapsed into the same notion of “going up a level.”

The tasks must also expose the distinctions that observational compression would otherwise erase. They should challenge the model with timing, treatment order, adaptation, rebound, combinations, and recovery. A representation that predicts untreated trajectories but fails after intervention has learned something real, but not yet the thing required for therapeutic control.

This is a proposal for how to build and assess such models, not a claim that one universal representation must exist. Different horizons and intervention classes may require different state descriptions. The useful architecture may be a collection of overlapping, partially compatible models, with explicit statements about when their conclusions can be combined.

The aspiration is not omniscience. It is a growing domain of reliable counterfactual competence: knowing what is likely to happen if we act, recognizing when the evidence is insufficient, and choosing observations that can reduce the uncertainty that matters.

Intelligence closes the loop

Read together, the traditions of Shannon and Wiener suggest a larger picture. Information theory asks what can be distinguished, encoded, and inferred. Cybernetics asks what those distinctions can accomplish when they are placed inside a feedback loop.

A representation earns its significance through what it enables. It can allow an organism to preserve a viable condition, a learner to recognize a new instance of a familiar relation, or a scientist to choose an experiment that separates explanations. Information becomes consequential when differences in the model produce appropriate differences in action.

Biology adds a final complication: the object of our intervention is already full of regulatory activity. We are not usually acting on passive matter that will remain wherever it is pushed. We are entering processes that preserve, compensate, repair, remember, and sometimes sustain pathological conditions. Effective medicine must learn the organization of those responses, not merely catalog the components through which they occur.

This is why compression, prediction, and control should be understood as a loop rather than a ladder. What we want to control determines what we must predict. What we must predict determines what we should preserve. What we preserve determines which actions appear possible. And the actions we take change both the world and the evidence from which the next representation will be learned.

The deepest connection is therefore not simply that intelligence uses models and medicine needs better models. It is that both confront the same problem: how to act effectively in a system whose full complexity exceeds what can be represented.

Intelligence does not require possessing the whole world. It requires learning which differences matter, preserving them in a usable form, and discovering when that form has become inadequate.

For medicine, the corresponding ambition is not to know every molecular detail before intervening. It is to know enough of the right relationships to change a trajectory—and enough about the limits of that knowledge to learn from what happens next.

References

[1] Fefferman, C., Mitter, S., and Narayanan, H. (2016). “Testing the Manifold Hypothesis.” Journal of the American Mathematical Society. Author manuscript: arXiv:1310.0425.

[2] Bialek, W., Nemenman, I., and Tishby, N. (2001). “Predictability, Complexity, and Learning.” Neural Computation, 13, 2409–2463. Author manuscript: arXiv:physics/0007070.

[3] Rissanen, J. (1978). “Modeling by shortest data description.” Automatica, 14(5), 465–471. Publication record: IBM Research, “Modeling by shortest data description.”

[4] Delétang, G., et al. (2023). “Language Modeling Is Compression.” arXiv:2309.10668.

[5] Tishby, N., Pereira, F. C., and Bialek, W. (1999; preprint posted 2000). “The information bottleneck method.” arXiv:physics/0004057.

[6] Donaldson-Matasci, M. C., Bergstrom, C. T., and Lachmann, M. (2010). “The fitness value of information.” Oikos, 119(2), 219–230. DOI: 10.1111/j.1600-0706.2009.17781.x.

[7] Nanda, N., Chan, L., Lieberum, T., Smith, J., and Steinhardt, J. (2023). “Progress measures for grokking via mechanistic interpretability.” arXiv:2301.05217.

[8] Schaeffer, R., Miranda, B., and Koyejo, S. (2023). “Are Emergent Abilities of Large Language Models a Mirage?” arXiv:2304.15004.

[9] Richens, J., Abel, D., Bellot, A., and Everitt, T. (2025). “General agents contain world models.” arXiv:2506.01622, version 5. Earlier versions were titled “General agents need world models.”

[10] Yi, T.-M., Huang, Y., Simon, M. I., and Doyle, J. (2000). “Robust perfect adaptation in bacterial chemotaxis through integral feedback control.” Proceedings of the National Academy of Sciences, 97(9), 4649–4653. DOI: 10.1073/pnas.97.9.4649.

[11] Aoki, S. K., et al. (2019). “A universal biomolecular integral feedback controller for robust perfect adaptation.” Nature, 570, 533–537. DOI: 10.1038/s41586-019-1321-1.

[12] Bérut, A., et al. (2012). “Experimental verification of Landauer’s principle linking information and thermodynamics.” Nature, 483, 187–189. DOI: 10.1038/nature10872.

[13] Careskey, H. E., et al. (2008). “Atorvastatin increases human serum levels of proprotein convertase subtilisin/kexin type 9.” Journal of Lipid Research, 49(2), 394–398. PMID: 18033751.

[14] Sabatine, M. S., et al. (2017). “Evolocumab and Clinical Outcomes in Patients with Cardiovascular Disease.” New England Journal of Medicine, 376, 1713–1722. DOI: 10.1056/NEJMoa1615664.

[15] Zhang, A., McAllister, R., Calandra, R., Gal, Y., and Levine, S. (2021). “Learning Invariant Representations for Reinforcement Learning without Reconstruction.” ICLR. Author manuscript: arXiv:2006.10742.

[16] Klenske, E. D., and Hennig, P. (2016). “Dual Control for Approximate Bayesian Reinforcement Learning.” Journal of Machine Learning Research, 17(127), 1–30.

[17] Liu, F., et al. (2010). “Feedback amplification of fibrosis through matrix stiffening and COX-2 suppression.” Journal of Cell Biology, 190(4), 693–706. DOI: 10.1083/jcb.201004082.