The half-life of AI knowledge: what expires and what compounds
In October 2025, OpenAI launched Agent Builder, a visual canvas for building AI agents. Eight months later it announced the shutdown. If you spent a weekend getting good at it, or at a framework’s abstractions, or at the exact phrasing that made last year’s model behave, you know the feeling.
Nobody told you your weekend expired. It just did.
The pace of AI isn’t the real problem. The problem is what the pace does to everything you learn: it makes it feel disposable. Why invest in anything that will be stale by spring?
But not everything you learn about AI expires at the same rate. Some of it hasn’t expired since 1847.
_(Disclosure up front: we’re building a learning tool around these ideas. It gets one paragraph near the end.)_
A radioactive sample loses half its atoms in a fixed amount of time. You can’t say which atom goes next, but you can say how long until half are gone.
The economist Fritz Machlup is usually credited with applying the idea to knowledge in 1962, asking how long the things people paid to learn stayed useful.
It’s a metaphor, not a law. But it makes one thing obvious: AI knowledge isn’t one thing that decays at one rate. It’s a stack of layers, and each layer runs on its own curve.
The stack, from fastest decay to slowest. The curves are a thought experiment; the dates below them are real.
[](https://substackcdn.com/image/fetch/$s_!OdIV!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F70474b2b-1dd2-47ac-90b2-6497ed3a66ef_1456x950.png)
**Tools and products: months to a couple of years.** It’s not just startup churn. LangChain 1.0 moved `LLMChain`, its original building block, into a legacy package. Flowise, an open-source builder for LLM apps, reached end of life in August as coding agents outgrew its drag-and-drop workflows. Different owners, different reasons, same shelf life. The tools that last, like NumPy and PyTorch, wrap stable ideas: arrays, automatic differentiation.
**Techniques and recipes: a few years.** In 2022, “Let’s think step by step” was the prompt trick everyone copied. In 2023, a Google DeepMind paper had a model search for better phrasings, and “Take a deep breath and work on this problem step-by-step” won on grade-school math for PaLM 2: 80.2% against 71.8% for the older trick. Today, OpenAI’s guide for its reasoning models advises against chain-of-thought prompts altogether, because the models already reason internally. The phrasing expired. The idea under it, that models do better when they spend tokens reasoning, got trained into the models themselves.
**Architectures: five to ten years.** The transformer was introduced in 2017 in “Attention Is All You Need” and still underpins nearly every frontier model nine years later. That’s a long run, not a permanent one. LSTMs, invented in 1997, carried most of language modeling from about 2014 until the transformer displaced them a few years later. Hybrids that swap most attention layers for cheaper state-space layers have been shipping since AI21’s Jamba in 2024. Notice what survives the swap, though: attention itself, letting a model weigh which inputs matter, dates to 2014 and will likely outlive the transformer. Even the hybrids keep some.
**Core ideas: decades.** Learning from examples instead of writing rules: Frank Rosenblatt’s perceptron, 1958. Shrinking a measure of error by following its slope downhill: gradient descent, which Augustin-Louis Cauchy described in 1847 to compute orbits. Working out how much each weight contributed to the error, so hidden layers can learn their own features: backpropagation, popularized by Rumelhart, Hinton and Williams in 1986 (the math has older roots) in a paper titled, fittingly, “Learning representations by back-propagating errors”. The model releases you’ll read about this year are these ideas again, at larger scale and in new arrangements.
The fair objection: “core” is easy to call in hindsight. In 1985, expert systems looked core and neural networks looked like a dead end. So there’s always an open frontier of exploration and new ideas keep being added, but most of the known core foundation keeps being useful over long periods.
Each core idea makes the next one cheaper to learn. Once you see training as shrinking a measure of error, fine-tuning and distillation stop being new topics: they’re the same move pointed at different targets, new data or a teacher model’s outputs. RLHF adds one idea on top: when you can’t write down the right answer, train a model to score answers, then push toward higher scores.
It also changes how you read the news. Say a lab ships a model with ten times the context. If you understand attention, you know its compute grows with the square of the context and its memory with every token, so you open the announcement with one question: what did they do about attention? You find the paragraph, see what was traded away, and within minutes the release reads as a variation on something you already know. If you only know the tool, every announcement has you feeling as if starting from scratch.
We drew this as a map (click to explore it): the ideas behind today’s AI, each one linked to what it rests on and what it unlocks.
**Explore the full atlas →** Pick an idea you’ve been nodding along to and trace it down to what it rests on.
You can’t know an idea’s half-life in advance. What you can spot is which ideas everything else rests on. Education researchers call them threshold concepts (Meyer and Land, 2003): hard to cross, but once crossed they change how you see a whole field, and they rarely get unlearned.
Train there. Neural nets, learned representations, attention: each one turns dozens of later topics into variations on something you already know. Pick up tools as work demands; they’ll land faster on roots you already have.
Next time something new ships, paste this into any LLM:
``` I want to understand [X]. Before explaining it: 1. Name the threshold concepts (from Meyer and Land) X rests on, most fundamental first. 2. Pick the deepest one I might not really understand and ask me one question that would reveal whether I do. Wait for my answer. 3. If I'm shaky, teach me that idea in under ten minutes with one concrete example. If I'm solid, explain X as a variation on what I already know and go deeper into details / expand into a strategic idea that builds on top. ```
We are building Zairai, an AI coach built on these and other top principles from education science, it finds where your understanding stops, and gives you high quality short sessions on technical AI, from your level to the frontier. **Join the waitlist →**