I watched certainly one of our engineers clarify the identical authentication sample to Claude Code for the fourth time final month. Not as a result of he forgot he’d defined it. As a result of the instrument forgot.
Each session, from scratch. “We use JWT validation on the gateway layer, not in particular person providers.” He’d stated it three days in the past. And the week earlier than. And each time he began a brand new session for the previous six months. Every time, the AI nodded alongside, adopted the directions completely, after which forgot all the pieces the second the session ended.
I stored fascinated about this, as a result of it felt just like the form of downside that ought to already be solved. It’s 2026. These fashions are genuinely succesful. They’ll cause about advanced codebases, debug delicate race circumstances, write stable checks. And but they function with what I can solely describe as aggressive amnesia — a pathological lack of ability to retain something previous the present session.
The autocomplete excuse
This made sense when AI coding instruments had been autocomplete engines. Copilot circa 2022 was finishing single traces of code. The context was one file. Why wouldn’t it want reminiscence? You kind, it suggests, you tab. Session reminiscence is irrelevant.
However that’s not what these instruments do anymore. We ask them to construct options throughout a number of recordsdata. Debug manufacturing points that require understanding system structure. Onboard new engineers to unfamiliar codebases. Run autonomously on GitHub points. And each single time, they begin from zero.
I hold coming again to this analogy: think about hiring an excellent contractor who reveals up each morning with complete amnesia. They’ll code. They’re quick. However daily you spend the primary hour explaining the challenge, the crew conventions, the choices you’ve already made, the errors you’ve already discovered from. And the following morning? Identical factor.
That’s the expertise proper now. For each crew. With each instrument.
The issues that by no means make it into code
Right here’s what bugs me most. The stuff the AI retains forgetting isn’t within the code. It’s the stuff that lives between the traces:
Why we selected Postgres over DynamoDB. (Efficiency for our question patterns, but additionally as a result of the crew has deep Postgres experience and 0 DynamoDB expertise.)
Why the notification service is a monolith module and never a microservice. (We tried microservices. It was a catastrophe. We reverted in Q3 and no person documented why.)
That the billing pipeline has a recognized edge case the place occasions get silently dropped beneath excessive load. (Two engineers learn about this. One in every of them simply gave discover.)
None of that is in a file the AI can scan. A few of it was in a Slack thread from eight months in the past. Most of it’s in individuals’s heads. And it’s precisely the form of context that determines whether or not the AI’s output is right or subtly, dangerously improper.
“Simply put it in a config file”
I do know what you’re considering, as a result of I assumed it too. CLAUDE.md. .cursorrules. System prompts. Simply write all of it down in a file and level the AI at it.
We tried. Everybody tries. And it really works — for about three weeks, till the file is stale and no person updates it as a result of updating a config file is upkeep work that doesn’t ship options. The one who wrote the unique file has moved on to different issues. New choices get made in conversations that by no means make it to the file. The file turns into a historic artifact that roughly corresponds to what the crew believed in some unspecified time in the future up to now.
It’s the wiki downside yet again. Somebody creates it with good intentions. It begins decaying instantly. Inside six months, builders actively mistrust it as a result of they’ve been burned by following outdated data.
“Simply use a much bigger context window”
The opposite in style reply. 200K tokens wasn’t sufficient, so now now we have 1M. Simply stuff all the pieces in.
I’ve spent numerous time fascinated about this, and I feel it essentially misunderstands the issue. An even bigger context window provides you extra room for the present session. It doesn’t provide you with reminiscence. It doesn’t let you know why the crew made a selected architectural choice final quarter. It doesn’t know in regards to the manufacturing incident that formed how the crew thinks about error dealing with. It doesn’t know that Sarah is the one one that understands the reconciliation pipeline.
An even bigger window is a much bigger scratch pad. The scratch pad nonetheless will get erased when the session ends. You haven’t solved amnesia — you’ve given the amnesiac a bigger pocket book that additionally will get burned each evening.
“Simply add extra brokers”
This one is more moderen and it’s the one which will get me. The reply to “the AI doesn’t know sufficient” is outwardly “add extra AIs that additionally don’t know sufficient, however give each a narrower job.”
A assessment agent. A testing agent. A deployment agent. Fifteen specialised brokers, every with hardcoded directions for one activity. Somebody on the crew wrote these directions. Somebody has to keep up them. When the assessment requirements change, somebody updates the assessment agent. When the testing framework adjustments, somebody updates the check agent. It’s the config file downside at a better degree of abstraction, with extra transferring elements.
And what none of these brokers know? Something about your group. They know what somebody hardcoded into their directions. They don’t know what the crew discovered final week.
The query that retains nagging me
Right here’s what I hold coming again to: what if the AI simply… remembered?
Not the uncooked dialog transcript. That’s noise. However the precise information — the choices, the patterns, the errors, the conventions — extracted from conversations and accessible in future periods. Not only for the engineer who had the dialog, however for the entire crew.
An engineer explains why we use occasion sourcing for the audit system. That clarification turns into a structured information merchandise — accessible to each different engineer, in each future session, with out anybody sustaining a file.
Somebody discovers a delicate coupling between two providers whereas debugging. That discovery will get captured. Subsequent time somebody touches both service, the AI already is aware of in regards to the coupling. Not as a result of somebody remembered to doc it, however as a result of the system was listening when the information was created.
The AI that helped you debug a billing concern on Tuesday begins your Thursday session already figuring out what you found. The brand new engineer who joins subsequent month has an AI that is aware of all the pieces the crew has discovered up to now 12 months — from day one.
I take into consideration this quite a bit as a result of it adjustments what the instrument essentially is. It stops being a coding assistant and begins being organizational reminiscence. Not a wiki that somebody has to keep up. A residing information base that grows as a result of individuals use the instrument.
The place this goes
The AI coding instrument market is about to separate. On one aspect: instruments that assist particular person engineers write code quicker. These are commoditizing. The fashions get cheaper each quarter. The wrappers get thinner. There’s no sturdy benefit.
On the opposite aspect: instruments that make a corporation’s collective intelligence accessible to each engineer, each session, completely. These don’t exist but. Not likely. Not in a manner that really works.
I’ve been spending the final 12 months fascinated about what the second class seems to be like. The way you construct it. What the structure must be. The place the business’s assumptions are improper.
All of it began from this one remark: your AI instrument has amnesia, and no person appears to suppose that’s an issue value fixing.
I feel it’s the one downside value fixing.