The Orchestrator’s Tax

The Incident Was Not Actually About Parallelism

4 subagents had been launched in a single wave. Three had clear runtimes:
roughly twelve minutes, 5 and a half minutes, and 7 minutes. A fourth
was nonetheless operating. Checked out a technique, that already justified the delegation.
Three duties ran concurrently, so wall-clock time was round twelve minutes
as a substitute of one thing nearer to 25 if the work had been
serialized.

However velocity was the seen byproduct, not the fascinating half. What I used to be
looking for was the associated fee, and my first intuition was that it needed to be the
duplicated effort of delegation itself: each subagent studying recordsdata,
reconstructing context, understanding the duty independently.

That was not the place the largest shock turned up.

At one level in the course of the work, the orchestrator recommended checking on the
operating brokers. It was a small, throwaway immediate: “examine on the brokers.” I
adopted it. As a substitute of a light-weight abstract, the device it used pulled again
the total uncooked transcript of a background agent: tens of hundreds of tokens of
JSONL, intermediate reasoning, and power output, imported wholesale into the
predominant thread. Then it occurred once more, for a second standing examine.

There is a crucial caveat right here. The declare that this polling behaviour
price greater than the duplication tax of 4 brokers was the orchestrator grading
its personal mistake. I did not have actual per-call token accounting, so deal with that
rating because the orchestrator’s account, not a measured truth. The reliable
half is narrower: the transcript dumps have been actual, the wall-clock timings have been
actual, and the status-check path clearly launched a big, avoidable price.
Whether or not it was the one largest price stays a speculation till the tooling
can instrument it correctly.

What mattered extra was that I would discovered a price I hadn’t been on the lookout for.

The Prices Had been Not All of the Similar

As soon as I ended treating the session as one lump “subagent price,” the
image broke into items that did not belong collectively.

First, two of the 4 subagents have been working in the identical space of the
response pipeline. Completely different duties, completely different recordsdata, however each needed to
perceive the identical structure, the identical testing conventions, and far of
the identical surrounding code earlier than both may start. Every paid that
orientation price independently. That is not an argument in opposition to delegation.
It is an argument that the work had been cut up too finely.

Second, one agent ran git stash and git stash pop whereas sibling brokers
have been writing elsewhere in the identical tree. Nothing broke, however the threat was
structural, as a result of repository-wide operations are completely affordable in a
single-threaded session and turn out to be a lot tougher to justify the second a number of
writers are energetic without delay.

By now I had a rising record of culprits: standing polling, duplicated
orientation, unsafe git operations. I discovered myself attempting to rank them. Which
one price essentially the most? The longer I attempted to reply that, the much less satisfied I
grew to become that rating them was the fitting query in any respect.

The Scarce Useful resource Is the Orchestrator’s Working Reminiscence

The transcript-polling incident saved bothering me for a purpose that had
nothing to do with token price. A token invoice is one-time, you pay it and it is
over. What occurred right here was completely different. The uncooked transcript stayed within the
orchestrator’s context after the device name accomplished, and each flip after
that carried it ahead, whether or not or not it was nonetheless helpful.

That was the second I spotted I had been treating two very completely different
sorts of price as if they have been the identical. Tokens are spent as soon as. Context
shapes each choice that follows. I wasn’t merely token
consumption anymore. I used to be trying on the high quality of the orchestrator’s
working reminiscence.

That realization was carrying two separate concepts, and I need to pull them aside
somewhat than allow them to blur collectively. The primary is what I simply described.
Air pollution left in context taxes each later flip. The second isn’t about
operating out of house in any respect. The extra that is sitting in context, competing for
consideration, the tougher it will get for a mannequin to select what issues proper now,
even with loads of room nonetheless free. A much bigger context window does not repair that.
It simply provides the noise extra room to pile up earlier than anybody notices.

Context home windows are solely going to get greater, that is a given. What issues
isn’t how a lot room there may be, however how a lot of what is sitting in that room is
well worth the mannequin’s consideration. That is the true drawback subagents want to unravel,
in the event that they’re used proper.

Seen this manner, the orchestrator is the one a part of the system that
accumulates understanding throughout a protracted session. It remembers why a design
choice was made, carries ahead architectural constraints, and is aware of which
trade-offs have already been mentioned. The subagents do not, and that is by
design. They’re presupposed to be disposable. Exploration, repeated file reads,
failed approaches, and noisy intermediate reasoning are supposed to keep in
employee contexts and by no means make the journey again to the primary thread.

Cognitive Locality Adjustments What Parallelism Is For

This reframes the duplicated-orientation drawback, which wasn’t actually “two
brokers studying the identical recordsdata.” It was two brokers independently reconstructing
the identical psychological mannequin of the codebase, as a result of the work had been partitioned
by process somewhat than by the information every process required. I’ve began calling
that distinction cognitive locality: Duties that want the identical psychological mannequin ought to often keep collectively.
Splitting them simply forces a number of brokers to rebuild the identical understanding
from scratch.

Parallelism nonetheless issues right here, it is simply not the primary level. Working 4
brokers concurrently is beneficial, however extraordinary. The actual profit is that they
preserve noisy intermediate reasoning out of the primary thread and return solely what
it nonetheless wants. That is the isolation subagents are supposed to supply, and it
solely holds if the primary thread respects it.

My working perception now: that is what subagents are literally for. Not that
they save time, however that they allow you to offload reasoning the orchestrator
does not want to carry onto, so it has much less to hold and fewer competing for its
consideration. Get the isolation proper, preserve issues native by cognitive locality,
and subagents turn out to be the device that protects the orchestrator’s working reminiscence,
not only a price you tolerate for parallelism. That is a perception, although, not a
measurement. What I’ve truly measured is the opposite facet of it, the price of
getting the isolation unsuitable.

Turning a Session into Standing Guidelines

My subsequent transfer was the plain one. Encode the lesson into CLAUDE.md, the
standing instruction file each session masses.

It will have been simple to put in writing a big corrective coverage right here, however each
further line in a standing instruction file is a price paid once more on each future
session. So I compressed the repair into the smallest algorithm that addressed
the failures I would seen. Each is admittedly answering the identical query. Does
this piece of knowledge, or this manner the work is cut up, earn a spot within the
orchestrator’s context?

1. Desire two to 4 brokers in a single wave. If the orchestrator needs 5 or
   extra, it ought to first ask whether or not duties sharing recordsdata or conventions must be
   merged.
2. Don’t ballot background brokers for standing when the reply could be given
   from what’s already recognized. Don’t fetch a full transcript to reply a
   light-weight query.
3. Don’t enable repository-wide git operations inside concurrent agent
   prompts.
4. Deal with overlapping file possession as a consolidation sign, not a cue to
   spawn extra brokers.

None of those inform the orchestrator precisely what to do in each case. Every
one provides it one thing to examine or ask itself earlier than appearing, not a script to
run, and none of them is profound by itself. Their solely actual worth is that
they’re all aimed on the similar factor, maintaining disposable reasoning disposable
and maintaining room within the orchestrator’s context for what it wants later within the
session.

The Subsequent Mistake Would Have Been Extra Governance

A later session surfaced a special hole. I had began the orchestrator
with specific abilities for the sort of work I needed, coding steerage in a single
case, design steerage in one other. I assumed that after a talent was energetic in
the primary thread, spawned subagents would comply with it routinely. They do not.
A subagent does not inherit abilities energetic within the guardian session until the
orchestrator passes them alongside explicitly.

My first intuition was so as to add a confirm-before-spawn gate. The orchestrator
would cease, record which brokers it needs to launch and which abilities every ought to
load, and look forward to my approval.

I am glad I did not preserve that model. It solved the unsuitable drawback. I did not
have proof that unhealthy spawn plans have been slipping via for lack of a
affirmation step. I would found a lacking truth about talent propagation, and
that is a special sort of hole. A common affirmation gate would have added
a round-trip to each related session, and earlier than lengthy I would nearly actually
have began approving these prompts on autopilot.

At that time, I spotted I wasn’t actually bettering governance. I used to be
simply including one other ritual.

And it nonetheless would not have caught the true drawback from earlier than, the
orchestrator polluting its personal context.

The narrower repair held up higher. Earlier than spawning, the orchestrator states
which energetic abilities are related to every agent’s process and factors the subagent
on the talent file to load, somewhat than pasting the entire talent inline.
Affirmation is barely required above the identical batch-size threshold already in
place, or when file possession is ambiguous.

That left me with a heuristic I now use greater than the rule itself: Earlier than
including a line to a standing instruction file, ask whether or not a fairly
competent orchestrator would make the fitting choice as soon as it knew the one
lacking truth.

If sure, the rule ought to simply state the very fact. If the repair begins specifying a
choice process, resembling approvals, checkpoints, obligatory steps, that is
often an indication I am encoding course of the place a small clarification would have
completed the job.

I do not know but whether or not that heuristic survives tougher circumstances. For now it
stops me from turning each fascinating incident right into a miniature
forms.

The place This Leaves Me

I haven’t got a settled view of how a lot governance is sufficient, and I do not
assume this piece earns one.

What I’ve as a substitute is a small flywheel, with a human nonetheless firmly within the
center of it. A session exposes a niche. Somebody has to note that it felt
unsuitable, cease the work lengthy sufficient to examine it, determine whether or not the issue is
actual or simply noise, and decide what deserves to turn out to be a standing rule. The
orchestrator can grade its personal session and floor clues, because it did right here, however
it can not make that judgment name itself. The selection of what to codify, what
to depart alone, and what is likely to be an overreaction continues to be mine. The subsequent
session then tells me whether or not that judgment improved the work or simply created
a special sort of waste.

The artifact of this spherical is the present model of
my CLAUDE.md.
It is not a completed prescription. It is the state of the calibration after this
iteration. The thresholds in it, two to 4 brokers per wave, 5 as a
consolidation sign, match the work I used to be doing once I wrote them. I would not
current them as something like common constants, and I would be suspicious of
any orchestration write-up that did. They have been additionally calibrated in opposition to Claude Sonnet 5,
and I have never examined how they maintain up in opposition to others. A distinct
mannequin would possibly moderately want a special stability. Just a few extra guidelines have amassed
within the file since. Deal with this file as a pattern, not a template, modify and optimize
it for no matter mannequin and workflow you are truly operating. What issues is not the
particular file, it is the behavior behind it: noticing a failure, asking what it truly
price, and writing the rule that will have caught it.

For years we optimized software program techniques round CPU, reminiscence, and throughput.
The primary wave of LLM tooling taught us to observe tokens. This session made me
suspect there is a third factor price watching in long-running agent workflows:
the standard of the orchestrator’s personal working reminiscence, the one useful resource that,
as soon as polluted, retains charging lease for the remainder of the session. I do not assume
that is a settled legislation but. It is a sample that held up within the classes I’ve
checked out to this point.

The tax I went on the lookout for was by no means on the subagents. It was on the
orchestrator, in what it selected to hold ahead. That is the query I carry
into each multi-agent design now: not what number of brokers to run, however what earns
a spot within the orchestrator’s context.

The questions I am left with are genuinely open:

• How do I measure this correctly, as a substitute of counting on the orchestrator’s personal
  account of its errors?
• When ought to a lacking truth go into the directions, and when does that flip
  into an excessive amount of course of?
• What is the subsequent orchestration mistake I am not seeing but?

I anticipate I will revise the file once more. That feels much less like a failure of
foresight and extra like the traditional price of working with a system opaque sufficient
that doubt itself turns into a part of the tactic.