The context window is not your constraint — what you put in it is

Every context problem I have had this year was a problem of what was resident, not of how much room there was.

The window is a ceiling. Ceilings matter when you hit them. Most of the time the defect arrives long before that, in the shape of a step nobody executed because it was surrounded by forty paragraphs that did not apply.

The case this post is about#

One shared instruction bundle, loaded by every generator worker, measured before and after scoping it by project type:

before   305,942 B
after    280,963 B
saved    ~6.2k tokens per worker  (8.2%)

My estimate before measuring was 40 per cent. The gap between those two numbers is the point of the measurement and nearly the point of the post, because on the strength of 40 per cent I was considering restructuring how the bundle was assembled.

This post follows that one bundle, because a bigger context window would have made all three numbers irrelevant and changed nothing about the actual problem: the step a worker needs is buried among forty that do not apply to it.

The symptom#

The failure does not look like running out of room. It looks like a step being skipped in a run that reports success.

An instruction file grows — not carelessly, but through care. Every hard-won detail is worth recording: this failed once, that ordering matters, here is why the obvious approach is wrong. Each addition is individually justified.

What degrades is findability. The executable parts — the steps, the commands, the halts — end up distributed among rationale that applies to one run in fifty. Nothing is missing. The document is complete and correct. And an agent working through it under task pressure reads past the one line that mattered.

A larger window does not touch that. Doubling the ceiling doubles how much irrelevant material can sit between two relevant lines.

Why the obvious fix failed#

The obvious fix is to trim. Write shorter, be concise.

"Keep it concise" is not a constraint, it is an aspiration, and it loses to the next individually-justified paragraph. I know because I wrote those paragraphs and each felt necessary at the time.

The second attempt is to move rationale elsewhere — and without a forcing function, the question does this belong here or in the reference? has no natural answer. Both are defensible for almost any paragraph, so the default is to leave it where you wrote it. The decision has no cost until something makes it expensive.

The third is to assume the model will handle it. Longer context windows do genuinely help with recall. They help much less with precedence: the difference between a document containing an instruction and a document where that instruction is the salient thing at the moment of action.

The actual mechanism#

Two mechanisms, and they work on different halves of the problem.

A hard numeric cap on anything always resident. Command runtimes here are capped at 200 lines and the cap is enforced on write. It converts an unanswerable question — is this paragraph worth its space? — into a forced local decision at the moment you are adding it. The refusal names the remedies and forbids the cheapest one: relocate, never delete.

Scope the shared material to the consumer. A bundle loaded by every generator carried guidance for several project types, so a worker on one type was reading instructions for the others.

The first improves findability inside what is loaded. The second reduces what is loaded at all. Both are about residency; neither is about capacity.

The fix#

The second one is where I have a number, and the number is the point of this article.

Before scoping, the shared bundle was 305,942 bytes. After scoping it per project type:

before   305,942 B
after    280,963 B
saved    ~6.2k tokens per worker  (8.2%)

My estimate before measuring was 40%.

The measurement is a fan-out over the same assembler, which is the only reason the two numbers are comparable at all — a before-and-after that rebuilds differently measures two things:

type Surface = { name: string; bytes: number; appliesTo: string[] };

function assemble(surfaces: Surface[], type: string | null) {
  const included = surfaces.filter((s) => type === null || s.appliesTo.includes(type));
  const excluded = surfaces.filter((s) => !included.includes(s));
  const bytes = included.reduce((n, s) => n + s.bytes, 0);
  return { bytes, included: included.map((s) => s.name), excluded: excluded.map((s) => s.name) };
}

function compare(surfaces: Surface[], type: string) {
  const all = assemble(surfaces, null);          // the unscoped bundle, as it was
  const scoped = assemble(surfaces, type);
  const savedBytes = all.bytes - scoped.bytes;
  return {
    beforeBytes: all.bytes,
    afterBytes: scoped.bytes,
    savedBytes,
    savedPct: Math.round(savedBytes * 1000 / all.bytes) / 10,
    savedTokens: Math.round(savedBytes / 4 / 100) * 100,   // ~4 bytes/token
    dropped: scoped.excluded,
  };
}

Run for the project type the workers were actually doing:

console.log(compare(loadSurfaces(), "kmp"));

{
  beforeBytes: 305942,
  afterBytes: 280963,
  savedBytes: 24979,
  savedPct: 8.2,
  savedTokens: 6200,
  dropped: [ "CORE_GAME.md", "CORE_STITCH_DESIGN.md", "CORE_STITCH_RECONCILE.md" ]
}

Three surfaces dropped, 8.2%, and the estimate was 40% — a factor of five, produced by looking at the list and judging how much of it felt irrelevant. The dropped field is why the gap is so large: the three genuinely inapplicable surfaces are small, and the bulk of the bundle is material every type needs. Nothing about reading the list conveys that, because a list shows you how many items are wrong and not how many bytes they are.

The gap matters more than the figure. On the strength of 40% I was considering a restructuring of how the bundle was assembled — real work, justified by a number I had produced by looking at the directory and thinking about it. The actual 8.2% is worth keeping and nowhere near enough to justify that.

That is the general shape of context intuitions in my experience: confidently wrong, in both directions, and cheap to check. The measurement took one script and a subtraction.

How we proved it#

The same audit surfaced something I was not looking for, and it is the more valuable finding.

A guide the generators genuinely needed had never been in the bundle at all. The glob that assembled it matched a naming convention that file did not follow, so it had been absent since the bundle was created.

Nobody noticed. That is the property worth sitting with: a missing instruction does not announce itself. The generator improvises, competently, and produces something slightly wrong for reasons nothing records.

So the fix carries a second half. Every bundle now declares what it contains, and a consumer asserts the surfaces it requires:

0  present
3  excluded for this project type
4  undeclared — asked for something nobody scoped
5  bundle not built

Any non-zero exit stops the run and names the missing surface. And it is equally forbidden to work around the refusal by reading the file directly — an assertion you can satisfy by going around it is a comment.

The direction nobody optimises#

Everything above is about loading less. The failure that actually cost me was loading too little, and it is worth stating why that asymmetry persists.

Over-loading is visible. It shows up in the token count, it is measurable, and anyone looking at a transcript can see where the budget went. It is an expensive problem with an obvious diagnosis.

Under-loading is invisible. The work completes, the output looks right, and the absence is recorded nowhere. You find it later, by counting something, or not at all.

Given that, the instinct to trim is dangerous precisely because it feels virtuous. Every paragraph removed is a small visible win against a small invisible risk, and nothing in the feedback tells you when you have crossed from one to the other.

Where a bigger window genuinely helps#

To be fair to the ceiling: there is a case where capacity is the binding constraint, and confusing it with the residency problem sends you to the wrong fix.

It is the long session, not the long document. A run that accumulates — every result collected, every decision about how work was split, the synthesis at the end — fills up for reasons no scoping addresses, because none of that material is optional. It is the work.

The tell is which direction the pressure comes from. If the problem is that a specific instruction gets read past, that is residency, and a larger window makes it marginally worse. If the problem is that a run cannot hold what it has legitimately produced, that is capacity, and no amount of tightening the loaded material will fix it.

The two need opposite responses — trim versus checkpoint-and-resume — which is the practical reason to tell them apart rather than treating "context problem" as one thing.

What this generalizes to#

Stop treating the window as the variable. Ask instead what is resident at the moment a decision is made, and whether the thing that decides it is findable among everything else that is there.

Put a hard numeric cap on whatever loads unconditionally, because it is a budget and unenforced budgets are not budgets. Scope shared material to its consumer — but measure the saving before restructuring anything, because the intuition is unreliable by a factor of five in my one measured case.

Then give every trimmed bundle a way to report that it trimmed too much. The costly mistake is not the one you can see in a token count.