A bigger context window gave me a worse answer, repeatably

TL;DR#

Every code-generating subagent in our framework is handed a context pack — a bundle of training guides assembled before the agent starts. It was one pack for every project type, on the reasoning that more reference material cannot hurt.

It can. A Kotlin Multiplatform feature was being generated with a game-engine guide and two design-time guides in its context, none of which any generator for that project type reads. The measurable cost was 8.2% of the pack's tokens. The unmeasurable one was worse: the generator had four hundred lines of a different discipline's idioms available to pattern-match against.

Scoping the pack by project type fixed it — and the same audit found the opposite defect, a guide that was supposed to be in the pack and had never been in it, because the glob that built the pack had never matched its filename.

The lesson is not "smaller contexts are better". It is that a context assembled by a wildcard is not a decision, and nothing tells you which half of the error you have.

The case this post is about#

A code generator dispatched once per feature. Before it runs, a build step assembles everything it might need to know about the codebase's conventions:

# impl-context-pack-build.sh — one pack, every project type
cat training-layer/instructions/stream-first/latest/CORE_*.md > "$PACK"

That glob was the whole policy. It produced a 305,942-byte pack, and the pack was loaded by every generator on every feature of every project — a Kotlin Multiplatform app, a game, anything.

The reasoning behind it was the obvious one. The model has a large context window, the guides are authoritative, and a generator that reads a guide it does not need has simply wasted a few thousand tokens. Cheap insurance.

The symptom#

Not a failure. A drift, which is why it took a deliberate audit rather than a bug report.

Generated Compose code kept acquiring game-engine shapes. A Kotlin Multiplatform feature — ordinary forms, lists, a repository — would come back with a frame-loop flavour to its state handling:

// generated for a settings screen. Nothing here needs a tick.
LaunchedEffect(Unit) {
    while (isActive) {
        state.advance(deltaMillis = 16)
        withFrameNanos { }
    }
}

Reviewable, correctable, and not obviously caused by anything. It read as the model being imaginative.

The pack's own contents had never been audited. When I finally printed what the glob matched, for a kmp project:

CORE_STORE.md            41,203 B   read by the store generator        ✓
CORE_DATA.md             38,887 B   read by the data generator         ✓
CORE_DATABASE.md         22,140 B   read by the data generator         ✓
CORE_GAME.md             18,402 B   read by kmp-game-*-gen ONLY        ✗ not this project type
CORE_STITCH_SPEC.md      12,655 B   design-time, no generator reads it ✗
CORE_STITCH_RENDER.md     9,918 B   design-time, no generator reads it ✗
…
                        305,942 B total

Three files, 40,975 bytes, that no generator on this project type could ever consult.

And the inverse, which I was not looking for. GAME_SCENE_BUILD_GUIDE.md — a guide the game generators genuinely need — was not in the pack at all, on any project type. The glob was CORE_*.md. The filename does not start with CORE_. It had never matched, and nothing had ever said so.

Why the obvious fix failed#

"The window is big enough, so this does not matter." The token cost is real and modest. The adherence cost is neither. A model asked to generate Compose code with a game-loop guide in context has that guide's idioms available as precedent, and precedent in context is a stronger signal than a convention described in prose elsewhere.

"Trim it to the smallest useful set." The instinct, and it is the direction that produces the dangerous error. A pack that quietly drops a surface the generator needed makes it improvise, with nothing in the output saying it was improvising. Under-inclusion fails silently; over-inclusion fails visibly, in review. They are not symmetric and should not be traded against each other casually.

"Estimate the saving and decide if it is worth it." My rough estimate was about 40%, because the excluded files felt large. Measured, it was 8.2% — 305,942 B down to 280,963 B, roughly 6.2k tokens per subagent. The change was still worth making, and if I had argued for it on the 40% figure I would have been arguing from a number I had not checked.

"Let each generator read what it needs, at runtime." Appealing, and it moves a fixed startup cost into a variable per-generator one, with no way to know in advance what a run will consume. The pack exists so the cost is bounded and knowable.

The actual mechanism#

Three things, and only the first is about tokens.

A context pack is a prompt, not a library. A library is consulted; a prompt is attended to. Everything in the window participates in the model's next-token distribution whether or not it is relevant, and relevance is being judged by the same model you are trying to steer.

Concrete code outcompetes abstract convention. Eighteen thousand bytes of working game-loop Kotlin is a far stronger signal than a sentence in a style guide saying "features use MVI". The model is not reasoning about which guide applies to this project type — it is pattern-matching against what is available, and the game guide is extremely available.

The glob is the policy, and a glob has no opinion about project type. CORE_*.md encodes one rule: filename starts with CORE_. It cannot express "guides this generator reads", so the pack's contents are an accident of naming — which is exactly how a needed guide goes missing and an irrelevant one stays in, at the same time, without either being noticed.

A wildcard is not a decision. It produces a context whose contents nobody chose, and the two failure modes — a surface that should not be there and one that should — are invisible in exactly the same way.

The fix#

Scope the pack by project type, stamp what went in and what was left out, and let a generator assert the surfaces it is about to rely on.

The assertion is the load-bearing half. Scoping alone converts a visible failure into a silent one; the assertion is what keeps it loud:

#!/usr/bin/env bash
# impl-context-pack-require.sh <surface> — called by a generator BEFORE it writes anything.
#
# Exit codes are the contract:
#   0  present in this type's pack — proceed
#   3  deliberately excluded for this project type — proceed WITHOUT it, and say so
#   4  undeclared: not in the pack and not excluded — the pack is wrong, HALT
#   5  no pack built for this type at all — HALT
set -euo pipefail

surface="$1"
pack_dir="${IMPL_CONTEXT_PACK_DIR:?pack dir not set}"
manifest="$pack_dir/PACK_MANIFEST.txt"

[ -f "$manifest" ] || { echo "no pack manifest at $manifest" >&2; exit 5; }

if grep -qxF "include $surface" "$manifest"; then
  exit 0
fi

if grep -qxF "exclude $surface" "$manifest"; then
  # An EXCLUSION IS A DECISION and must have been written down. The generator
  # continues without the surface, which is correct — but it is told, so it can
  # refuse to emit anything that would have depended on it.
  echo "note: '$surface' excluded for project type $(cat "$pack_dir/TYPE")" >&2
  exit 3
fi

# Neither included nor excluded. This is the GAME_SCENE_BUILD_GUIDE case: a surface
# nobody declared either way, which a glob-built pack cannot distinguish from one
# that does not exist. Refuse rather than improvise.
echo "surface '$surface' is undeclared for type $(cat "$pack_dir/TYPE")" >&2
echo "  it is neither in the pack nor listed as excluded — the pack definition is wrong." >&2
exit 4

Run against the pack as it stood, with the three surfaces a KMP store generator actually consults plus the one that exposed the hole:

$ impl-context-pack-require.sh CORE_STORE.md       ; echo "exit=$?"
exit=0

$ impl-context-pack-require.sh CORE_GAME.md        ; echo "exit=$?"
note: 'CORE_GAME.md' excluded for project type kmp
exit=3

$ impl-context-pack-require.sh GAME_SCENE_BUILD_GUIDE.md ; echo "exit=$?"
surface 'GAME_SCENE_BUILD_GUIDE.md' is undeclared for type kmp
  it is neither in the pack nor listed as excluded — the pack definition is wrong.
exit=4

Exit 3 and exit 4 are the whole design. Both mean "this file is not in your context", and they are opposite situations: one is a decision someone made, the other is a gap nobody knew about. A pack built by a glob returns the same silence for both.

And one rule that has to be stated or the gate is worthless: a generator hitting exit 4 must halt, not read the file directly. Working around the assertion restores exactly the condition it exists to detect, with the added problem that the pack still looks correct.

How we proved it#

The size change, measured rather than estimated:

pack, all types (before)   305,942 B
pack, type=kmp  (after)    280,963 B
                            24,979 B removed   = 8.2%   ≈ 6.2k tokens per subagent

Worth stating plainly that this is not the 40% I had guessed. The three excluded files were large in isolation and small against a 300KB pack. The change stands on the adherence argument; the token saving is a secondary benefit I had overestimated by roughly five times.

The adherence change, across twenty generated features — ten before, ten after, same ten feature specs each time:

                                        before    after
frame-loop constructs in non-game code     4/10     0/10
design-time idioms in generated source     2/10     0/10
module placement correct                  10/10    10/10

The last row matters as a control: module placement was correct before and after, which is what you want from a change that was supposed to affect idiom selection and nothing else.

That the assertion catches the inverse. The GAME_SCENE_BUILD_GUIDE.md gap had existed since the pack was introduced and had never produced a reported bug — the game generators improvised, and their output was plausible. Adding the gate surfaced it on the first game-feature run:

$ ./run-generator kmp-game-scene-gen --feature arena
surface 'GAME_SCENE_BUILD_GUIDE.md' is undeclared for type game
HALT production-affecting / pack_surface_omitted

That halt is the result I value most from this whole exercise, and it is the one the token measurement would never have found.

Two limits worth naming. Twenty features is a small sample, and the four frame-loop cases were all in state-handling code — I cannot claim the effect generalises to every kind of drift from every kind of irrelevant guide. And the per-type pack path means a game drive and a KMP drive now build separate packs, which doubles the build step's disk use; that was a deliberate trade to stop the two clobbering each other's pack.

What this generalizes to#

Everything in the window is a prompt, including the parts you consider reference. There is no inert context. A model cannot be told "consult this only if relevant" in any way that is as strong as the content itself being present.

Inclusion built by pattern is inclusion nobody chose. A glob, a directory scan, an "attach the whole folder" is a policy expressed in filenames. It will be wrong in both directions eventually, and neither direction announces itself.

Over-inclusion and under-inclusion are not symmetric. Too much shows up as odd output in review. Too little shows up as confident improvisation that reads exactly like competent work. Any system that trims context should get louder, not quieter, as it trims.

Measure the saving before you argue from it. My estimate was five times the real figure. The change was right anyway, for a reason that had nothing to do with tokens — and a decision defended with an unverified number is one that falls over when somebody checks.

Related#

• Context is a budget, not a bucket — ablating blocks to find what earns its tokens
• The prompt that was a config file — versioning the text that determines behaviour
• The eval that passed on a stub — why a correlated verifier catches nothing

Keep reading#

This is part of Context engineering for code — what actually goes into the window, what it costs, and how to tell. Follow MobileByteSensei for the rest.