One warm context beats five cold ones

A feature needs five layers generated. The obvious arrangement is five workers, one per layer, each told what to produce.

Each of them starts by reading the same several hundred kilobytes of conventions.

TL;DR#

• Splitting codegen by layer makes every worker re-read the same 305,942-byte conventions pack — the preamble is paid per worker, so five cold workers spend 1,529,710 bytes to five times nothing.
• One warm worker generating all five layers in sequence pays it once, and the layers it writes are consistent with each other for free rather than by convention.
• Scoping the pack by project type saved 8.2 per cent, not the ~40 per cent I estimated — and the measurement is the only reason I know that.
• The same measurement found a surface that had never been in the pack at all, because the build globbed for it instead of declaring it; the fix is an explicit SURFACES table and an assertion that exits 4 on an undeclared surface.
• Read on for the pack builder, the require that refuses, and why a thin pack fails more quietly than a fat one.

The case this post is about#

One feature, profile, in a Kotlin Multiplatform app. Generating it means producing five layers — a DTO set, a store, a view model, a screen, and the DI wiring — and every one of those has to obey the same conventions: where files go, what they are named, which patterns this codebase settled on two years ago.

Those conventions are not in anybody's head. They are a built artifact:

core/scripts/impl-context-pack-build.sh --type kmp
# → .claude-runtime/state/impl-context-pack.kmp.md     305,942 bytes

That is the thing each worker has to load before it can write a line. The question this post is about is how many times it gets loaded, and the answer follows entirely from where the work is split:

# split per layer — five workers, five loads
for layer in dto store viewmodel screen di; do dispatch "$layer" profile; done

# split per feature — one worker, one load, five layers
dispatch profile                     # loads the pack once, generates all five

Same five layers either way. The second arrangement reads that pack once instead of five times, and the difference is not only the bytes — it is that one worker makes one set of decisions and applies them five times.

(That 305,942 is the pack before it was scoped by project type. Scoping takes it to 280,963, and the two numbers are two different savings that are easy to conflate — the arithmetic separating them is in the fix.)

The symptom#

The cost of the run is dominated by something that is not the work. Five workers, five full context loads, five passes over the same training material — and the actual generation, the part anybody wanted, is a small fraction of what was spent.

The second thing is subtler and costs more. Each worker reconstructs its understanding of the codebase independently, from the same material, and arrives somewhere slightly different. The naming is consistent within a layer and drifts between layers; two files that should mirror each other do not quite; a convention is applied in four places and interpreted differently in the fifth.

The third is that none of this appears as a failure. Every layer is individually good. The inconsistencies are the kind a reviewer reads as style, and the cost is a number on a bill that nobody attributes to the arrangement that produced it.

The fourth is what happens when the feature is large. More layers means more workers means more loads, so the cost grows with the thing you are generating rather than with its difficulty — and a feature twice the size costs more than twice as much, which is the wrong direction for any arrangement to scale in.

The fifth thing is organisational rather than technical and it is why the arrangement persists: one worker per layer is a beautiful diagram. It maps onto how people already think about the codebase, it is easy to explain, and every individual worker is simple. None of those properties has anything to do with what it costs, and all of them make it hard to argue against.

Why the obvious fix failed#

Cache the context. Helps with the reading and not with the rest. The expensive part is not fetching bytes; it is a worker arriving at a working understanding of them, and that happens per-worker whatever the transport.

Trim what each worker loads. Right instinct and dangerous in the obvious implementation. Trimming to what a layer "obviously needs" removes exactly the cross-layer conventions that make layers agree, and the resulting output is cheaper and less consistent.

Run the workers in parallel. Makes the wall-clock better and the cost identical. It also makes the divergence worse, because no worker can see what the others decided — parallelism is a schedule, not an economy.

Pass a summary between workers. Sounds like the fix and loses precisely what matters. Conventions live in specifics — the exact file path, the exact naming shape — and a summary is a lossy encoding of specifics, so what survives is the part a worker could have guessed.

Let a worker read only what it asks for. The sophisticated version, and it assumes the worker knows what it does not know. A generator unaware that a convention exists does not go looking for it; on-demand loading works for the context you can name and is silent about the context you cannot.

The actual mechanism#

Load the context once, into one warm worker, and run every generator inside it.

pack      the conventions, assembled once, as one artifact
warm      one worker loads it and generates every layer for the feature
fan out   at the FEATURE boundary, not the layer boundary

The third line is the whole design. Layers share context; features mostly do not. Splitting at the layer boundary means paying for context assembly at every split, while splitting at the feature boundary means each worker pays once and amortises it across everything it produces.

The consistency benefit is the one I did not expect to matter as much as it does. A single worker generating five layers makes one set of decisions and applies it five times, so the layers agree by construction rather than by convergence — and nothing has to check that they do.

The sub-case that makes inline generation safe rather than merely cheap is the pack itself. Generating inline without it is not warm, it is improvisation: a worker authoring a layer from general knowledge rather than from this codebase's conventions. The pack is the thing that makes the shortcut legitimate, which means it has to be loaded first and verifiably.

And the dangerous failure here is the inverse of the obvious one. A pack that is too large is simply expensive; a pack quietly missing a surface produces a worker that improvises in one area with nothing saying so. That asymmetry is why the pack must be asserted rather than assumed — each generator declaring the surfaces it needs and halting by name when one is absent, rather than proceeding on whatever happens to be loaded.

There is a staleness question sitting underneath all of this that is worth naming even though it is somebody else's post: the pack is a derived artifact, so it can be out of date with the sources it was built from, and a worker loading a stale pack is confidently applying last month's conventions. The pack needs the same currency check as any other generated thing, which is a cost the cold arrangement did not have.

The fix#

build     one pack per project type, from a declared source set
assert    each generator declares its surfaces; a missing one halts by name
load      once, at the start of the warm context
split     at the feature boundary; never per layer

Here is the whole of it for profile, in the order the four lines run.

Build. The pack is the concatenation of a declared source set, filtered by what this project type actually reads. The filter is the entire saving, and the excluded list it returns is what makes the next step able to tell two different absences apart:

// impl-context-pack-build.ts
type Surface = { id: string; path: string; types: string[] };

const SURFACES: Surface[] = [
  { id: "CORE_STORE",  path: "training/CORE_STORE.md",            types: ["kmp", "game"] },
  { id: "CORE_DATA",   path: "training/CORE_DATA.md",             types: ["kmp", "game"] },
  { id: "GAME_SCENE",  path: "training/GAME_SCENE_BUILD_GUIDE.md", types: ["game"] },
  // … 24 more rows, same shape — 27 surfaces in all
];

export async function buildPack(type: string, outDir: string) {
  const included: string[] = [];
  const excluded: string[] = [];
  let body = "";

  for (const s of SURFACES) {
    if (!s.types.includes(type)) {        // this type never reads it — pure per-worker cost
      excluded.push(s.id);
      continue;
    }
    body += `\n\n<!-- ${s.id} -->\n` + await Deno.readTextFile(s.path);
    included.push(s.id);
  }

  // The manifest is not bookkeeping. Without `excluded`, the assertion below cannot
  // distinguish "scoped out on purpose" from "nobody ever declared this", and those
  // need opposite responses.
  await Deno.writeTextFile(`${outDir}/pack.${type}.md`, body);
  await Deno.writeTextFile(
    `${outDir}/pack.${type}.json`,
    JSON.stringify({ type, included, excluded, bytes: body.length }, null, 2),
  );
  return { included, excluded, bytes: body.length };
}

Assert. Every generator names the surfaces it is about to rely on, before its first write. The three-way split is the point — the failure path is what the whole mechanism exists for:

// impl-context-pack-require.ts
export function requireSurfaces(declared: string[], manifestPath: string): number {
  let m: { included: string[]; excluded: string[] };
  try {
    m = JSON.parse(Deno.readTextFileSync(manifestPath));
  } catch {
    console.error("pack not built — run impl-context-pack-build first");
    return 5;
  }

  const missing = declared.filter((id) => !m.included.includes(id));
  if (missing.length === 0) return 0;

  const undeclared = missing.filter((id) => !m.excluded.includes(id));
  if (undeclared.length === 0) {
    console.error(`excluded for this project type: ${missing.join(", ")}`);
    return 3;                 // scoped out on purpose — generator wired to the wrong type
  }
  console.error(`undeclared surface: ${undeclared.join(", ")}`);
  return 4;                   // nobody declared it — a generator about to improvise
}

Load and split. The orchestrator loads the pack once and fans out at the feature boundary, so every layer of profile is generated inside the one warm context. features is the list being generated — ["profile", "settings", …] — and worker(pack, fn) is whatever starts a context with that pack loaded and runs fn inside it; the shape of that primitive does not matter here, only that it is entered once per feature:

const OUT = ".claude-runtime/state";
const MANIFEST = `${OUT}/pack.kmp.json`;

// what each layer's generator relies on — the declarations the assert checks
const NEEDS: Record<string, string[]> = {
  dto:       ["CORE_DATA"],
  store:     ["CORE_STORE", "CORE_DATA"],
  viewmodel: ["CORE_STORE"],
  screen:    ["CORE_UI", "CORE_DESIGNSYSTEM"],
  di:        ["CORE_DI"],
};

const pack = await Deno.readTextFile(`${OUT}/pack.kmp.md`);   // ONCE, for every feature

await Promise.all(features.map((feature) =>                   // fan out per FEATURE
  worker(pack, async (ctx) => {
    for (const [layer, needs] of Object.entries(NEEDS)) {     // all five in ONE context
      const code = requireSurfaces(needs, MANIFEST);
      if (code !== 0) throw new Error(`halt ${code} on ${feature}/${layer}`);
      await ctx.generate(layer, feature);
    }
  })
));

Two things in that block are load-bearing. pack is read once, outside the fan-out — move it inside and you have rebuilt the arrangement this post argues against. And the layer loop is sequential inside each worker on purpose: that is what makes the five layers share one set of decisions, which is the consistency half of the benefit.

Run on profile, with the pack built for kmp. Three cases, which are the three the split exists to tell apart:

$ deno run -A impl-context-pack-build.ts --type kmp
pack.kmp.md  280,963 bytes  included 24  excluded 3

$ deno run -A impl-context-pack-require.ts CORE_STORE CORE_DATA
exit 0

$ deno run -A impl-context-pack-require.ts CORE_STORE GAME_SCENE
excluded for this project type: GAME_SCENE
exit 3

$ deno run -A impl-context-pack-require.ts CORE_STORE CORE_NAV
undeclared surface: CORE_NAV
exit 4 — halt on profile/store

The two failing runs look alike and mean opposite things, which is the whole reason the manifest carries excluded.

GAME_SCENE is in SURFACES with types: ["game"], so a kmp build puts it in excluded by design. The store generator asking for it is a generator wired to the wrong project type — a real mismatch, worth reporting, and not a reason to halt the run: the pack is correct and a kmp worker legitimately does not carry it. If that were also exit 4 the assertion would fire on correct runs and be switched off within a week.

CORE_NAV is in neither list. Nobody ever declared it, so nothing scoped it out and nothing included it — the pack is silently short of something a generator believes it has. That is exit 4, and it is the one that matters: without it the store layer is produced anyway, from general knowledge rather than from this codebase's conventions, and it compiles.

The two savings are not the same size, and I had them the wrong way round for a while. Scoping the pack is the visible change — a smaller file, a number that moves — so it feels like the win. The arithmetic says otherwise:

per-layer, unscoped   5 loads × 305,942  = 1,529,710 bytes
per-feature, scoped   1 load  × 280,963  =   280,963 bytes

saved by splitting per feature   4 × 305,942 = 1,223,768   (98% of the total)
saved by scoping the pack            24,979               ( 2% of the total)

The split does almost all of the work. Scoping is a real improvement on top of it and it is a second-order one, and quoting the two together as "the saving" overstates what scoping buys — which is exactly what I did before working the numbers out.

Scoping is also the only part of this that degrades on its own. New guides get added, patterns get renamed, and a scoping rule written once drifts out of alignment with what is on disk — so it needs a periodic read, not just an initial one.

assert is the line I would not ship without. An omitted surface has no symptom: the generator produces plausible code, the build passes, and the convention it did not know about is simply absent from the output. A declared-and-asserted surface converts that into a halt with a name on it, which is the difference between a silent defect and a stopped run.

split is the line that feels wrong the first time. Splitting per feature means a single worker doing what looks like five jobs, and the instinct that this is too much for one worker is strong — it is also, as far as I can tell, entirely about how the work looks rather than about what the worker can hold.

How we proved it#

The scoping saving is measurable and was smaller than I guessed. Scoping the pack by project type took it from 305,942 bytes to 280,963 — about 6,200 tokens per worker, 8.2 per cent. A rough estimate beforehand had suggested something near 40 per cent, and the gap between those two numbers is the reason I am quoting the measured one.

The same measurement found a surface that was never in the pack at all — and finding it is why the implementation above uses an explicit SURFACES table rather than a glob. The original build did not have that table. It globbed:

# what the build used to do
CORE_*.md            → CORE_STORE.md, CORE_DATA.md, CORE_DATABASE.md …
# the file that pattern never matched
GAME_SCENE_BUILD_GUIDE.md

So that guide had been absent from every worker since the pack existed — discovered by counting bytes while scoping, not by anything failing. A generator that needed it improvised, and the output compiled.

The glob is the real villain here, and it is worth being precise about why. A glob has no idea what it was supposed to match, so a file outside the pattern is not an error — it is simply not a result, and an empty non-result looks exactly like a correct one. The explicit table cannot fail that way: a surface is a row or it does not exist, and a row that nobody wrote is a row somebody can notice is missing. Replacing the glob is what converts this from an invisible gap into exit 4.

The cost reduction is the least interesting result. Eight per cent is worth having and would not on its own justify restructuring anything; what justified it was the consistency, which does not show up in any measurement I took and which I only noticed by its absence from review.

The consistency improvement I cannot quantify. Layers generated in one warm context agree in ways that previously needed correcting, and I have no before-and-after count of those corrections — the evidence is that a class of review comment stopped appearing.

The feature-boundary claim is the part I would like a cleaner experiment for. It follows from the arithmetic, and the arithmetic assumes layers share nearly all their context while features share little. That is true in this codebase and I have not tested a project where features are heavily coupled, where the right boundary would presumably be somewhere else entirely.

What I have not solved: the ceiling. A warm context holds a finite amount, and a feature large enough to exhaust it has to be split — at which point the split is governed by what fits rather than by what shares context, and the amortisation argument stops applying cleanly.

And a cost worth naming: one warm worker per feature is less parallel than one per layer. The arrangement trades wall-clock for cost and consistency, and on a day when somebody is waiting, that trade is felt. It is the right default and it is not free.

The thing that reframed it was realising the question is not how to make context loading cheaper but where to put the boundary so it happens less often. Every architectural choice about splitting work is implicitly a choice about how many times shared state gets rebuilt, and that cost is usually invisible because it does not appear as a line item anywhere.

Which is a general property of worker-per-task designs: the task decomposition people reach for is the one that matches the domain — a layer, a file, a module — and the one that matters economically is the one that matches where state is shared. Those coincide less often than the first instinct suggests.

What this generalizes to#

When each unit of work pays a fixed setup cost, the unit boundary is an economic decision and is usually made on other grounds.

The same arithmetic governs a surprising amount: a CI job that installs dependencies per step, a process spawned per file, a database connection opened per query, a container started per request. In every case the work is cheap and the preparation is not, and the fix is never to make preparation faster — it is to do it once and do more inside it.

So the question worth asking of any fan-out is what does each worker have to rebuild before it can start. If the answer is "the same thing as every other worker", the split is in the wrong place, and moving it costs nothing but a change of mind about what a unit of work is.