Skills, tools, and the difference that matters

A tool is a function the model can call. A skill is a set of instructions the model reads.

They get discussed as variations on one idea, and they have opposite cost profiles: a tool you do not call costs nothing, and a skill you do not use costs exactly as much as one you do.

The case this post is about#

One capability set, counted:

{
  listed: 357,
  humanOnly: 42,
  alwaysLoaded: 64260,
  reclaimed: 7560,
  conditional: 2952600,
}

Three hundred and ninety-nine capabilities. The descriptions of 357 of them are loaded every session whether or not any is used — 64 KB — and their bodies total 2.95 MB, paid only on invocation.

A 46× ratio between the two, and they behave completely differently under pressure. The always-loaded figure competes with the actual work; the conditional figure does not exist until something asks for it.

This post follows those numbers, because reclaimed: 7560 is the interesting line — that is the listing space freed by marking 42 capabilities human-only, which I did for safety and which turns out to be almost exactly the headroom.

The symptom#

The problem shows up at scale. This framework has 399 skills. Their instructions, loaded together, are far past any context window, and even the portion that would fit is the wrong thing to spend a window on — the task needs that space.

But the model cannot choose a skill it does not know exists. So there is a floor: something about every skill must be present, always, or the skill is unreachable.

That is the whole tension, and it is specific to skills. A tool's schema is small and fixed, so carrying all of them is affordable. A skill's body is a document — the reason it is a skill rather than a tool is that it carries judgement, worked steps, edge cases — and documents do not fit.

The failure mode when you get this wrong is not an error. It is a context two-thirds full before the work starts, and a model that reads past the relevant step because it is buried among forty irrelevant ones.

Why the obvious fix failed#

The obvious fix is summarisation: load a short version of everything, expand on demand.

It fails because the short version has to be short enough to be worth being short, at which point it stops being an instruction and becomes a label. A one-line summary of a deploy procedure is not a smaller deploy procedure; it is a pointer with no operational content at all. You have not compressed, you have deferred — which is fine, but then be explicit about it rather than pretending the summary is the skill.

The second attempt is relevance filtering: load only the skills related to the current task. This requires knowing what the task is before reading the instructions that describe what tasks exist, and the chicken-and-egg is genuine. It also fails badly at the boundary — the task that turns out to need an unrelated capability.

The actual mechanism#

Split by what the content is for, and load each tier at the moment that purpose becomes live.

Four tiers, and the questions they answer:

Always — enough to decide. Name, one-line description, when to use it. This is the only tier with a hard budget across all 399, and it contains no operational content whatsoever. Its job is to let the model recognise that a capability applies.

On invoke — enough to execute. The skill's actual body: the steps, the ordering, the checks. It loads when the skill is chosen and not before.

On trigger — enough for the hard cases. A deep runtime the body points at, pulled in only for workflows complex enough to need it. Most invocations never touch this.

Never, unless asked — enough to understand. Rationale, worked examples, edge-case tables, history. Genuinely useful to a person reading it deliberately; pure weight during execution.

The same principle applies to rules. Rather than loading every rule always, a hook matches the tool call's target against per-domain shards and injects only what applies to the file being touched. The permanently-loaded portion is a small map from domain to shard.

The fix#

Two details that took me longer to arrive at than the tiering did.

The always-loaded tier needs a hard numeric limit, or it absorbs everything. Every tier below it is conditional, so the pressure is always to move one more sentence up where it is guaranteed to be read. Here the always-loaded runtime files are capped at 200 lines and the cap is enforced on write, which converts "is this worth its space?" from a matter of taste into a forced decision at the moment of adding.

Removing a skill from the listing is a budget decision as well as a safety one. A set of capabilities here — publishing a release, pushing a secret, merging a pull request — are marked so the model cannot invoke them at all. I added that for safety. It also reclaims their share of the always-loaded listing, which is the scarcest tier. A capability that should only ever be typed by a human does not need to be described to the model at all.

The budget is countable, which is what turns both of those from principles into a number you can look at before adding anything:

type Capability = {
  name: string;
  listingBytes: number;        // the description loaded EVERY session
  bodyBytes: number;           // loaded only when invoked
  humanOnly: boolean;          // removed from the listing entirely
};

function budget(caps: Capability[], ceilingBytes: number) {
  const listed = caps.filter((c) => !c.humanOnly);
  const alwaysLoaded = listed.reduce((n, c) => n + c.listingBytes, 0);
  const reclaimed = caps.filter((c) => c.humanOnly).reduce((n, c) => n + c.listingBytes, 0);
  // Conditional cost is unbounded on purpose — it is paid per invocation, not per session.
  const conditional = caps.reduce((n, c) => n + c.bodyBytes, 0);

  return {
    listed: listed.length,
    humanOnly: caps.length - listed.length,
    alwaysLoaded,
    reclaimed,
    conditional,
    overBy: Math.max(0, alwaysLoaded - ceilingBytes),
    verdict: alwaysLoaded > ceilingBytes
      ? `OVER by ${alwaysLoaded - ceilingBytes} B — mark one human-only or shorten a description`
      : `within ceiling, ${ceilingBytes - alwaysLoaded} B of headroom`,
  };
}

The real set, with the action-tier capabilities marked:

const caps: Capability[] = [
  ...Array.from({ length: 357 }, (_, i) => ({
    name: `cap-${i}`, listingBytes: 180, bodyBytes: 7_400, humanOnly: false })),
  ...["release", "secrets-push", "pr-merge", "git-commit", "context-end", "project-remove",
      "server-promote", "social-publish", "lib-publish", "framework-release",
      ...Array.from({ length: 32 }, (_, i) => `action-${i}`)]
    .map((name) => ({ name, listingBytes: 180, bodyBytes: 7_400, humanOnly: true })),
];
console.log(budget(caps, 72_000));

{
  listed: 357,
  humanOnly: 42,
  alwaysLoaded: 64260,
  reclaimed: 7560,
  conditional: 2952600,
  overBy: 0,
  verdict: 'within ceiling, 7740 B of headroom'
}

Two numbers are worth sitting with. reclaimed is 7,560 bytes — almost exactly the headroom, which means that without the human-only marking this set would be over the ceiling on the safety change alone. And conditional is 2.95 MB against an always-loaded 64 KB: a 46× ratio, paid per invocation rather than per session, and the entire reason the tiering is worth the complexity.

How we proved it#

The measurable claim is that moving content down a tier does not lose it. Across four refusals from the line cap in one session:

runtime          199 / 200 lines
sub-modules      4, each one cohesive topic
steps removed    0
commands removed 0

Nothing was deleted; four coherent documents exist that would not otherwise have been written.

And the tier-scoping measurement, which is the one that surprised me. A shared instruction bundle loaded by every generator was scoped so each worker gets only its project type's material:

before   305,942 B
after    280,963 B
saved    ~6.2k tokens per worker  (8.2%)

I had estimated 40%. The real figure is 8.2% — worth keeping, and nowhere near the size that would have justified the restructuring I was contemplating on the strength of the estimate.

The same audit found the inverse failure, which is the one to watch for: a guide the generators genuinely needed had never been in the bundle at all, because the glob that assembled it matched a naming convention that file did not follow. Absent since creation, and nobody noticed — a missing instruction does not announce itself, the generator simply improvises.

The honest limitation: every tier boundary is a bet that the lower tier will be fetched when needed, and a bet you lose silently. Over-trimming produces a competent-looking result built without something, and there is no error. The mitigation is to make each bundle declare its contents and let consumers assert what they require — but that only catches surfaces someone thought to declare.

Where the distinction stops being clean#

The framing above suggests skills and tools are different kinds of thing. In practice the boundary moves, and the direction it moves is informative.

Anything that becomes fully mechanical wants to be a tool. A skill whose instructions stop containing judgement — do these steps, in this order, always — is a script with extra steps, and turning it into a callable drops its context cost to zero and makes its behaviour deterministic.

The signal is watching the same decision get made the same way repeatedly. Judgement that has hardened into a habit is a function waiting to be written, and moving it out of the skill tier makes room for the judgement that is still real.

What this generalizes to#

Sort instruction content by the question it answers — decide, execute, handle the hard case, understand — and load each at the moment that question arises. Put a hard numeric limit on the tier that loads unconditionally, because everything else is conditional and the pressure only goes one way.

Then make each bundle declare what it contains, so a consumer can assert on it. Over-loading is visible in the token count and annoying; under-loading is invisible and produces confident work built on something that was never there.