A bigger context window gave me a worse answer, repeatably
What you leave out of a context pack is a design decision, not waste.
ai
46 posts
Running a model on-device is a memory decision first
Quantisation, mmap and the low-memory killer decide whether your 3B model ships.
Streaming a response changes your UI contract, not just your latency
Partial output means every intermediate state is user-visible, including the wrong ones.
Context is a budget, and what you load first wins
Loading everything relevant is not a strategy. Measure tokens per decision.
The retrieval worked and returned the wrong file
Semantic similarity is not relevance. A test file and its subject are near-identical in embedding space.
The agent fixed the test, not the bug, and the test was right
Given a failing assertion, weakening it is the shortest path to green.
The eval that passed because the function returned a plausible stub
An LLM judge scoring output shape rather than behaviour will approve a stub.
An agent loop without a terminal condition is a budget with no bottom
Declare the halt classes before the loop runs. 'Until it looks done' is not a condition.
Your agent's tool output is untrusted input
A file the agent reads can contain instructions. The fix has the same shape as SQL injection's.
One warm context beats five cold ones
Per-layer workers each pay full context assembly. Loading once and generating inline amortises it — and the measured saving was smaller than estimated.
A step that never ran is not a step that passed
A pipeline that reports green with steps that never ran has two states where it needs three. Not-run must block completion or it silently becomes success.
Is this derived thing current? Make it a comparison
Staleness checked by timestamp or by memory fails quietly. A digest of the declared inputs, stamped beside the output, makes currency a comparison.