Updating a model without shipping an app release
Shipping a model outside the store release means versioning, integrity, rollback and a census of what is actually running — and a human gate on the promote.
On-Device AI
Models that run on the phone — Gemini Nano, ML Kit, Core ML, and the latency and privacy maths behind them.
21 posts
An AI feature that still works on a plane
Reading offline is the easy half. The write path is where it breaks: stale cold starts, unresolved conflicts, unorchestrated retries and torn pages.
The on-device privacy argument, stated honestly
Running the model locally is not a privacy guarantee — it is a precondition for one. Only a mechanism that refuses by default makes the boundary real.
Prompt injection does not go away because the model is local
A denylist of three dangerous commands was bypassed by a fourth. Why injection defence has to be allowlist-shaped, and what on-device quietly takes away.
Quantization has a quality cliff, and you can see it
Quantization degrades non-linearly, so an average score hides the cliff. Find it with a suite that fails per-case, and keep the per-case results, not the mean.
Streaming tokens into Compose without recomposing the world
Streaming output updates state far faster than a normal screen. Narrow what reads the changing value, and let the smallest possible scope recompose.
How do you test a screen whose text is generated?
Snapshot tests die the moment output varies per run. Asserting invariants instead of strings, the vacuous-check trap, and a verdict that means could-not-determine.
What a small on-device model simply cannot do
A small local model has a hard ceiling, and the design question is what happens at it. Find the boundary deliberately, then build the escalation path first.
The latency maths that decides on-device versus an API call
Comparing a local model to an API is not a race between two averages — it is tail latency, failure rates, and how often one task crosses the boundary.