Core ML and LiteRT solve the same problem differently
A platform-native runtime and a portable one differ in ways the type system will not catch. What the shared declaration may promise is the whole design.
All posts
On-device semantic search over the user's own data
Semantic search on a phone is not a vector-database problem at user scale — it is an index-freshness problem, invisible until someone searches.
Designing the fallback for a device that cannot run your model
A fallback that silently became the default for 303 of 304 artifacts. Detect capability before attempting, and record on every artifact which path ran.
Gemini Nano on-device: what it actually costs you
On-device inference has no per-call bill, and that hides six real costs — eligibility, size, battery, thermal, capability ceiling and operations.
Routing between a local model and a remote one
Routing between a local and a hosted model needs a decision you can audit. Route before attempting, record the path on every result, and watch the proportion.
One inference API, two platform runtimes
Designing an expect/actual seam so platform differences stay on one side. What the shared declaration must not promise, and where the abstraction leaks.
How to know your on-device model got worse
A tiny eval suite that fails the build, the budget that bounds it, and why a verdict needs a third value meaning neither pass nor fail but undetermined.
Wiring ML Kit GenAI into a Compose screen without blocking it
Wiring an on-device GenAI capability into Compose is mostly state design. Model the availability and failure states before the happy path, not after.
Shipping a model in your APK without breaking the size budget
Bundling a model trades download size for a guarantee. The decision turns on first-run experience, and the budget needs a gate rather than good intentions.
Updating a model without shipping an app release
Shipping a model outside the store release means versioning, integrity, rollback and a census of what is actually running — and a human gate on the promote.
An AI feature that still works on a plane
Reading offline is the easy half. The write path is where it breaks: stale cold starts, unresolved conflicts, unorchestrated retries and torn pages.
The on-device privacy argument, stated honestly
Running the model locally is not a privacy guarantee — it is a precondition for one. Only a mechanism that refuses by default makes the boundary real.