An AI feature that still works on a plane

Making a feature readable offline takes an afternoon. Cache the last response, show it when the request fails, done.

The afternoon is real and the problem is not the one you solved. Offline-first stops being a caching exercise the instant the user changes something while disconnected, and every hard part lives on that side.

The case this post is about#

One note, in four states, resolved against the server copy:

a: Winner(value=Note(id=a, body=typed offline, ...), why=local has unsynced edits, remote is older)
b: NeedsUser(local=Note(id=b, body=typed offline, ...), remote=Note(id=b, body=from server, ...))
c: Winner(value=Note(id=c, body=from server, ...), why=remote is newer, local is clean)
d: Winner(value=Note(id=d, body=synced, ...), why=local is current)

Row b is the case this post exists for. Both copies changed since the last sync, and a bare upsert(remote) resolves all four rows to the server's version — correct for c, harmless for a and d, and for b it replaces text somebody typed on a train with a version they have never seen.

This post follows those four rows, because offline-first reads as a caching problem right up until the user writes something, and then every one of the five ways the cache and the writes disagree is structural.

The symptom#

The feature works beautifully on the train. The content is there, the screen renders, nothing spins.

Then the user acts — edits a note, deletes an item, marks something done — and the system has to hold two beliefs at once: what the server last told you, and what the user has since done that the server has not heard about. Everything difficult follows from that pair.

The failure is rarely a crash. It is the user's change appearing, then silently reverting when connectivity returns, because a refresh overwrote local state with a server response that predates the edit. From the user's side, the app forgot what they did. From the logs' side, nothing went wrong.

Why the obvious fix failed#

The obvious fix is try-the-network-then-fall-back-to-local, which I have written about failing in another context for exactly the same reason: it conflates we are offline with this request was broken, and it makes the degraded path the default without telling anybody.

The second attempt is a write queue — park the mutations, replay them on reconnect. Right shape, and it quietly assumes the five things that actually bite:

• that the replay order is the order the user intended
• that nothing on the server changed underneath in the meantime
• that a replay failure is visible rather than a silently dropped entry
• that the local read path shows the queued change before it is confirmed
• that two devices doing this do not produce a merge nobody specified

Each is a design decision. Skipping one does not produce a compile error — it produces a feature that works on your desk and loses data on a train.

The actual mechanism#

I can be concrete here, because the five failures are machine-checked in this codebase. They are checked because all five shipped to production first.

The cold start reads stale and cannot ask for fresh. A repository offering stream() and nothing else gives a caller no way to say I want this re-fetched. The app opens, shows what was in the cache, and has no vocabulary for refreshing it. The fix is structural: a stream() for the cached path and a streamFresh() beside it, so the call site chooses rather than inherits.

Writes bypass the store. A mutation that goes straight to the network, with the cache updated separately, means the two can diverge — and they do, exactly when the network call half-succeeds. Every write goes through the same object that owns the reads, or you have two sources of truth with no arbiter.

Conflicts have no resolver. Offline writes and server changes will collide, and last write wins is a decision you should make deliberately rather than inherit from whichever code path ran last. A mutable store without a named conflict resolver is not resolving conflicts; it is having them silently.

Retry has no orchestrator. A queued write needs something that watches connectivity, enumerates the failed syncs, and backs off between attempts. Without it, retries are either immediate (and hammer a flaky connection) or never (and the queue is a data graveyard).

Pagination is not atomic. Replacing a page of cached results is several row operations. If they are not in one transaction, a reader can observe the list mid-swap — half the old page, half the new, briefly, and reliably enough that users report it as flicker.

The fix#

Two of those five carry most of the weight, and they are the two that are invisible until a user loses work.

Make the read path state its intent. stream() versus streamFresh() looks like API noise and is the difference between a screen that can recover from staleness and one that cannot. The call site knows whether this is a cold open, a pull-to-refresh or a background update; the repository never does.

Name the conflict policy in a type. Not a comment, not an implicit ordering — an object whose job is to take the local version and the remote version and return the winner. Once it exists, the policy is reviewable and changeable. Until it exists, the policy is whatever the last upsert happened to do, and nobody can tell you what that is without reading the call stack.

The other three are checklist items once you know to look: writes through the store, an orchestrator with backoff, @Transaction around the page swap.

The two that carry the weight are both about making an implicit thing nameable. The read intent, and the policy that picks a winner:

enum class Freshness { CachedOk, MustBeFresh }

interface ConflictPolicy<T> {
    fun resolve(local: T, remote: T): Resolution<T>
}

sealed interface Resolution<out T> {
    data class Winner<T>(val value: T, val why: String) : Resolution<T>
    data class NeedsUser<T>(val local: T, val remote: T, val why: String) : Resolution<T>
}

data class Note(val id: String, val body: String, val updatedAt: Long, val dirty: Boolean)

// A reviewable object rather than whatever the last upsert happened to do.
class LocalEditsWin : ConflictPolicy<Note> {
    override fun resolve(local: Note, remote: Note): Resolution<Note> = when {
        // The case that loses work if the policy is implicit: unsynced local edits
        // against a newer remote. Silent last-write-wins deletes what the user typed.
        local.dirty && remote.updatedAt > local.updatedAt ->
            Resolution.NeedsUser(local, remote, "both changed since the last sync")
        local.dirty -> Resolution.Winner(local, "local has unsynced edits, remote is older")
        remote.updatedAt > local.updatedAt -> Resolution.Winner(remote, "remote is newer, local is clean")
        else -> Resolution.Winner(local, "local is current")
    }
}

class NoteRepository(private val dao: NoteDao, private val api: Api, private val policy: ConflictPolicy<Note>) {

    // The CALL SITE knows whether this is a cold open or a pull-to-refresh.
    // The repository never does, which is why the intent has to be a parameter.
    fun stream(id: String, freshness: Freshness): Flow<Note> = flow {
        val local = dao.get(id)
        if (local != null) emit(local)
        if (freshness == Freshness.CachedOk && local != null) return@flow

        val remote = api.fetch(id)
        when (val r = if (local == null) Resolution.Winner(remote, "no local copy") else policy.resolve(local, remote)) {
            is Resolution.Winner -> { dao.upsert(r.value); emit(r.value) }
            is Resolution.NeedsUser -> emit(local!!)      // keep the user's text on screen
        }
    }
}

The four states a note can be in, run through the policy:

val p = LocalEditsWin()
listOf(
    Note("a", "typed offline", 1700, dirty = true)  to Note("a", "from server", 1600, false),
    Note("b", "typed offline", 1700, dirty = true)  to Note("b", "from server", 1900, false),
    Note("c", "synced",        1700, dirty = false) to Note("c", "from server", 1900, false),
    Note("d", "synced",        1900, dirty = false) to Note("d", "from server", 1700, false),
).forEach { (l, r) -> println("${l.id}: ${p.resolve(l, r)}") }

a: Winner(value=Note(id=a, body=typed offline, ...), why=local has unsynced edits, remote is older)
b: NeedsUser(local=Note(id=b, body=typed offline, ...), remote=Note(id=b, body=from server, ...), why=both changed since the last sync)
c: Winner(value=Note(id=c, body=from server, ...), why=remote is newer, local is clean)
d: Winner(value=Note(id=d, body=synced, ...), why=local is current)

Row b is the one an implicit policy gets wrong, and it gets it wrong silently. A bare upsert(remote) resolves every one of these four rows to the remote copy, which is correct for c, harmless for a and d, and for b replaces text the user typed on a train with a version they have never seen. NeedsUser is not a nicer outcome — it is the only outcome that does not lose anything.

How we proved it#

The evidence is that all five are now gates, and the gates exist because the defects shipped.

cold-start-stale        stream() with no streamFresh()
mutations-bypass-store  write path that skips the owner of reads
conflict-unresolved     mutable store with no ConflictResolver
sync-unorchestrated     no networkMonitor + failed-sync enumeration + backoff
page-not-atomic         replacePage() without @Transaction

Each is a structural property a script can check — does this file declare that symbol, is this annotation within three lines of that function — rather than a behaviour someone has to reproduce. That matters because the behaviours only appear with real latency, real connectivity loss and real concurrent edits, which is precisely the state a test environment does not have.

The honest limitation of that approach: a structural check confirms the mechanism is present, not that it is correct. A ConflictResolver that always returns the remote version satisfies the gate and still loses the user's work. The gate moves the failure from nobody thought about conflicts to somebody made a conflict decision you can go and read, which is a smaller problem but not the same as a solved one.

What this has to do with a model#

The boundary, stated plainly: this is measured on data sync, not on an on-device model, and the inference half I have not shipped.

What transfers unchanged is everything above — a feature whose output the user can edit, annotate or discard has the same write path and the same five failures, whether the content came from a server or from a local model.

What is genuinely different is that a local model makes the generation available offline too, which removes the most visible failure and leaves every one of these intact. That is worth saying because it cuts against the intuition: shipping the model on-device makes the feature look more robust while changing nothing about the part that actually loses data. The spinner disappears; the conflict does not.

The piece I would expect to be new is cost. A queued write is a few bytes and a retry is cheap. A queued generation is seconds of compute and a battery draw, so a retry policy tuned for network calls is wrong for inference — and "replay the queue on reconnect" stops being obviously correct when each replay is expensive and the user may no longer want the result.

What this generalizes to#

Offline-first is a property of the write path. Reading from a cache is the part you finish first and the part that was never going to lose anything.

Give the read path a way to ask for fresh data, route every write through the thing that owns the reads, name the conflict policy in a type you can open, orchestrate retries against a connectivity signal with backoff, and wrap the page swap in a transaction. Then check those structurally, because the behaviours that violate them need a bad connection and a second device to reproduce — and the build that ships on Tuesday will not have either.