An agent drove a KMP app to the Play Store. The codegen was the easy half.

Two Kotlin Multiplatform apps reached the Play Store in August, a day apart, with an agent doing the implementation work. That is the headline and it is the least interesting part.

The interesting part is everything that had to be true before "the agent finished" could be allowed to mean anything.

The case this post is about#

Two Kotlin Multiplatform apps, a day apart in August, with an agent doing the implementation work and a Play Store listing at the end of each.

The codegen is not the interesting half. What stands between generated code and a build that does what it claims is a stack of gates, each named after a failure that had already happened:

device truth        a claim needs a capture taken after force-stop, md5-matched
no stubs            NoOp / Placeholder / empty handler bodies fail the build
dead clickables     every on_click reaches a declared target
dead routes         every route is reachable from a declared entry point
design conformance  the rendered screen is checked against the mockup
executed tests      the suite is run, not written

Six gates, six incidents. This post follows the first one — device truth — because it is the gate the other five depend on: every one of them ends in a claim about a running app, and a claim about a running app is worth nothing without proof of which build was running.

The symptom#

An agent completes a feature. The build is green, the files are in the right modules, the code reads well, and the summary says the work is done. The screen, on a device, is a grey box.

Nothing in that sequence malfunctioned. The agent generated a composable with the right name, the right parameters and the right structure; the compiler accepted it; the test it wrote passed. Every check in the path answered its own question correctly, and no check was asking whether the thing renders.

The second thing is how confident the report is. A generated summary says "implemented the settings screen with the three sections from the mockup" because that is what the plan said to do, and the plan is the agent's best evidence about its own output. Self-report is not observation, and a model writing a summary of work it just did is doing recall, not inspection.

The third is that the failures are not random. They cluster: a handler with an empty body, a route registered and never navigated to, a NoOp implementation standing in for a real one, a screen that matches the mockup's structure and none of its content. Each of those is a thing a plausible continuation produces when the context runs out before the intent does.

The fourth is the one that cost the most time. A fix is applied, the app is reinstalled, a screenshot is taken, and the screenshot shows the old screen — because reinstalling over a running app does not kill the process, and the composition on screen is from the previous build. An agent that takes that screenshot as evidence will report a fix that did not happen, and will do it sincerely.

Sincerity is the part worth sitting with, because it rules out the responses that feel natural. There is no deception to detect and no carelessness to correct — the agent followed a reasonable procedure and the procedure had a hole in it. Everything that works from here is about closing the hole rather than about improving the agent.

Why the obvious fix failed#

Review the generated code. Works and does not scale to the volume an agent produces. Reviewing every file the way you would review a colleague's pull request is slower than writing it yourself, which removes the reason for the agent.

Better prompts. Real improvement, and it moves the distribution rather than bounding it. A better instruction produces a better first attempt; it does not establish whether this particular attempt is correct, which is the question that has to be answered every single time.

Trust the build. The build proves the code is well-formed. Nearly every failure described above compiles — that is what makes them the interesting failures. A green build is necessary and carries almost no information about behaviour.

The same is true one level up, for the test suite, and it is a more comfortable illusion. A suite that runs and passes proves that the assertions written hold against the code written. Whether those assertions describe the behaviour anybody wanted is a separate question, and it is not one the suite can be asked.

Have the agent test its own work. The right instinct and it degenerates without an outside constraint, because an agent that writes both the implementation and its test can satisfy both. The test that was authored alongside a stub tends to be a test a stub passes.

The actual mechanism#

The useful frame is that an agent needs an external definition of done that it cannot satisfy by assertion. Every gate below exists because a claim was made that turned out to be false, and each one replaces a sentence with an observation.

device truth        a claim needs a capture taken after force-stop, md5-matched
no stubs            NoOp / Placeholder / empty handler bodies fail the build
dead clickables     every on_click reaches a declared target
dead routes         every route is reachable from a declared entry point
design conformance  the rendered screen is checked against the mockup
executed tests      the suite is run, not written

The first line is the keystone and the least obvious. A capture is only evidence if the process was killed first and the installed binary matches the one just built — otherwise it is a photograph of the previous build, and the agent has produced a sincere false claim. Checking the installed APK's hash against the built one costs a second and converts the whole category from "probably fine" to known.

The sub-case that took longest to get right is design conformance, because the obvious implementation does not work. Scanning the source for the elements the mockup specified passes on a structural shell: the composable is named right, the parameters are there, the states are declared, and it renders a grey circle where the mockup called for a colour-coded status chip with a drill-down. The only check that catches that is a rendered one, and rendering is a much heavier gate than reading.

There is a second-order property that matters more than any individual gate. Once the gates exist, the agent's own behaviour changes — not because it is smarter, but because a blocked claim is feedback it can act on. An agent that cannot say "done" without a capture goes and takes the capture.

The fix#

define    done as a set of observations, never as a report
gate      each failure class that has actually happened, by name
verify    on the device, after force-stop, with a hash match
block     the claim, not just the merge — an unverified "fixed" is the defect

The verify step is the one worth writing out, because every part of it exists to defeat a specific way the previous version lied:

#!/usr/bin/env bash
set -euo pipefail
pkg="com.example.storefront"
apk="app/build/outputs/apk/debug/app-debug.apk"

# 1. The APK ON THE DEVICE must be the one just built. `install -r` is not enough —
#    a failed install leaves the old APK and the next step screenshots it happily.
adb install -r "$apk" >/dev/null
built=$(md5 -q "$apk")
on_device=$(adb shell md5sum "$(adb shell pm path "$pkg" | cut -d: -f2 | tr -d '\r')" | cut -d' ' -f1)
[ "$built" = "$on_device" ] || { echo "HALT apk-mismatch: $built != $on_device"; exit 1; }

# 2. force-stop, NOT just relaunch. `install -r` does not kill a running process, so a
#    screenshot taken without this shows the OLD composition and looks completely fine.
adb shell am force-stop "$pkg"
adb shell am start -n "$pkg/.MainActivity" >/dev/null
sleep 3

# 3. The capture has to be FRESH. A stale file on the host passes every other check here.
adb exec-out screencap -p > /tmp/after.png
[ -s /tmp/after.png ] || { echo "HALT empty-capture"; exit 1; }
echo "verified on device: $on_device"

Each numbered step is a bug that shipped. The hash comparison exists because a failed install is silent; the force-stop exists because install -r leaves the old process running and the screenshot of it is indistinguishable from success; the size check exists because a capture that failed produces a zero-byte file that every downstream step treats as an image.

Run against a device where the install silently failed:

$ ./verify-on-device.sh
HALT apk-mismatch: 6f3a…c91d != b0e7…2a44

Without step 1, that run screenshots the previous build, the screenshot looks correct, and the agent reports the fix as verified.

The third line is the one most teams will not have, because it needs real devices in the loop and that is genuinely inconvenient. Everything above it is static analysis and can be built in an afternoon; the device half is where the remaining failures live, and skipping it leaves exactly the class of bug that a user finds in the first thirty seconds.

The fourth line is the cultural one and the hardest to hold. It is tempting to treat an agent's "I fixed it" as a status update and verify at the end. In practice the unverified claim is itself the defect, because every subsequent decision is made on top of it — and by the time something is found, the context in which it could be cheaply understood is gone.

What I would tell somebody starting this is that the gate stack is not overhead on top of agent work. It is the thing that makes agent work possible at a volume a human could not review, and the apps that shipped did so because the gates said yes, not because the generation was good.

How we proved it#

Two applications reached a store, four days apart in August 2026, with the implementation done by an agent under this gate stack. That is the only outcome evidence I have and it is the one that matters: the gates are not a thought experiment, they were the condition of those releases.

Every gate is traceable to a specific failure. None was designed in advance. The stub detector exists because a NoOp export engine shipped and nothing noticed; the dead-route check exists because nine screens had no caller; the force-stop requirement exists because a screenshot proved a fix that had not happened. The stack is a record of what actually went wrong, which is why I would not expect somebody else's to look the same.

What I cannot tell you is the false-negative rate. I know what the gates caught. I do not know what they let through, because the only instrument for that is a user finding something, and absence of reports is weak evidence. The honest claim is that the gates caught a lot and not that they are sufficient.

What I have not solved: conformance at a fine grain. The rendered check catches a grey box where a chip was specified. It does not catch spacing that is close but wrong, a colour slightly off the token, or an animation that is technically present and feels bad. Those still need a person, and a person looking at a screen for thirty seconds remains the highest-yield check I have.

That is worth saying plainly because it cuts against the direction everything else in this post goes. The gates buy volume: they make it possible to accept work faster than a human could read it, and they catch the failure classes that have actually occurred. They do not replace somebody opening the app with an opinion about whether it is any good, and the apps that shipped had both.

And a cost worth naming plainly: this is slow. A device capture per claim, a rendered conformance pass, an executed suite — the loop is minutes rather than seconds, and there is a real temptation to skip it on a change that is obviously fine. The changes that are obviously fine are not the problem; the problem is that the category "obviously fine" is assigned by the same agent whose claim is in question.

The thing I would do differently is build the device-truth gate first. I built the static checks first because they were easy, and they are genuinely useful, and they would all have passed on the screenshot that was lying. Ordering the work by what is easy to build gets you a stack that is strong exactly where the failures are cheap.

What this generalizes to#

An autonomous worker's report is not evidence about the world, and the whole discipline follows from taking that seriously.

This is not really about agents. It is the same structure as a test suite that was written to pass, a status report assembled from the plan rather than the work, a migration marked complete because the script exited zero. In each case a thing that is downstream of the work is being read as though it were the work, and the gap only shows up when somebody looks at the actual artifact.

So the question worth asking of any automated pipeline is what would still be true if the step had silently done nothing. If the answer is "the logs, the summary and the status" then the pipeline is reporting on itself. The fix is always the same shape: find an observation that is downstream of the real effect rather than of the attempt, and refuse to accept the claim without it.