adb install -r doesn't kill your app, so your screenshot is a lie

A fix went in, the app was reinstalled, a screenshot was taken, and the screenshot showed the bug still present. The reasonable conclusion — the fix did not work — was wrong.

The screenshot was of the previous build.

The case this post is about#

One fix, one reinstall, one screenshot:

./gradlew :app:installDebug
adb exec-out screencap -p > shot.png

Two commands, both successful, and the image shows the bug the fix removed. The reasonable reading is that the fix did not work.

What actually happened is that installDebug replaced the APK on disk without killing the running process, so the app on screen was still the composition drawn by the previous build. The screenshot is a true photograph of a stale process, and nothing in a PNG records which APK drew it.

That is the property that makes this worth a post rather than a footnote: the evidence is unfalsifiable. A screenshot cannot be wrong about what it shows, and it cannot tell you what it is showing.

The symptom#

Here is the loop, and there is nothing obviously wrong with it:

./gradlew :app:installDebug
adb exec-out screencap -p > shot.png

Install, capture, look. It is the fastest feedback loop on Android and everybody uses it.

The problem is what the image does not contain. A screenshot is pixels. There is nothing in it that identifies the build that drew it — no version stamp, no commit, no timestamp that means anything. So when the picture disagrees with your expectation, the image cannot tell you which of two very different things happened:

• the new code ran and the fix is wrong
• the old code is still on screen and the fix is untested

Those demand opposite responses. Debug the fix, or debug the deployment. And the evidence is equally consistent with both, which makes it evidence for neither.

Why the obvious fix failed#

The obvious fix is to wait longer. Sleep a second, capture again. It helps often enough to be dangerous — it converts a reproducible wrong answer into an intermittent one.

The second attempt is to watch the install output. installDebug prints Installed on 1 device, and that statement is true and useless: it reports that the APK landed, which was never the question. The question is which composition is on the display.

The third, and the one I defended longest, is to trust that a reinstall restarts the app. Sometimes it does. Whether the process survives depends on the install path, the flags, and what the launcher does next, and I have watched the same command produce both outcomes on the same device on the same afternoon. That is the worst possible property for a verification step: not broken, just unreliable, so it passes review and fails in production.

Notice the shape. Every one of those attempts tries to make the screenshot more trustworthy. None of them puts anything in the screenshot that says which build it is.

The actual mechanism#

Two changes, and neither is about waiting.

Force-stop before capturing, so the process cannot be the old one:

adb shell am force-stop com.example.app
./gradlew :app:installDebug
adb shell monkey -p com.example.app -c android.intent.category.LAUNCHER 1
adb exec-out screencap -p > shot.png

force-stop is not politeness, it is the thing that makes the capture mean something. After it, there is no surviving process that could render an older composition.

Then prove the APK on the device is the APK you built, which is the part that closes the loop:

DEV=$(adb shell pm path com.example.app | sed 's/^package://' | tr -d '\r')
adb shell md5sum "$DEV"
md5 -q app/build/outputs/apk/debug/app-debug.apk

Two hashes. If they differ, the screenshot is about something you did not build, and no amount of staring at it will help. If they match, the image is admissible.

That is the whole idea: the screenshot gains build identity from beside it, because it cannot carry any itself.

The fix#

What this became, as a rule rather than a habit: a claim of fixed about anything on screen requires a fresh capture taken after a force-stop, with the device APK hash matching the built one, read back and confirmed.

The clause that does the work is the ordering. A capture taken before the force-stop is not weaker evidence, it is not evidence — and the distinction matters because the two images look

identical.

As a script, each step earning its place:

pkg="com.example.app"; apk="app/build/outputs/apk/debug/app-debug.apk"

install_apk "$apk"

# 1. The APK ON THE DEVICE must be the one just built. A failed install leaves the
#    old APK in place and reports nothing useful.
built=$(hash_file "$apk")
remote_path=$(device_path_for "$pkg")
on_device=$(device_hash "$remote_path")
[ "$built" = "$on_device" ] || { echo "HALT apk-mismatch: $built != $on_device"; exit 1; }

# 2. FORCE-STOP, not relaunch. Reinstalling does not kill a running process, so a
#    capture without this shows the OLD composition and looks completely fine.
stop_app "$pkg"; start_app "$pkg"; settle 3

# 3. The capture must be FRESH. A stale host-side file passes every check above.
capture_screen > /tmp/after.png
[ -s /tmp/after.png ] || { echo "HALT empty-capture"; exit 1; }
echo "verified on device: $on_device"

Against a device where the install silently failed:

$ ./verify-on-device.sh
HALT apk-mismatch: 6f3a…c91d != b0e7…2a44

Without step 1 that run screenshots the previous build, the screenshot looks right, and the fix gets reported as verified.

Each of those three steps exists because it shipped a false "fixed" first. The hash check came after an install failed on a full device and the next capture showed the previous build. The force-stop came after a reinstall left the old process running, which is the default behaviour and reads as a successful install in every log. The size check came after a capture produced a zero-byte file that every downstream step treated as an image.

What they have in common is that none of them fails loudly on its own. A failed install, a surviving process, an empty capture — all three produce ordinary-looking output, and all three end with a screenshot that somebody reads as proof. The script is not defensive about unlikely events; it is specific about three that each happened.

The honest limit is that this proves the APK on the device is the one just built and the capture is fresh. It says nothing about whether what is on screen is correct — that is still a person looking at the image, or an assertion about what should be visible. What it removes is the much more common failure where the image was never of the thing anybody thought.

There is a broader habit underneath the three steps, and it is the one worth taking away: when a piece of evidence is produced by a chain of operations, the evidence is only as good as the weakest link that could have failed silently. A screenshot is not a primitive observation — it is the end of a chain that includes building, transferring, installing, launching and capturing, and four of those five can fail in ways that still produce an image.

That reframing changes what you check. The instinct is to scrutinise the image: is the layout right, is the colour right, did the state render. Those are the questions you want to be asking, and they are only meaningful once you have established that the image is of the build you think it is. The cheap checks come first not because they are more important but because they are the preconditions for the expensive judgement being about anything at all.

It also explains why this class of mistake is so persistent among people who are being careful. Every individual step reported success. The install printed a success line, the launch worked, the capture wrote a file. Reading those three results and concluding the screenshot is current is not sloppy reasoning — it is correct reasoning from evidence that happens to be about the wrong thing, and the only defence is to check the specific property rather than the chain of successes. identical. You cannot tell a good screenshot from a bad one by looking at it. Only the procedure that produced it separates them, which is exactly why it has to be written down rather than remembered.

One thing I got wrong on the first pass: I wrote the rule as "rebuild clean if in doubt". That is expensive advice. Tearing down the Gradle daemon costs 15–30 minutes of cold reconfigure and recompile, so "just rebuild to be safe" is a tax heavy enough that people skip it and go back to guessing. The hash comparison is under a second and answers the same question, which is why it replaced the advice entirely.

How we proved it#

The direct test is whether the two paths disagree, so I ran them against a build where the fix was genuinely present.

Capture without the force-stop:

fix visible: NO

Same device, same APK, capture after am force-stop:

fix visible: YES

The code was correct the whole time. The first capture was a photograph of a process that started before the change existed.

And the hash check, on a run where the install had silently not replaced the package:

device: 4a9f2c1e8b7d3f06a5c2e91b4d7a8f30
built:  b7e14d09fa62c8531e0d4a7b29c6f815

Different. Every screenshot from that session was about a build nobody had on disk, and nothing in any of those images would have told me so.

The honest limitation: this proves the binary on the device matches the one you built. It does not prove the screen you are looking at is the screen your change affects. A capture of the wrong route, taken correctly, passes every check here — the hash matches, the process is fresh, and the image is of a different screen. That needs a separate assertion, and the cheap one is to require a testTag from the route under test to be present in the capture.

The same bug where there is no device at all#

I hit this again today, in a place with no adb anywhere near it, which is what convinced me the lesson is not about Android.

A static site generator writes its output to content/, a build step turns that into dist/, and an SEO gate then audits the result. The gate reported:

✅ E1 all 113 post(s) declare og:image

112 posts had just been generated. The gate was reading dist/ — the previous build — because it ran before the rebuild. Every statement it made was true about yesterday's output and silent about today's.

That is the screenshot bug exactly. A verification sampled the state at a moment when the change was not yet in it, and reported a pass with total confidence, because nothing in dist/ says when it was made either.

The same session had the same shape a third time: a post whose body was the literal placeholder Placeholder body — this post is scaffolded, not written was serving HTTP 200 on the live site. The file said status: draft, the build ignored status: entirely, and a comment in the file claimed the preflight gate would catch it. The gate never ran, and the comment was reassuring enough that nobody checked whether it had.

Three instances, one shape: the evidence was collected from the wrong moment, and carried no timestamp that would reveal it.

What this generalizes to#

Any artifact you inspect to confirm a change — a screenshot, a build directory, a cached response, a dashboard, a log tail — was produced at some moment, and usually does not say which. If that moment could precede your change, the artifact cannot distinguish "my fix is wrong" from "I am looking at the old thing", and those two failures need opposite work.

So before trusting the inspection, force the state to be new (kill the process, clear the directory, bust the cache) and then attach an identity to what you captured (a hash, a version, a build id). One makes the artifact fresh. The other lets you prove it, later, to someone who was not there.