A flaky test is a timing assumption you didn't know you made

Day 86 of 100. Flakiness has one dominant cause in Compose tests, and one family of fixes.

The symptom

A test that passes locally and fails in CI:

@Test fun ordersLoad() = runComposeUiTest {
    setContent { OrderScreen(viewModel) }
    Thread.sleep(500)                                   // "give it time to load"
    onNodeWithText("Order #4711").assertIsDisplayed()
}

On a fast machine, 500ms is plenty. On a loaded CI runner it sometimes isn't, so the test fails perhaps one run in twenty. Someone raises the sleep to 1500ms, the flakiness becomes rarer, and the suite gets three seconds slower per test.

Nobody has fixed anything; the failure has been made less frequent.

Why the obvious fix fails

The obvious fix is a longer sleep, then a retry rule, then @FlakyTest.

Each step makes the signal worse. A retried test hides a real intermittent bug just as effectively as it hides a timing artefact, and a suite with retries stops being evidence about anything.

More to the point, the sleep was usually unnecessary. Compose tests synchronise automatically, and the sleep was covering for something the framework couldn't see.

The actual mechanism

Before every finder and assertion, the test framework waits until the composition is idle — no pending recompositions, no pending layout, no pending draw, and no running animations.

That's why most tests need no waiting:

@Test fun ordersLoad() = runComposeUiTest {
    setContent { OrderScreen(fakeViewModel) }
    onNodeWithText("Order #4711").assertIsDisplayed()     // already synchronised
}

The assertion doesn't run until the composition settles. A Thread.sleep before it is at best redundant.

Idleness is tracked through a set of registered IdlingResources, and the framework registers the Compose ones for you. Everything below is about the cases where that tracking has a gap or never completes.

Cause 1: something is never idle

The most common, and Day 66's subject: an infinite animation means the composition never reports idle, so every implicit wait times out.

mainClock.autoAdvance = false

With the clock stopped, the animation advances only when you say so, and idle-waiting stops being relevant. Then step time explicitly:

mainClock.advanceTimeByFrame()    // let one frame happen
mainClock.advanceTimeBy(300)      // past an animation

A shimmer placeholder, a pulsing dot or a progress spinner on the screen under test is enough to trigger this, and the timeout message never mentions animation — which is why it costs people an afternoon the first time.

Cause 2: work the framework can't see

Compose knows about its own recomposition. It doesn't know about your coroutine hitting a network, a Handler.postDelayed, or a background thread.

Two fixes, in order of preference:

Make the work synchronous in tests. An injected test dispatcher is the honest approach:

@Test fun ordersLoad() = runComposeUiTest {
    val dispatcher = StandardTestDispatcher()
    val viewModel = OrderViewModel(repo = FakeRepo(), dispatcher = dispatcher)
    setContent { OrderScreen(viewModel) }

    dispatcher.scheduler.advanceUntilIdle()      // run the pending coroutines
    onNodeWithText("Order #4711").assertIsDisplayed()
}

No waiting at all — the work completes deterministically because the test controls when coroutines run. This is the fix that removes flakiness rather than reducing it.

Register an idling resource, when the work genuinely can't be made synchronous:

val resource = object : IdlingResource {
    override val isIdleNow: Boolean get() = !repository.hasInFlightRequests
}
composeTestRule.registerIdlingResource(resource)

Now the framework's automatic waiting includes your work, and the tests stay sleep-free.

The same mechanism is what makes Espresso and Compose tests interoperate: both consult the registry, so an idling resource registered for a Retrofit call serves whichever framework is asserting. In a mixed codebase that's worth knowing before writing a second one.

Cause 3: a genuinely asynchronous wait

Sometimes you're testing something that legitimately takes an unknown time — an integration test against a real service, say. waitUntil is the tool, and it's still better than a sleep:

composeTestRule.waitUntil(timeoutMillis = 5_000) {
    onAllNodesWithText("Order #4711").fetchSemanticsNodes().isNotEmpty()
}

It polls a condition and returns as soon as it's true, so it's fast in the common case and only slow when something is wrong. waitUntilExactlyOneExists, waitUntilAtLeastOneExists and waitUntilDoesNotExist are the readable shorthands for the common conditions.

A waitUntil in a tier-2 test is a smell — it means the test isn't controlling its dependencies. In a tier-3 integration test it's legitimate.

The rule

In order:

1. Nothing. Most tests need no synchronisation at all.
2. A test dispatcher, if the screen does asynchronous work.
3. mainClock.autoAdvance = false, if animation is involved.
4. An idling resource, if there's background work you can't control.
5. waitUntil, for genuinely unknown timing.
6. Thread.sleep — never. There is no case where it's the right answer.

Two smaller traps

waitForIdle() doesn't help with a never-idle screen. It's the explicit form of the automatic wait, so it has the same failure mode. Reaching for it when a test times out is the natural move and the wrong one; the clock is the fix.

State changed from the test thread needs a frame. Setting a mutableStateOf directly in a test schedules a recomposition rather than performing one:

visible = false
mainClock.advanceTimeByFrame()      // needed when autoAdvance is off
onNodeWithTag("banner").assertDoesNotExist()

With autoAdvance on this is handled; with it off, the missing frame advance produces a test that asserts against the previous state and fails confusingly.

How to prove it

The direct test for flakiness is repetition. Run the suspect test two hundred times:

./gradlew :app:testDebugUnitTest --tests "*.OrderScreenTest" --rerun-tasks

A timing-dependent test fails somewhere in that run. A properly synchronised one doesn't, because there's no timing left for it to depend on.

The cheaper signal: grep the suite for Thread.sleep and waitUntil. Every hit is a place where the test is guessing, and each one is a candidate flake.

A lint rule banning Thread.sleep in test sources is worth adding once. It's the kind of thing that creeps back in under deadline pressure, and a build failure is a cheaper conversation than a flaky suite six months later.

What this generalizes to

The principle is determinism comes from controlling inputs, and time is an input — Day 66's line, and it holds across the whole testing pillar.

A test that waits is a test whose result depends on the machine it ran on. Every technique above replaces a wait with control: a test dispatcher controls when coroutines run, the test clock controls when frames happen, an idling resource makes hidden work visible to the framework. The flakiness disappears not because the waiting got better but because there is nothing left to wait for.

It's worth noticing that this is the same move as Day 62's animation argument, one level up. There, replacing a scripted journey with a declared target removed the need to handle interruption. Here, replacing an elapsed-time assumption with an explicit clock removes the need to handle slowness. Both work by deleting the thing that could vary rather than by accommodating it.

Tomorrow, Day 87 closes the testing pillar with the patterns — what a good Compose test suite actually contains.

---

Day 86 of a 100-day series on Jetpack Compose, working through the official documentation in order. Source: Testing synchronisation.