commonTest runs on every target, which is the point and the cost

TL;DR#

A test in commonTest is not one test. It is one test source that becomes a separate execution per target — five targets, five runs, five chances to disagree.

That is precisely the value: a shared function that behaves differently on Kotlin/Native is caught by a test you wrote once. It is also the cost: a test with an accidental platform assumption — a timing dependency, a locale, a file path, floating-point rounding — fails on two targets and passes on three, and the failure reads like flakiness rather than information.

When a commonTest fails on some targets and not others, that is a result, not noise. Nearly every time, the test found a real difference.

The case this post is about#

A shared formatter with the obvious test:

// commonMain
fun formatRelative(then: Instant, now: Instant): String {
    val minutes = (now - then).inWholeMinutes
    return when {
        minutes < 1  -> "just now"
        minutes < 60 -> "$minutes min ago"
        else         -> "${minutes / 60} h ago"
    }
}

// commonTest
@Test
fun formatsHours() {
    val now = Clock.System.now()                       // ← the problem, unnoticed
    assertEquals("2 h ago", formatRelative(now - 2.hours, now))
}

@Test
fun formatsScore() {
    assertEquals("0.1", (0.1f).toString())             // ← also the problem
}

Both passed locally. Local is jvmTest, which is what ./gradlew test runs by default when you are in a hurry.

The symptom#

CI, on a branch that touched neither function:

./gradlew allTests

jvmTest                 FormatterTest  PASSED
androidUnitTest         FormatterTest  PASSED
iosSimulatorArm64Test   FormatterTest  FAILED
    formatsScore: expected <0.1> but was <0.100000001490116>
wasmJsTest              FormatterTest  FAILED
    formatsScore: expected <0.1> but was <0.1>          ← identical strings?
    formatsHours: expected <2 h ago> but was <1 h ago>

Three things here, and each one is a different lesson.

The iOS failure is a real difference in Float.toString() — Kotlin/Native and the JVM do not produce the same shortest-representation output for every float.

The wasm formatsScore failure prints two strings that look identical. They are not: one is a String from a JS number and the comparison is against a Kotlin Float's representation, and the assertion message renders both the same way. An error message that shows no difference is its own kind of problem.

And formatsHours failed on exactly one target because the test reads the wall clock, so it depends on where the minute boundary falls during the run.

Why the obvious fix failed#

"Those tests are flaky, retry them." The fastest way to lose the information. formatsHours is timing-dependent, so retries make it pass most of the time — and the dependency it is reporting stays in the code.

"Move the failing tests to jvmTest." This is the one I did first and it is the worst option, because it looks like a fix. The tests then cover the one platform that was never broken. I had converted a real finding into permanent blindness, and the commit message said "stabilise tests".

"Pin the float comparison with a delta." Right for a numeric assertion and wrong here — the function returns a string, so a delta does not apply. The assertion was testing Float.toString(), which is a platform behaviour I did not mean to depend on at all.

"Only run allTests on main." Faster PRs, and it defers every platform finding to after merge. Target coverage is the reason the tests are in commonTest; running them later means finding out later.

The actual mechanism#

commonTest is a source set, not a test suite. Each target's test compilation includes it and produces its own binary, run by its own runner — jvmTest on the JVM, androidUnitTest under Robolectric or plain JUnit, iosSimulatorArm64Test as a native executable in the simulator, wasmJsTest in a browser or Node.

So one @Test function becomes N independent executions, each with a different:

Standard library implementation. Float.toString(), String.format, collection iteration order in some cases, and anything touching text encoding are implemented per platform. They agree on the contract and not always on the exact output.

Numeric behaviour. wasm and JS back numbers differently from the JVM; Float on Kotlin/Native has its own shortest-round-trip printing. Arithmetic that is exact on one target rounds on another.

Environment. Locale, time zone, filesystem presence, and available dispatchers all differ. A test that reads any of them is testing the environment as much as the code.

Concurrency model. runTest behaves consistently, but a test that spawns real threads has nothing to spawn on wasm.

The useful framing:

A commonTest failure that is target-specific is a measurement. Either the production code differs across platforms — which you wanted to know — or the test has a platform assumption you did not intend to write. Both are findings. Neither is flakiness.

The fix#

Fix the tests' assumptions, and keep every one of them in commonTest.

Inject the clock. Never read it in a test.

// commonTest — no wall clock anywhere
@Test
fun formatsHours() {
    val now = Instant.parse("2026-10-15T12:00:00Z")    // fixed, everywhere
    assertEquals("2 h ago", formatRelative(now - 2.hours, now))
}

This is the same lesson as making the clock injectable in production code, arriving from the test side. A test that reads Clock.System.now() is non-deterministic on every platform; the multi-target run just made it visible by sampling five times.

Do not assert on platform toString(). The real bug was production code leaning on float formatting. Fix it there and the test becomes portable for free:

// commonMain — explicit formatting, identical on every target
fun formatScore(score: Float): String {
    val tenths = (score * 10).roundToInt()
    return "${tenths / 10}.${tenths % 10}"
}

// commonTest
@Test
fun formatsScore() = assertEquals("0.1", formatScore(0.1f))

The iOS and wasm failures were telling me that Float.toString() is not part of my contract. The fix is not a tolerant assertion; it is not depending on it.

When a difference is genuine and intended, assert it per platform. Some things legitimately differ, and there is a seam for that:

// commonTest
expect val expectedDispatcherParallelism: Int

@Test
fun parallelismMatchesPlatform() =
    assertEquals(expectedDispatcherParallelism, currentParallelism())

// jvmTest / androidUnitTest / iosTest: > 1    wasmJsTest: 1

expect/actual works in test source sets, so an intended platform difference is declared rather than worked around — and a new target has to state its value.

Run allTests on PRs; let it be the slow job. With the compile/link split from the build-time work, the test matrix is affordable, and it is the only configuration that gives the coverage the source layout implies.

How we proved it#

The claim to verify is that the fixed tests are genuinely portable, not that they pass today.

./gradlew allTests

jvmTest                 FormatterTest  PASSED   (2 tests)
androidUnitTest         FormatterTest  PASSED   (2 tests)
iosSimulatorArm64Test   FormatterTest  PASSED   (2 tests)
iosArm64Test            FormatterTest  PASSED   (2 tests)
wasmJsTest              FormatterTest  PASSED   (2 tests)

Five green is necessary and not sufficient — passing everywhere could also mean the tests stopped checking anything. So: break the production code and confirm all five notice.

// deliberately: tenths / 10 → tenths
jvmTest  FAILED · androidUnitTest FAILED · iosSimulatorArm64Test FAILED
iosArm64Test FAILED · wasmJsTest FAILED

Five independent executions catching one shared bug. That is the property worth paying for, and it is why the earlier instinct — move the test to jvmTest — was so expensive: it would have left one.

The timing test needed its own check, because "it passes now" is exactly what a flaky test says. Run formatsHours 200 times across the matrix:

before (Clock.System.now)   1,000 executions → 7 failures, all near minute boundaries
after  (fixed Instant)      1,000 executions → 0 failures

Seven in a thousand is the shape of a bug that survives for months: frequent enough to annoy, rare enough to retry away.

And the sub-case that nearly undid all of it: allTests runs the targets your host can build, not the targets you declared. On a Mac the list above is five. The same command on the Linux CI runner is three — the Apple targets are not skipped with a warning, they simply are not in the task graph, and the build is green:

# macOS laptop
./gradlew allTests → jvmTest, androidUnitTest, iosSimulatorArm64Test, iosArm64Test, wasmJsTest

# Linux CI, same commit, same command
./gradlew allTests → jvmTest, androidUnitTest, wasmJsTest          # ← exit 0

Which means the float-printing failure that started this whole post — the one that only ever appeared on Kotlin/Native — cannot fail CI on a Linux runner. It is not caught and deferred; it is not looked for. The green tick is honest about what it ran and silent about what it did not, and those read identically in a PR.

The check is to assert the count rather than the colour. Have CI print the executed target list and compare it against the declared targets, failing when they diverge — the same monoculture problem as moving a test to jvmTest, arriving one layer up where nobody is looking for it.

One further limit worth naming. commonTest gives you the shared source; it does not give you platform integration. A real Keychain, a real Android Context, a real browser API still need iosTest/androidInstrumentedTest. commonTest proves shared logic on every platform's runtime, not that the platform wiring is correct.

What this generalizes to#

A test that passes on some environments and fails on others has found something. The failure is the useful event, and the reflex — retry, quarantine, move it somewhere quieter — destroys the finding. Ask what differs before asking how to make it green.

Determinism is not a test-quality nicety, it is what makes multi-environment testing possible. Wall clocks, locales, default time zones, iteration order: each one turns one test into a different test per environment. Inject them, all of them.

Do not assert on what you did not mean to depend on. Float.toString() was never part of the contract; the assertion made it one. When a cross-platform test fails on formatting, the question is usually whether the production code should be formatting that way at all.

Declare intended differences; never let them be implicit. Where platforms genuinely differ, expect val in the test source set turns an unspoken assumption into a value each target must supply. The new target then cannot be forgotten.

Related#

• A DI module that compiles on every target and only works on one — why the test must live in commonTest
• One Gradle build, five outputs, and the cache that only helps three — making the matrix affordable
• expect/actual is not an interface, and the difference shows up at link time — the seam that makes code testable

Keep reading#

This is part of 100 Days of Kotlin Multiplatform — the boundaries, the sharp edges, and the things that only show up on the second target. Follow MobileByteSensei for the rest.