Engineering

The module that compiled and was never installed

A dependency module nobody included compiles cleanly and fails at runtime. Declaration and installation are separate acts and only one of them is checked.

MobileByteSensei 10 min read
A rank fully present, correctly voiced, and cut off from its wind
kotlinarchitecturetooling

A dependency injection module is declared. It lists its bindings, it is syntactically perfect, and every type it mentions exists. The compiler is entirely satisfied with it.

It is not in the list of modules the application starts with, so none of it is real.

The case this post is about

One dependency module, declared correctly and never included in the graph:

$ ./gradlew verifyDiGraph
  declared but NOT installed: profileModule  <- fails at first inject
  installed twice: networkModule             <- later binding silently wins
FAILED

profileModule compiled, its bindings were right, its file was reviewed. Nothing referenced it from the graph root, so at runtime the first injection into the profile screen threw — on a build that had been green the entire time.

The second line is the one that is worse. A module installed twice does not throw at all; the later binding wins, silently, and which one that is depends on include order. This post follows both, because the compiler cannot see either and the test that would catch them is a test nobody writes for wiring.

The symptom

The build is green and the app crashes on a screen. Not at launch — on the one screen that needs something from the module nobody installed, which may be three taps deep and may not be the screen anybody opens while developing the feature next to it.

The error says a dependency could not be resolved. That reads as a missing binding, so the first instinct is to go and look at the module — where you find the binding, correctly declared, exactly as expected. Several minutes go past before anybody asks whether the module is loaded at all, because the file in front of you is right.

The one knob sitting flush, identical to its neighbours in every other way

The second thing is that this is undetectable by the type system in principle, not by oversight. Declaring a module and installing it are two separate acts, and a declaration that nothing references is just an unused value. The compiler has no basis for complaining — it cannot know that this particular unused value was meant to be wired into a runtime graph.

The third is where it tends to happen: the second and subsequent modules in a feature. The first one gets installed because somebody was setting up the feature and had the startup list open. The fourth one, added months later for a sub-feature, is written by somebody looking at the third module as a template — and the third module does not contain its own installation.

The fourth is that it survives tests. Unit tests construct what they need directly; the module list is an application-startup concern. A feature can have thorough coverage and a module that has never once been loaded, because nothing in the test path goes through the startup graph.

Why the obvious fix failed

Remember to add it. The convention that produced the bug. It depends on somebody holding, at the moment they create a file, a fact about a different file in a different module — which is precisely the shape of thing people do not hold.

Fail fast at startup by resolving everything eagerly. Tempting, and it converts a deep-screen crash into a launch crash, which is genuinely better. It does not help with the module that is missing, because an eagerly-resolved graph only knows about the modules it was given — it will confidently resolve everything it has and still not contain the absent one.

Write a test that starts the real graph. This is the real answer for resolution failures and it has the same blind spot. The test starts the application's module list; if a module is missing from that list, the test starts the same incomplete graph the app does, and passes — unless the test happens to request something from the missing module, which is the case you did not know to write.

One giant module. Avoid the problem by never having more than one thing to install. It works and it gives up module boundaries, which exist for good reasons — and it merely delays the issue until the module is large enough that somebody splits it.

The actual mechanism

Make installation structural rather than remembered: the module cannot exist without being included, because the inclusion is derived from the declaration.

The same move as any other registry problem. If a processor can see the declaration, it can emit the installation, and the two can no longer disagree. A module that is declared is a module that is installed, by construction, because there is no separate act to forget.

One trunk, and every rank fed from it rather than from its own supply

Where generation is not available, the fallback is a check rather than a convention: enumerate the declared modules, enumerate the installed ones, and fail the build on any declared module that is not installed. That is a structural comparison between two lists, both of which can be derived from the source, and it needs no judgement.

The sub-case that determines whether the check is worth anything: it must enumerate from the code, not from a manifest. A check that compares the installed list against a hand-maintained list of what should be installed is the same problem one level up — two lists, both maintained, both able to drift. The enumeration has to come from finding the declarations themselves.

There is a related failure the same check catches, which is the opposite one: a module that is installed and no longer exists, or is installed twice. Those are less dangerous and more confusing, and they are free once the comparison exists.

The fix

derive      installation from declaration where a processor can see it
or check    declared modules vs installed modules, enumerated from source
fail        at build time, naming the module
cover       the reverse case: installed but absent, or installed twice

Both directions from one comparison — declared modules from the annotation processor, installed ones parsed from the graph root:

abstract class VerifyDiGraphTask : DefaultTask() {
    @get:InputFiles abstract val sources: ConfigurableFileCollection
    @get:InputFile  abstract val graphRoot: RegularFileProperty

    @TaskAction
    fun verify() {
        val declared = sources.asFileTree.matching { include("**/*.kt") }
            .flatMap { DECLARES.findAll(it.readText()).map { m -> m.groupValues[1] } }
            .toSet()

        val installedList = INSTALLS.findAll(graphRoot.get().asFile.readText())
            .map { it.groupValues[1] }.toList()
        val installed = installedList.toSet()

        val missing = (declared - installed).sorted()
        val orphan  = (installed - declared).sorted()
        val twice   = installedList.groupingBy { it }.eachCount().filter { it.value > 1 }.keys

        if (missing.isEmpty() && orphan.isEmpty() && twice.isEmpty()) return
        throw GradleException(buildString {
            missing.forEach { appendLine("  declared but NOT installed: $it  <- fails at first inject") }
            orphan.forEach  { appendLine("  installed but not declared: $it  <- stale include") }
            twice.forEach   { appendLine("  installed twice: $it            <- later binding silently wins") }
        })
    }
}

A module written, annotated, never wired:

$ ./gradlew verifyDiGraph
  declared but NOT installed: profileModule  <- fails at first inject
  installed twice: networkModule             <- later binding silently wins
FAILED

The second line is the subtler one. A module included twice does not error — the later binding wins, so an override nobody wrote on purpose is in effect, and the only symptom is behaviour that disagrees with the code somebody is reading.

The point of the build-time failure is that it moves the discovery from "a user found a screen that crashes" to "the build said so" — and the module name is in the message, which removes the several minutes of looking at a file that turns out to be correct.

This is cheap to implement and it is the kind of check that is easy to not get around to, because the failure it prevents is intermittent and someone always just fixed the last one by hand.

There is a weaker version of this worth knowing about, because it is available immediately and requires no tooling: make the startup list impossible to write incompletely by construction. If every feature exposes exactly one module and the application's list is assembled from the feature list it already maintains for navigation or dependencies, then adding a feature adds its module, and a feature with no module is a compile error rather than a silent omission.

That does not cover the second module within a feature — which is where the problem actually concentrates — so it is a partial answer. It is worth mentioning because it costs nothing and narrows the window: the first module of each feature stops being forgettable, which removes the easiest cases and leaves the harder ones visible.

How we proved it

The compiler's silence is the demonstrable part. Declare a module, do not install it, build. The build succeeds. That one observation is the whole case: there is no type-level relationship between declaring and installing, so no amount of care in the type system will catch it.

The structural check is a comparison of two enumerable sets, which means it is exact rather than heuristic. Either a declared module appears in the installed list or it does not, and both lists are derivable from source. There is no tuning, no false-positive class, and nothing to maintain.

Five declarations, uniform at the top, and one with nothing live beneath it

What I cannot tell you is how many times this has actually bitten. I know the shape of the failure and I have not kept a count of incidents. The honest framing is that this is a check built from reasoning about a mechanism rather than from a tally of outages, and its value is that the failure it catches is silent at build time and loud at the worst possible moment.

What I have not solved: conditional installation. Some modules are legitimately installed only in certain configurations — a debug-only module, a platform-specific one. A naive declared-versus- installed check flags those as missing, and the fix is either an annotation marking them conditional or an exclusion list, both of which are a hand-maintained thing of the sort this post is arguing against. Mine uses an exclusion list, which is small and is a real wart.

The test that would have caught it is worth naming even though it is not what I built, because it is the version most codebases can adopt today. A single test that starts the real graph and resolves every binding the application declares — not every binding the graph contains, which is circular, but every binding enumerated from source — turns a missing module into a failing test. It is the same comparison as the build check, executed at test time, and it needs no processor.

I did not take that route because a test that enumerates from source is doing static analysis at runtime, which is an awkward place for it. But a check in an awkward place beats no check, and the difference between the two is much smaller than the difference between either and nothing.

And a limitation of the generated approach: it binds installation to declaration, which is right almost always and removes a choice that is occasionally meaningful. If you genuinely want a module that exists and is not loaded — staged work, a feature behind a flag — generation has made that awkward, and the awkwardness arrives at a moment when somebody is trying to do something reasonable.

The thing that makes this worth a whole post rather than a line in a style guide is the ranking of the three failure modes. A missing dependency binding fails at startup, loudly, naming the type — that is a good bug. A missing registry entry fails when something looks for it, which is later and more confusing. A missing cleanup hook fails never, in the sense that nothing ever reports it, and manifests only as data outliving the session it belonged to.

The same omission, three severities, and the severity is decided entirely by what happens to consult the thing. That is worth knowing before you choose which of them to generate first, because the instinct is to automate the one that annoys you most and the right order is to automate the one that is silent.

What this generalizes to

Declaring a thing and activating it are different acts, and most systems check only the first.

A class that implements an interface and is never registered. A migration that exists and is not in the list. A route defined and not mounted. A scheduled job written and not scheduled. A feature flag read and never created. In every case the artifact is correct, the compiler is satisfied, and the system behaves as though it does not exist — because from the system's point of view it does not.

The tell is any sentence of the form "and then add it to the list". Lists maintained by hand beside code are where this class of bug lives, and the two durable answers are to derive the list from the code or to compare them mechanically. Which one you can have depends on your tooling; having neither is the common case, and it is the one where everything compiles.

One post a week, no digest

New writing when it lands. No roundups, no course pitch, no "10 tips".

Follow by RSS