6.5.6 Fail-Closed Target Selection

Target selection saves CI work only when a narrow result is trustworthy. When the selector cannot establish that trust, the safe response is not an empty target list or a warning that the pipeline ignores. It is an explicit unknown outcome that routes the invocation to a complete, protected fallback and leaves evidence that the fallback actually ran.

Make Uncertainty a Routing Decision

6.5.5 Incomplete Selector Evidence identifies the cases that cannot support a reliable negative result: stale or incompatible snapshots, missing mappings, and incomplete evidence for deletes or renames. The CI orchestrator must now convert that state into work. The preceding selector-evidence-lab and selector-invalidators fixtures make those typed evidence verdicts reproducible; neither fixture claims to perform the CI routing below.

Keep the routing table small and explicit:

Selector outcome	CI route	What the route establishes
affected	Run the returned roots within the declared universe.	The selector found evidence of impact.
unaffected	Skip only the work covered by that universe.	The supported comparison completed without finding impact.
unknown	Run the protected full scope.	Missing selector evidence cannot create a false green.

The protected full scope is the complete set of roots and configurations that this CI lane is required to validate. It is not necessarily //..., and Bazel does not choose it for you. The technical contract needs its identity as an input; H.7.3 Evidence Selection determines the organization-level protection policy that supplies its contents.

A fail-closed response can therefore look like this:

{
  "selector_outcome": "unknown",
  "reason": "base ownership snapshot is unavailable",
  "requested_universe": "presubmit-linux-v3",
  "route": "protected-full-scope",
  "fallback_manifest": "<immutable-manifest-id>"
}

This is an interface sketch, not a required schema. The important property is that the same record binds the selector failure to the fallback work. A pipeline that logs “falling back” but loses the fallback job, narrows its configuration, or never reconciles its results still fails open in practice.

Target Determinator provides one concrete example of conservative behavior: its before-revision query failure policy defaults to returning the eligible candidates from the after revision's configured query rather than silently producing a narrow result.¹ That is an implementation-specific fallback, not a universal proof. You still need to check whether those candidates cover deletes, the requested configurations, and the protected roots in your service contract.

Observe the Fallback as a Service Path

A fallback that is never measured tends to decay. Record at least:

• the selector request, outcome, reason, implementation, and version;
• the protected-scope manifest chosen for the fallback;
• whether every fallback partition started and reached a terminal result;
• selector and fallback label sets for later comparison;
• fallback rate and duration by reason, repository, configuration, and selector version.

Alert on both absence and excess. No observed fallbacks may mean the failure fixtures stopped reaching the production route. A sustained increase can indicate stale snapshots, an unsupported change class, or a broken selector deployment. Availability pressure must not turn either condition into “skip and continue.”

Seed a failure that the contract requires to become unknown, such as withholding the base mapping. The test passes only when the protected-scope manifest is selected and its completion is reconciled. A unit test of the selector's return value is insufficient because the dangerous defect can live in orchestration after the selector responds.

The runnable CI fallback reconciliation project isolates that service boundary. Its typed cases distinguish a completed fallback, a missing fallback partition, a failed protected test, and an incomplete shadow lane; its router rejects every path that would turn those gaps into green. It is a deterministic contract model, not a replacement for a CI orchestrator or a Bazel selector.

Validate Narrow Results in Shadow

Fail-closed routing protects known uncertainty. It does not reveal false confidence: cases where a selector says affected or unaffected but omits work it should have selected. Find those omissions by running an independent comparison lane.

Three comparison modes answer different questions:

Mode	Operation	Strength of evidence
Full comparison	Run the selector and protected full scope for the same request.	Measures omissions for every request in the cohort.
Shadow comparison	Let the selector control visible CI while the broader lane runs without gating that decision.	Measures drift continuously, provided shadow completion is retained.
Sampled comparison	Run the broader lane for a declared sample of requests.	Estimates behavior for that sample; it cannot prove zero false negatives.

For each completed comparison, compute set differences before collapsing results into rates:

omitted = full_scope_failures - selector_scope_failures
extra   = selector_scope_work - work_needed_by_the_comparison_model

The first set contains the safety signal: failures observed outside the selected work. Preserve the labels, revision pair, universe, configurations, selector explanation, and full-lane result so the case can become a regression fixture. The second set is an efficiency signal; it can motivate better precision, but reducing it must not weaken the first guarantee.

Shadow data is evidence only when its own completeness is checked. A cancelled shadow lane, an incompatible revision, or a missing result is unknown comparison evidence, not a successful validation of the selector. 6.5.7 CI Work Distribution continues with partitioning the established work set; distribution must not quietly change the comparison universe.

Compare Implementations Through One Contract

Tool selection begins with the service contract from this section, not with an algorithm or product name. Two maintained implementations illustrate why. bazel-diff generates and compares hashes from two Bazel graphs and can use query or cquery; its CI scripts orchestrate the two-revision files and consume machine-readable impacted targets.² Target Determinator compares configured-query results and documents both conservative query-failure handling and cache-key boundaries: user and system bazelrc files, environment variables, and host hardware are not represented in its result-cache key.³

Neither description is a blanket guarantee. Evaluate each candidate with the same matrix:

Contract dimension	Evidence to require
Revision and universe identity	Immutable base/head identities, roots, configurations, and implementation/schema version
Configuration coverage	Declared behavior for flags, platforms, toolchains, transitions, and top-level roots
Deletes and renames	A two-sided fixture that selects former consumers or returns unknown
Stale or missing inputs	An explicit incompatible/unknown result and a proven fallback route
Explanations	Inspectable reasons for selected, omitted, broadened, and unsupported cases
Operational behavior	Machine-readable output, bounded failure modes, cache identity, monitoring, and replayable fixtures

Run the same adversarial fixture suite against every candidate and against upgrades of the chosen one. Include source edits, build-metadata invalidators, a configuration-only edge, deletion, rename, stale base data, missing mappings, and incompatible versions. A product passes a row only when its versioned behavior either returns a complete affected set for the declared universe or refuses the claim and activates the conservative route.

Graph distance deserves a separate classification. bazel-diff can report target and package distance, and Aurora used package distance as a cost-versus-risk dial for expensive CI jobs.⁴ Distance filtering deliberately chooses not to run some transitively affected work. It can be a policy input, but it is not a correctness-preserving affected-target algorithm. Keep its omissions visible in comparison lanes and do not relabel a distance cutoff as “unaffected.”

Historical savings and false-negative observations remain case-study measurements. They do not transfer to a repository with different graph shape, test ownership, configurations, or protected scope. Re-establish them with your cohorts and retain the raw comparison evidence.

Promote Only with Negative-Path Evidence

A selector is ready to control CI when all of these are true:

1. Its supported universe and evidence inputs are versioned.
2. Every unsupported, stale, or incompatible case has an explicit outcome.
3. A seeded unknown reaches and completes the protected fallback.
4. Adversarial delete, rename, configuration, transition, and invalidator fixtures never become silent negative results.
5. Full, shadow, or sampled comparisons retain omissions and incomplete comparison evidence.
6. Upgrade and rollback criteria use the contract matrix rather than a product version or aggregate savings claim.

This is a technical qualification gate. Decisions about which scopes must always run and how much measured risk is acceptable belong to the organization operating the monorepo. This implementation must keep uncertainty visible, make conservative work executable, and let every candidate be challenged with reproducible evidence.