5.6.3 Bisecting Dependency Issues

A dependency upgrade can fail hundreds of targets while the actual regression is one commit in a ruleset. Do not compare releases by repeatedly editing MODULE.bazel or WORKSPACE. Clone the dependency once, keep Bazel pointed at that clone, and let git bisect check out candidate commits in it.

The useful result is not merely “version 0.53 is bad.” It is the smallest suspect commit range that a dependency maintainer can inspect, revert, or fix.

Choose the override that matches dependency resolution

For a module selected through Bzlmod, override the module name. If module resolution or migration is unfamiliar, M1 WORKSPACE → Bzlmod supplies that background:

bazel test --override_module=rules_go=/work/deps/rules_go //app:all

--override_module=<module-name>=<path> replaces the selected module with the module at that local path. An absolute path is the least ambiguous choice in an automated bisect. Relative paths are interpreted from the command's current working directory. A path beginning with %workspace% is interpreted from the main workspace root.¹

Do not substitute --override_repository for a registry module when candidate commits may change its MODULE.bazel. A repository override replaces repository contents, but Bazel's command reference warns that changes to the overridden module's MODULE.bazel do not take effect when the module came from a registry. --override_module is the module-aware operation.¹

For a dependency declared by the legacy WORKSPACE mechanism, override the repository name visible from the main repository:

bazel test --override_repository=io_bazel_rules_go=/work/deps/rules_go //app:all

The two names may differ: rules_go is a Bzlmod module name, while io_bazel_rules_go is the legacy repository name in this example. Verify the name used by the consumer instead of deriving one from the Git repository name. The same local clone can serve both forms when a project still exercises both dependency models.^1,2

Prove the predicate before bisecting

A bisect is valid only if one command classifies commits reliably. Reduce the original failure to the smallest build, test, or query that still reproduces it, but keep the same Bazel version, configuration, target platform, and relevant environment.

Before starting, check both endpoints through the override:

cd /work/consumer
bazel test --override_module=rules_go=/work/deps/rules_go //app:upgrade_regression_test

Check out the known-good dependency revision, run the command, then repeat at the known-bad revision. The good revision must return zero and the bad revision must return nonzero for the reason under investigation. If the command sometimes fails for unrelated reasons, fix that instability or teach the driver to return 125 for an untestable commit. A flaky predicate produces a precise-looking but unreliable answer.²

Also inspect the candidate range before adding --first-parent. That option is appropriate only when the regression is known to lie on the first-parent history you intend to search. Otherwise it can exclude the responsible commit.

Run a guarded automatic bisect

Keep the driver outside the dependency clone: git bisect changes that clone's checkout, so a script stored only in the candidate history can disappear or change semantics. This driver is intentionally conservative:

#!/usr/bin/env bash
set -u

consumer=/work/consumer
dependency=/work/deps/rules_go

if [[ ! -d "$consumer" || ! -d "$dependency/.git" ]]; then
  printf '%s\n' 'consumer or dependency checkout is missing' >&2
  exit 125
fi

cd "$consumer" || exit 125

bazel test \
  --override_module="rules_go=$dependency" \
  //app:upgrade_regression_test
status=$?

case "$status" in
  0) exit 0 ;;       # good
  125) exit 125 ;;   # preserve git bisect's skip signal
  *) exit 1 ;;       # bad: the predicate reproduced
esac

Replace the module flag with --override_repository="io_bazel_rules_go=$dependency" for the legacy WORKSPACE case. More importantly, do not automatically map every nonzero Bazel result to “bad” unless the chosen command has no plausible infrastructure failure. If network, credentials, or an unsupported intermediate commit can fail independently, recognize those conditions explicitly and return 125. Unexpected failures should stop for inspection rather than silently bias the search.²

After verifying the driver once at each endpoint, start in the dependency clone:

cd /work/deps/rules_go
git bisect start v0.53.0 v0.51.0
git bisect run /work/consumer/tools/bisect-rules-go.sh

git bisect start BAD GOOD checks out candidates between the two endpoints. The driver follows git bisect run's convention: zero marks a candidate good, a normal nonzero result marks it bad, and 125 skips a commit that cannot be tested. When the search finishes, save git show --stat BISECT_HEAD and the reproducing command, then restore the dependency checkout with:

git bisect reset

The first reported bad commit is a lead, not proof of causality. Re-run its parent and the commit itself, confirm that the observed failure is the original one, and inspect whether generated files, submodules, patches, or lockfile state require additional setup. If many candidates were skipped, Git may report an ambiguous set rather than one definitive commit.²

Preserve warm state, but do not promise a warm analysis

There is usually no reason to run bazel clean between dependency candidates. Keeping the same Bazel server and the same override flag and path allows Bazel to retain unaffected in-memory state and action outputs. That can make a bisect substantially faster than starting cold every time.²

However, a checkout changes files inside an external repository. Bazel must notice those changes, re-evaluate affected repository and package state, and re-analyze configured targets that depend on the changed definitions. A change to a widely loaded .bzl file, a module extension, dependency metadata, toolchain registration, or generated repository can invalidate a large portion of the graph. Warm-server reuse is therefore an opportunity, not a guarantee that “only the changed dependency parts” will be analyzed.

Avoid clean --expunge unless stale external state is itself part of the controlled hypothesis. It shuts down the server and destroys precisely the state and cached outputs that can accelerate repeated trials. If the failure depends on repository fetching rather than checked-out source contents, design that as a separate experiment. The source's Bazel-version bisect used explicit cleanup for such a download failure, while its local dependency bisect deliberately retained the server.²