6.7.2 Operating Bazel Builds in Air-Gapped Environments

extra
Air-gap network architecture, internal artifact and container registries, and offline credential distribution.

An air-gapped Bazel build is not merely a build with downloads disabled. It is a proof that every input needed before and during the build can be obtained through approved internal paths, starting from fresh state. That distinction matters: a warm developer machine can succeed offline while hiding missing registry metadata, archives, generated repositories, toolchains, or container images.

Define the closure you intend to support

Start with a named build surface, not with the claim that “the repository builds offline.” Record the target patterns, Bazel version, platforms, configurations, and subcommands that the environment must support. Repository discovery is configuration-sensitive, and some Bazel subcommands use tool dependencies that ordinary build targets do not reach.1

For each supported surface, enumerate the dependency classes that must cross the air-gap boundary:

Dependency classEvidence of closure
Module resolutionRequired registry metadata is present internally and the selected module graph is reproducible
Repository downloadsEvery required archive is available through an approved mirror, downloader, cache, or vendor directory and passes its declared integrity check
Generated repositoriesTheir repository rules can complete with only approved inputs; rules marked local = True or configure = True are provisioned and proved separately because vendor mode excludes them
ToolchainsEvery platform selected by the supported configurations can resolve and acquire its tools without public access
ImagesBuilder, worker, and target/runtime images are available in internal registries under the identities expected by the build and infrastructure

This inventory extends the intake mechanisms in 6.7.1 Serving Dependencies Inside the Organization. A registry answers module-metadata queries; a mirror serves objects; a repository cache reuses fetched material; vendor mode keeps selected repositories local. An air-gapped design composes these mechanisms and then proves that their union covers the declared build surface.

A lockfile is valuable evidence of resolution state, but it is not the closure itself. It does not, by itself, make every referenced object available, cover configuration-dependent repository work, or place required images in an internal registry. Likewise, build outputs do not prove that repository fetches can be repeated from empty state.

Choose a supply path, then remove the escape hatch

Vendor mode is useful when the desired closure should travel with the workspace. For targets selected under a particular configuration, bazel vendor gathers the external repositories Bazel discovers, except repositories whose rules declare local = True or configure = True; Bazel always excludes those from vendoring. Bazel also vendors registry files fetched during module resolution. A subsequent build can use that directory without a repository cache or network access.1

That convenience has precise limits. Changing targets, configuration, external dependencies, or the Bazel version may require re-vendoring. bazel vendor //... does not include every implicit tool used by Bazel subcommands; the official workflow explicitly adds @bazel_tools//tools:tools_for_bazel_subcommands when those commands must operate offline.1 These are reasons to define several tested closure profiles rather than bless one vendor-directory snapshot as universally complete.

The existing vendor-mode proof demonstrates this bounded claim. Its assertion script vendors an archive-backed dependency, removes the original archive, and builds with a fresh output base and empty repository cache. The captured terminal result is:

fresh build resolved the removed dependency archive from vendor_src

That result proves one vendored repository path. It does not prove public egress denial, service credentials, toolchain or image closure, or repositories excluded by local = True or configure = True. Provision the host inputs for those excluded repository rules explicitly, record their identities in the closure contract, and test them from the same disposable environment rather than counting them as vendor coverage.

At organization scale, internal registries and mirrors may be a better operating boundary than checking in a large vendor directory. A production case study used a four-stage loop: catalog dependencies, mirror missing objects to internal blob storage, verify with public access blocked, and handle exceptional URLs through a downloader policy.2 The exact WORKSPACE-era cataloging implementation is not a current universal recipe, but the operational loop remains useful: discover, materialize, isolate, and repair the inventory.

Whichever supply path you choose, enforce the air gap independently of Bazel. Network policy should deny public egress from the test environment. Bazel-side download policy can add a second control and clearer diagnostics, but a forgotten fallback flag must not be able to invalidate the proof. Permit only named internal registries, mirrors, credential services, and container registries.

Prove closure from fresh state

Run the closure test in a disposable worker or namespace with no reusable Bazel output base, repository cache, home-directory cache, or pre-pulled image layer unless that layer is explicitly part of the supported base image. Provide only the committed repository state, the declared offline material, scoped credentials, and the Bazel binary or launcher included in the closure contract.

Then exercise every supported profile:

  1. resolve modules and repositories;
  2. run required maintenance subcommands, if they are in scope;
  3. analyze, build, and test the declared target patterns for each supported configuration and platform;
  4. start any required execution workers and pull every required image;
  5. preserve fetch, authentication, integrity, and network-denial evidence.

The test should fail if it attempts public access even when an internal copy later makes the build green. Otherwise a race or retry can hide a policy violation. A case study's blocked-internet CI job was the step that exposed URLs missed by its catalog and mirror process.2

Treat closure as versioned evidence. Store the Bazel version, selected profiles, dependency inventory, internal object identities, and test result together. When a module, toolchain, base image, configuration, platform, or Bazel version changes, invalidate the affected proof and regenerate it. This turns “offline worked once” into a reviewable service contract.

Make that contract inspectable. A deployment-neutral record can define what the environment must prove without pretending that a repository fixture can enforce the organization's network boundary:

profile: linux-release
bazel_version: 9.1.0
targets: [//app/...]
platforms: [//platforms:linux_x86_64]
configurations: [--config=release]
subcommands: [build, test, mod_tidy]
supplied_inputs:
  registry: {service: registry.internal, snapshot: sha256:...}
  archives: {inventory: sha256:...}
  host_repositories: {inventory: sha256:...}
  toolchains: {inventory: sha256:...}
  images: {inventory: sha256:...}
network_policy_evidence:
  status: pass
  evidence_uri: run://airgap/linux-release/egress-denials
positive_result:
  status: pass
  evidence_uri: run://airgap/linux-release/build
negative_controls:
  - case: missing_mirrored_archive
    injected_object: archive:sha256:...
    expected_boundary: mirror.internal
    observed_result: fetch_failed_without_origin_fallback
    evidence_uri: run://airgap/linux-release/missing-archive
    status: pass

The digests and evidence URI above are schema examples, not captured values. In production, generate them from the deployment under test. Keep the positive run separate from each negative control: name the injected case and object, expected failure boundary, observed result, evidence URI, and case status. Retain the network-policy decision log alongside the Bazel invocation. A portable Bazel workspace can prove vendor and integrity behavior, but it cannot prove that a CI namespace, VM firewall, proxy, credential service, or container registry had the intended policy. That evidence must come from the enforcing platform.

Test missing objects and failed trust boundaries

A complete happy-path run proves availability only for the state tested. Remove one required object at a time to prove that failures remain visible and correctly located.

InjectionRequired outcome
Registry metadata absentResolution fails at the internal metadata boundary; no public registry is consulted
Mirrored archive absentFetch fails with the missing internal object identified; no origin fallback occurs
Mirrored bytes alteredIntegrity verification rejects them
Toolchain repository or image absentThe relevant configuration fails rather than selecting an undeclared host tool or public image
Credential missing or scoped to the wrong serviceAuthentication fails without exposing the secret or silently changing route
Vendored marker staleThe test catches any attempted refetch instead of accepting a warning followed by public access
Provisioned input for a local or configure repository absentRepository evaluation fails at the named host-input boundary rather than consulting public services or an undeclared host tool

The last case follows directly from vendor mode's behavior: Bazel uses a vendored repository when its marker is current or the repository is pinned; otherwise it warns and falls back to fetching the current repository version.1 In a real air gap that fetch should fail, but the test should report it as stale vendored state rather than as a mysterious network outage.

Keep outage behavior equally explicit. Decide whether builds may continue when an internal mirror, registry, or credential service is unavailable and identify the approved fallback, if any. A second internal replica can preserve the air-gap contract. A public origin cannot. Reliability and integrity are separate axes: an archive available from two mirrors is still unusable if neither copy matches the declared digest, while a correct digest does not make a forbidden public route acceptable.

3.1.9 Offline / Air-Gapped Builds introduces the developer-facing offline workflow. At the shared-service layer, the deliverable is stronger: a versioned inventory, a fresh-state isolated run, deliberate missing-object tests, and evidence that fallback cannot cross the trust boundary.

key takeaway

An air-gapped Bazel service is complete only for a declared set of targets, configurations, platforms, subcommands, toolchains, and images. Registries, mirrors, repository caches, and vendor mode can supply pieces of that closure; neither a lockfile, a warm cache, nor existing build outputs prove the whole path.

Prove closure in a fresh environment whose public egress is independently denied. Record what was tested, invalidate that proof when relevant inputs change, and remove registry metadata, archives, toolchains, images, and credentials one at a time. Safe operation means every missing or corrupted input fails at its internal boundary and no unnoticed public fallback can turn the build green.

Check your understanding · 4 questions

1.A closure profile vendors every repository reached by its build targets, but one toolchain repository rule declares configure = True. What is the safe conclusion?

Select one answer

2.Which conditions belong in a credible fresh-state closure test?

Select all that apply

3.Match each closure-record field to the question it answers:

Drag each answer onto the matching prompt, or click an answer and then click a prompt

Answers
Targets, configurations, platforms, and subcommands
Supplied-input inventories and immutable identities
Network-policy status and evidence URI
Negative control: case/object, expected boundary, observed result, evidence URI, and status

4.Classify these claims about air-gap and vendor evidence:

Choose True or False for each sentence

A repository-contained vendor fixture can prove that an organization's CI firewall denied every public route.
A vendor fixture can prove a bounded dependency path from a fresh output base and empty repository cache.
A correct archive digest makes a public-origin fallback acceptable in an air-gapped profile.
Missing platform evidence leaves the network-boundary result unknown, even when Bazel's build result is green.
0 of 4 answered

Footnotes

  1. Vendor Mode — target- and configuration-sensitive vendoring, subcommand tools, registry files, marker freshness, and fetch fallback 1 2 3 4

  2. How we build without the public internet — dependency cataloging, internal mirroring, blocked-egress verification, and downloader-policy exceptions 1 2