6.3.11 Remote Output Availability
A remote action can have a perfectly published result and still fail the person or local tool that needs its files. 6.3.7 Remote Execution Storage established the required starting point: the ActionResult exists and every output blob or tree it names is reachable through the intended remote path. This page starts there. The decision is whether to bring those bytes to this machine now, leave them remote until a supported local access layer needs them, or deliberately accept that a later consumer must obtain them another way.
Download policy is a consumer contract
For Bazel 9.1.0, bazel help build lists --remote_download_outputs=all|minimal|toplevel, with toplevel as the default. The versioned Bazel 9.0 reference documents the same choices and semantics.1 These modes govern local materialization after a remote result; they neither redo an action nor change the result that was published remotely.
| Choice | What Bazel makes local | Good first fit | What you still must prove |
|---|---|---|---|
| all | All remote outputs | A workstation or job that must work without another remote read after the build | Local disk capacity and that the required result really was downloaded |
| toplevel | The minimal set, plus outputs of the requested top-level targets | A normal developer build where the requested artifact is the usual next thing to inspect or run | Generated intermediates, debug companions, and non-top-level files used by a tool are present too |
| minimal | Only remote outputs required by local actions | A transfer-sensitive build whose local graph is the only immediate consumer | Any human, IDE, debugger, test viewer, or script that reads an output later has another access path |
minimal does not mean “no files will ever be downloaded”: a local action can require an earlier remote output, and that required output is materialized. Likewise, toplevel is not “all final-looking files.” It adds outputs of the targets named on the command line; it does not infer every adjacent artifact a future tool might want. --remote_download_regex is a targeted exception: it forces paths matching a supplied pattern to download regardless of the selected mode.2 Use it to make a known small compatibility set local, not as a growing, unexplained inventory of every tool's accidental dependencies.
The three choices are Bazel download modes. An on-demand filesystem is a different arrangement. In 9.1.0, --remote_download_symlink_template tells Bazel to create symbolic links whose target can contain the blob hash and size; the command reference explicitly gives a FUSE filesystem that loads CAS objects on demand as an example.2 That is a supported handoff point, not a promise that an arbitrary symlink or FUSE mount can read an output. The local service must resolve the link, authenticate to the right remote store, and fetch the named object while it remains available. BuildBarn's Remote Output Service is one implementation pattern: it makes bazel-out look populated while a client-side virtual filesystem fetches bytes when they are opened.3
Choose from the next consumer, not a fleet-wide default
Write down what will read each result after the build and test the actual read. A green Bazel invocation only proves that its own required steps had what they needed. It does not prove that a later consumer has a local file, valid credentials, or a readable remote blob.
| Consumer after the invocation | A sound starting choice | Evidence to collect before reducing downloads |
|---|---|---|
| IDE indexer or generated-source browser | all, a precisely downloaded generated-source set, or a qualified on-demand service | Open or index a generated file through the real editor process, including a cold local cache. The toplevel mode is insufficient unless that file is actually a top-level output. |
| Debugger | all, an explicit small set, or qualified on-demand access | Launch the debugger and read the executable, runfiles, symbols, generated source, and any format-specific companion it follows. Do not assume the executable alone is enough. |
| Test-result viewer or local failure investigation | The files that the viewer really follows, eagerly downloaded or available through the qualified access service | Read the test log, XML, undeclared output, and attachment paths the tool reports. 1.2.7 Test Output & Debugging explains ordinary Bazel test-log locations and controls; this check is about their availability after remote execution. |
| CI job step that consumes one artifact | Eagerly materialize that artifact or use a service the step has been qualified against | Run the real step under its CI identity and in the expected network boundary. The distributed question of whether all CI outcomes and artifacts were collected is 6.5.8 Complete CI Results with BEP/BES, not a consequence of this flag. |
| Human inspection or a script after bazel build | toplevel when the requested target really is the object of inspection; otherwise all, a documented regex exception, or on-demand access | Open the named path after the build, including the cold-cache and restricted-network case that matters to the workflow. |
| A later local Bazel action | minimal is normally enough for this graph consumer | Trace the local action and confirm which remote outputs Bazel materialized for it. This says nothing about files read outside the graph. |
The table is deliberately about consumers rather than artifact formats. A container-image build, for example, can use metadata-oriented outputs to avoid materializing large layer blobs remotely, while a downstream tool may still need those blobs through a different access path.4 Apply the format-specific checks for your debugger, IDE, or packaging tool. Here the portable question is simply: which named paths must be readable by this consumer, through which path, at which time?
Remote access has an expiry and identity boundary
Remote availability is not a local durability guarantee. For retention, eviction, replication, and repair policy, see 6.2.6 Remote Cache Lifecycle. The authorization that allowed the build client to read a CAS object may also differ from the identity used by an editor, sidecar, CI step, or a later developer session. A remote-output service can reduce eager transfer, but it does not erase either boundary.
For each required consumer, record a small access contract:
- the output path and its result/blob or tree digest;
- the consumer identity and remote endpoint it is allowed to use;
- the expected remote retention window and the point at which offline use is required; and
- the recovery action if the local copy and the remote object are unavailable.
That last line decides whether you must materialize eagerly. If a release investigation, debugger session on an airplane, or later CI step cannot tolerate a new remote read, copy the required artifact while it is reachable to the durable, authorized location that workflow uses. Do not call an on-demand link an offline artifact. Conversely, do not keep every output forever merely to avoid stating the consumer and retention requirement; that turns all into an unbounded policy rather than a justified choice.
A missing blob is evidence, not a download-mode result
When a consumer fails to open a remote output, first retain the action/result identity, output path, blob or tree digest, consumer identity, endpoint, authorization result, selected output mode, and whether a local copy existed. Then test the first boundary that can distinguish the problem:
| Observation | Classify first as | Immediate response |
|---|---|---|
| The path was never selected for local materialization, but the authorized CAS read succeeds | A consumer-policy gap | Materialize the needed path now and update the consumer matrix or qualified on-demand configuration. |
| A symlink or on-demand service cannot authenticate | An access-identity or local-service failure | Repair the identity/service path; changing minimal to all only helps if Bazel can read the object during the invocation. |
| The CAS read returns a missing blob or tree for an already-published result | A storage-retention, visibility, or dangling-result incident | Preserve the digests and hand the remote publication evidence to 6.3.7 Remote Execution Storage; do not describe it as an IDE bug. |
| The remote object has expired and no required local/durable copy remains | A recovery decision | Restore it from the storage system if possible, or reexecute only when the action is safe and the workflow accepts a newly produced result. |
Reexecution can make an output available again, but it is not proof that the expired bytes were retained, nor a substitute for an artifact that had to remain available. Preserve the original missing-digest evidence before retrying. 6.3.13 Remote Action Failure Handling covers retry and fallback policy. This article's narrower result is a clear answer to whether a consumer needs an eager copy, qualified remote access, or an explicit recovery path.
Decide: A developer uses minimal successfully, then opens a generated source file from an IDE after disconnecting from the network. The file was not needed by any local action. What should the team change first: the remote executor, the download policy, or the storage retention setting?
Reveal
Start with the consumer contract. The file was outside minimal's local-action exception, so the immediate failure is an unfulfilled IDE/offline requirement. Choose eager materialization of that file or a set containing it before going offline; an on-demand service is only suitable while its remote read and authorization path exist. Then check retention if the chosen design depends on later remote access. Executor behavior is not implicated because the result had already been published.
Output downloading begins only after a remote result is reachable. In Bazel 9.1.0, all materializes every remote output, minimal materializes the ones needed by local actions, and the default toplevel adds outputs of requested top-level targets. Use regexes only as deliberate exceptions. A symlink-template handoff can support on-demand access, but the external service—not the link alone—must supply authorization and a still-retained CAS object.
Pick the policy from the next consumer and prove it by opening the required paths under realistic identity, cache, network, and retention conditions. If the workflow needs files after remote access can expire or disappear, eagerly materialize or durably copy them before that boundary. A missing blob is storage or publication evidence first; reexecution is a controlled recovery option, not evidence that the original result was available.
Check your understanding · 3 questions
1.Match each output choice to its Bazel 9.1.0 behavior:
Drag each answer onto the matching prompt, or click an answer and then click a prompt
2.A CI step must read an artifact after the build, but its service identity cannot read the remote CAS object. Which fact should it establish before changing download modes?
Select one answer
3.Classify these claims about remote output access:
Choose True or False for each sentence
Footnotes
-
Remote Output Download Modes in Bazel 9.0 — version-pinned all, minimal, and toplevel semantics; the identical option set and default were verified with this repository's Bazel 9.1.0 bazel help build. ↩
-
Command-Line Reference — remote download modes, aliases, regex override, and symlink-template behavior. ↩1 ↩2
-
Remote Output Service - How not to have your bytes and eat them too — a client-side virtual-filesystem implementation that fetches output bytes on access. ↩
-
rules_img: A Ground-Up Redesign of Container Image Building for Bazel — image outputs illustrate metadata-versus-large-blob materialization and downstream compatibility trade-offs. ↩