Bazel Knowledge Hub

2.3.3 Non-Determinism Sources

<a class="cross-ref" href="/book/2~3~2" title="2.3.2 Sandboxing"><span class="cross-ref-id">2.3.2</span> Sandboxing</a> answers one question: did the action read only the files Bazel staged for it? Non-determinism is the next question: if Bazel reruns the same action with the same declared inputs and action definition, does it produce the same bytes again? When the answer is no, the build can look fine on one cached run and then change after a clean build, cache eviction, or execution on another machine.1,2,3

Same action, different bytes

The simplest example is intentionally boring:

genrule(
    name = "date",
    outs = ["date.txt"],
    cmd = "date >$@",
)

That rule is valid, but its output is a function of wall-clock time. Bazel will happily reuse the first result until the action is forced to run again. Then date.txt changes because the tool itself asked the clock, not because Bazel lost track of a declared dependency.1,3

That is the important distinction from 2.3.1 Hermeticity. Hermeticity is about Bazel knowing the real inputs. Non-determinism is about the tool still producing unstable output after those inputs are fixed.1,2

Common ways entropy leaks into outputs

The pattern is always the same: some value inside the action is less stable than the output contract assumes.

  • Build time and file metadata. Code generators, archive writers, and similar tools often stamp the current time or per-file timestamps into outputs. In C/C++, the preprocessor macros __DATE__ and __TIME__ are a classic source: they embed compilation wall-clock time into every translation unit that uses them.1,4,3
  • Host-specific identities and paths. Some tools embed PID, UID, GID, sandbox paths, output-base paths, or other absolute paths, so the bytes differ across runs or machines even when the logical source tree is the same.1,5,6
  • Unstable ordering. Hash-map iteration, unsorted directory walks, and serializers that do not sort their keys can reshuffle emitted output between runs.1,5,3
  • Randomness and live network reads. Random seeds and fetched data make the output depend on changing external state instead of only on declared source inputs.1,4,3

One subtlety: the damage only propagates as far as the changed bytes travel. A later action can still produce stable output from unstable input if it normalizes the data, but any downstream action that hashes the varying bytes will have to re-run when the unstable action re-executes.1

Why sandboxing still matters, but isn't enough

Sandboxing is still essential because undeclared inputs are usually missing from the action's staged working directory, so ordinary relative reads fail instead of silently poisoning the cache. But it does not rewrite a ZIP header to a fixed timestamp, sort a tool's serialized map output, or stop a generator from writing random bytes. Depending on strategy and platform it may also constrain network access differently, so the safe mental model is: sandboxing reduces hidden inputs. Determinism removes unstable behavior inside the action.1,4,7

recommended

If the same commit keeps rebuilding from clean builds or cache reuse never seems to stabilize, treat it as an execution problem, not just a dependency problem. 5.7.2 Execution Log Analysis later covers execution-log and artifact-diff workflows for finding the first action whose outputs diverge between runs, and 4.6 Toolchains & Platform Resolution explains how Bazel makes compiler and SDK selection explicit so machine-by-machine tool differences are no longer mistaken for "the same build."4,3,6

For a concrete paired-run escalation, the pre-release execlog-diff map routes argument, environment, input, and output drift through its tree/sawmill/ comparators and then follows downstream rebuild influence in tree/diff.go. Treat this as evidence for a diagnostic workflow, not as a replacement for Bazel's stable execution-log formats or a guarantee that the pre-release CLI will remain unchanged.

key takeaway

Sandboxing asks whether the action saw only the world Bazel prepared. Determinism asks whether that prepared world is enough to reproduce the same bytes every time. Bazel needs both: one removes hidden inputs, the other keeps outputs stable once those inputs are fixed.1,2,7

Check your understanding · 3 questions

1.Which of the following are common sources of non-determinism in build actions?

Select all that apply

2.What is the key distinction between a hermetic build problem and a non-determinism problem?

Select one answer

3.True or false: how non-determinism propagates and how sandboxing relates to it.

Choose True or False for each sentence

If a non-deterministic action produces different bytes on a re-run, downstream actions that consume those bytes will also be forced to re-run.
Enabling sandboxing is sufficient to prevent non-determinism in build actions.
0 of 3 answered

Footnotes

  1. Bazel and action (non-) determinism — date >$@ example, common entropy sources, propagation, and why sandboxing cannot fully stabilize actions ↩1 ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10

  2. Artifact-Based Build Systems — Bazel's action model assumes outputs are a function of declared inputs, but actions can still be made non-deterministic ↩1 ↩2 ↩3

  3. Debugging and fixing build non-determinism with Bazel — timestamps, random numbers, unstable dictionary ordering, and exec-log based investigation ↩1 ↩2 ↩3 ↩4 ↩5 ↩6

  4. How to keep a Bazel project hermetic? — archives, environment shaping, workspace status, and other practical sources of unstable output ↩1 ↩2 ↩3 ↩4

  5. Deterministic npm dependencies with Bazel — absolute paths and timestamps in generated dependency files causing cross-machine and cross-output-base differences ↩1 ↩2

  6. Reproducible builds with Bazel — reproducibility framing, absolute paths as output leaks, and hermetic toolchains as the long-term answer ↩1 ↩2

  7. Sandboxing — filesystem isolation, sandbox strategy differences, and platform-dependent enforcement limits ↩1 ↩2

← PreviousSandboxingNext →First-Response Debugging Flags