2.4 Caching & Incrementality
Bazel's caching story is easy to misread from the outside. A repeated build may
feel instant, while the same request after bazel shutdown spends time loading
and analyzing even though the compiler does not run. In a different invocation,
an action may execute again even though no source file obviously changed. This
section makes those observations feel like one system instead of unrelated
tricks.
The useful shift is to stop asking whether "the cache" was warm. Bazel has several memories, and each one answers a different question. Some memory lives only in the long-running server. Some lives under the output base. Some stores downloaded repositories. Some can be shared through a disk or remote cache. Above those locations sits the harder question: what makes two action executions count as the same computation?
Reuse Has Layers And A Key
The sequence starts with 2.4.1 Skyframe & Incrementality because incremental builds are not only about saved output files. Skyframe is the in-memory evaluation graph behind warm-server behavior from 1.1.3 Server/Client Architecture: it remembers package loading, analysis, file state, artifacts, and action execution state well enough to invalidate only affected nodes after a change. That article gives the first boundary: losing server memory makes loading and analysis cold, but it does not automatically delete reusable action outputs.
2.4.2 Where Bazel Caches Things then separates the places where reuse can live. It distinguishes what bazel clean and bazel shutdown affect, what bazel info output_base locates, and where repository downloads, --disk_cache, and remote cache setup fit. The useful diagnostic question is which layer could have helped this run. It keeps local output-base reuse, optional disk cache, and shared remote cache from blurring into one vague "cache" knob.
2.4.3 What Makes a Cache Hit turns from storage to identity. Bazel does not reuse work because a target label is the same or because a file timestamp looks fresh. It reuses an action result when the declared inputs, command, relevant configuration, environment, tools, and platform still describe the same action. This is where caching reconnects to 2.3.1 Hermeticity: a key is only trustworthy if Bazel's recorded inputs are the real inputs.
Finally, 2.4.4 Diagnosing Cache Misses gives the first-response workflow when the model says "this should have been a hit" but Bazel rebuilt something anyway. Its flags do not replace the cache model. They expose pieces of it in a form a human can triage. Timing-profile analysis belongs in 5.4.1 Timing Profile Analysis, while deeper action comparison waits in 5.7 Cache & Execution.
Match The Symptom To The Reuse Layer
First name the symptom: did Bazel repeat loading or analysis, or did an action execute again? Then match it to the reuse layer and its observable:
- In-memory graph state. Symptom: loading or analysis became cold after the Bazel server restarted. Observable: a JSON trace profile shows that phase work without corresponding action execution. Bazel records phase and action timing in this compressed profile. 5.4.1 Timing Profile Analysis explains how to capture, compare, and interpret it without treating one lane as a diagnosis.1
- Local persistent reuse. Symptom: an action executes again even though a local output or optional disk-cache result seemed reusable. Observable: the explain log names the rerun action and the input, command, environment, or other change that made its previous result unusable. The command workflow is in 2.4.4 Diagnosing Cache Misses.
- Remote action reuse. Symptom: an action executes after reaching a configured remote cache. Observable: the remote-cache client or service reports a miss, lookup failure, or unavailable blob from the Action Cache (AC) or Content Addressable Storage (CAS). Those records are backend-specific. 5.5.3 Remote Cache Diagnostics shows how to capture and compare them rather than inventing a universal Level 2 command.
Do not check all three layers speculatively and consult evidence afterward. Capture the observation first, test the matching hypothesis, and move to another layer only when the evidence points there. The diagram in 2.4.2 Where Bazel Caches Things remains the map of the storage boundaries.
A Hit And A Rebuild Both Need Explanation
Do not treat every rebuild as a cache bug. Changed flags, platform choices, tools, stamped metadata, and environment can all make an action legitimately different. But do not treat every cache hit as harmless either. If an action reads host state that Bazel never modeled, Bazel may reuse a result under a false identity. This connects cache behavior back to 2.3 Hermeticity & Sandboxing and forward to 3.2 Project Configuration, 3.3 Configurable Builds & Platform Basics, and 6.2 Shared Remote Cache.
Where The Confusion Usually Comes From
The first confusion is mixing memory lifetimes. Skyframe disappears when the Bazel server exits. The output tree and action cache under the output base can survive. The repository cache is shared differently again. Disk and remote caches have still wider scope. If a command gets slower after bazel shutdown, that does not by itself mean Bazel rebuilt every output.
The second confusion is mixing target names with action identity. //app:server can expand into many actions, and the cache decision happens at the action level. The top-level label may be unchanged while one compile command, generated input, toolchain, or configuration value changed underneath it.
The third confusion is using the wrong diagnostic lens. Failed sandboxed actions belong to 2.3.4 First-Response Debugging Flags first: see the command, inspect the sandbox, fix the missing declaration. Successful builds that rerun unexpected work belong here: write an explain log, add command details only when needed, and check rc-file or configuration drift before assuming the cache is broken.
Trace: After bazel shutdown, a repeated build is slower. Its profile shows
new loading and analysis time, but the explain log names no rerun compile action.
Which reuse layer became cold, and what evidence would have pointed somewhere
else?
Reveal
The in-memory Skyframe graph became cold when the server stopped. The profile shows the cost of reconstructing loading and analysis state, while the absence of a rerun compile action shows that persistent action reuse still worked. If the explain log instead named the compile action and a changed input or command, inspect action identity and local persistent reuse. If the action reached a configured remote cache and its client or service reported a miss or unavailable blob, continue with the remote branch. The artifacts select the branch. Elapsed time alone does not.
The stamping-status snippet demonstrates the cache-sensitive boundary directly: its rule consumes stable and volatile status files as distinct declared inputs.
Choose The Reuse Layer First
Read 2.4.1 Skyframe & Incrementality first if repeated commands feel mysterious. It gives the in-memory model that explains why warm builds are different from cold ones. Read 2.4.2 Where Bazel Caches Things next if your immediate question is "where did Bazel put this state?" or "what did this cleanup command remove?"
Then slow down on 2.4.3 What Makes a Cache Hit. It is the conceptual center of the section, because it turns caching from a storage topic into an identity topic. The details in that article make later remote caching, platform configuration, and cache-miss debugging much less surprising.
Use 2.4.4 Diagnosing Cache Misses as a workflow article. You do not need it on every normal build. You need it when the build succeeds but rebuilds more than you expected, especially after flag changes, .bazelrc changes, stamping changes, or machine-to-machine drift. If the explain log points at something too deep to compare manually, stop at the Level 2 boundary and carry the question to 5.7 Cache & Execution later.
Bazel reuses work at several layers, but action identity decides whether a prior result is valid. Separate warm-server state, persistent local reuse, and remote reuse before asking why a build was fast or why an action ran again.
Footnotes
-
JSON Trace Profile — profiles record build-phase and action timing for later inspection ↩
Sections in this chapter · 4
Skyframe's in-memory dependency graph, invalidation closure, change pruning, and warm-server incrementality.
The scope and lifetime of Skyframe, repository, output-tree, disk, and remote cache layers.
The action inputs, command, environment, configuration, and tools that determine whether outputs are reusable.
First-response diagnosis of unexpected rebuilds with --explain, --verbose_explanations, and --announce_rc.