5.7.1 Cache Internals Reference

Bazel cache incidents become much easier to reason about once you stop treating “the cache” as one store. A warm build can reuse an in-memory Skyframe value, an output already present in one output base, a downloaded archive, or an action result stored in a disk or remote AC/CAS. Each layer answers a different lookup, has a different lifetime, and loses different evidence when you clear it.

6.2.1 How Remote Cache Keys Work continues from this local cache-layer model by separating Bazel's output-base action-cache state from HTTP and REAPI shared cache identity across clients.

A warm build can still be cold at another cache layer

First identify what Bazel is trying to reuse. Each family has its own lookup, scope, and stored value.

BUILD COMPUTATIONS

Skyframe graph

Lookup: SkyKey

Value

Incremental evaluation result

Boundary

One Bazel server · memory

REPOSITORY MATERIAL

Download + contents caches

Two distinct lookups

blob digest → downloaded blob

repo inputs → eligible fetched directory

Values

A download hit may still require extraction and repository evaluation

Boundary

Configured paths can be shared by workspaces and output bases on one machine

ACTION RESULTS

Output tree or AC/CAS

Current state and historical results use different mechanisms

metadata + current files → reuse in one output base

action digest → ActionResult → CAS blobs

Values

An AC record is unusable if a referenced CAS blob is missing

Boundary

Output tree: one output base · disk cache: every client with path access · remote: service policy

Diagnostic order: subject → scope → lookup identity → stored value → retention policy

Start with the subject being reused

The layers fall into three families. Skyframe caches computations about the build. Repository caches reuse external-repository material. The remaining layers reuse action results. This classification is more useful than a list of directories because it tells you which identity Bazel is trying to match.¹

Layer	What it reuses	Scope and lifetime	Primary diagnostic question
Skyframe graph	Incremental evaluations, including package loading and the analyzed action graph	One Bazel server. Memory is lost when that server exits	Did loading or analysis become cold, even if execution outputs remain reusable?
Repository download cache	Downloaded blobs, commonly archives	Normally shared across output bases on one machine	Was the download avoided, but extraction or repository evaluation still required?
Repository contents cache	Fetched repository directories	Shared across workspaces. Enabled by default under the repository cache unless configured otherwise	Could Bazel reuse the materialized repository, not merely its archive?
Output tree plus persistent action cache	Outputs currently present in one output base and metadata validating them	One output base. Survives a server restart	Are the current output bytes still valid for the current action?
Disk cache	Action results and output blobs in local ac/ and cas/ stores	A configured filesystem location, shareable by output bases that can access it	Does this action digest have an ActionResult, and are its referenced blobs present?
Remote cache	The same AC/CAS roles exposed by a cache service	Shared according to service, credentials, and instance policy	Did the client ask for the same action digest, accept the result, and obtain every referenced blob?

The first row is broader than action caching. Skyframe memoizes fine-grained computations such as glob() evaluation and action-graph construction. Keeping that graph warm is a major reason Bazel uses a long-lived server. Losing it can make loading and analysis expensive without forcing every action to execute, because persistent action-output layers are separate.¹ 5.9.1 Skyframe Data Model develops the node model behind this in-memory layer.

This is the deeper diagnostic continuation of 2.4.3 What Makes a Cache Hit and 3.6.6 Cache Warm/Cold Considerations: the earlier articles establish what affects action reuse and how CI can be warm at one layer but cold at another. Here the lookup identities and retention boundaries make those observations testable.

The two repository caches are also deliberately separate. The path reported by bazel info repository_cache has historically been a download cache: it stores content-addressed downloaded blobs and can be shared across workspaces, but a hit does not by itself avoid extraction or repository-rule work.¹ Current Bazel also has --repo_contents_cache, whose default location is {--repository_cache}/contents. It stores fetched repository directories for reuse across workspaces.² Bazel admits only eligible results: reproducible, non-local repository rules must opt in, so results with undiscoverable inputs or unsafe cross-workspace state are not assumed cacheable merely because this cache is configured.³ This is repository-rule reuse, not an action-cache entry and not another name for the CAS used by remote execution.

Two different mechanisms are called “action cache”

Inside one output base, Bazel keeps produced files in the output tree and a persistent index under action_cache/. bazel dump --action_cache exposes entries containing fields such as actionKey, usedClientEnvKey, and digestKey. Bazel compares that metadata with the current action and with the outputs already on disk. If they still agree, it can keep those outputs without executing the action.¹

This layer remembers only the currently materialized state. In an A → B → A sequence, the second build replaces the useful A state with B. Returning to A does not give this output base a historical A result. A disk or remote cache can retain both results under different action digests.¹

Do not equate the output-tree entry's actionKey field with the remote-cache action digest. Both participate in deciding whether an action result is reusable, but they belong to different formats and lookup mechanisms.¹ In an AC/CAS cache, the Action Cache maps an action digest to an ActionResult. That result contains digests for output files or trees. The Content-Addressable Store maps those digests to bytes. An AC hit with a missing referenced CAS blob is therefore not a usable result.⁴

The local disk cache follows this same division and normally exposes ac/ and cas/ directories. It is “remote-cache-shaped” storage on a local filesystem, not a copy of the output tree.^1,4 6.2.2 Setting Up a Shared Remote Cache covers the wire protocol, service selection, and operational setup. The distinction here is the internal lookup you must trace.

Check persistence and garbage collection in the current configuration

Old cache taxonomies often say that repository and disk caches grow forever. That accurately described important older releases, but Bazel added several different collectors in stages:

Version	What changed	Bazel 9 default
Before 7.4	The local --disk_cache had no built-in retention policy. Users needed external cleanup.	—
7.4	Bazel introduced experimental disk-cache GC by maximum size and/or entry age.	Both limits remain 0, so GC is inactive until configured.
9.0	Bazel added periodic GC for the persistent output-base action cache.	Its maximum age is 0, so this collector is also opt-in.
9.0	Bazel introduced the local repository contents cache for eligible fetched repositories, with its own age-based GC.	Maximum unused age is 14 days.

The disk- and action-cache collectors run after the Bazel server has been idle long enough. Their default idle delay is five minutes. Action-cache GC also requires enough stale entries to cross its configured threshold.^2,5 Therefore “Bazel 9 has cache GC” does not mean every cache is bounded out of the box: disk-cache and persistent-action-cache limits still need explicit configuration.

The repository contents cache's 14-day policy does not apply to the adjacent repository download cache. State the Bazel version, identify which repository cache you mean, and inspect the effective flags before diagnosing capacity from an old rule of thumb.²

Use Bazel to resolve paths and current settings rather than deleting guessed directories:

bazel info output_base
bazel info repository_cache
bazel dump --action_cache
bazel help fetch

The dump answers what the output-base index remembers. It does not enumerate a disk or remote AC/CAS, prove that a remote lookup occurred, or explain why two invocations computed different action digests. For those questions, preserve both invocations and compare execution evidence as described in 5.7.2 Execution Log Analysis. Clearing state first destroys the very entries and contrasts needed to identify the responsible layer.

A layer-first incident checklist

When reuse surprises you, classify the symptom before reaching for a cleanup:

1. Name the reused subject. Is the cost in repository fetching, loading and analysis, or action execution?
2. Name the expected scope. Same server, same output base, same machine, or another machine? A layer cannot hit outside its sharing boundary.
3. Name the lookup identity. Skyframe key, downloaded-blob checksum, repository-rule inputs, output-tree validation metadata, or AC action digest?
4. Check the value side. For AC/CAS, an AC record alone is insufficient if a referenced CAS blob is absent. For the output tree, the bytes must still be present and validate against the persistent metadata.
5. Check retention and policy. Establish whether an entry expired, was garbage-collected, was bypassed by flags or tags, or was never written.

Only then choose a layer-specific experiment. A fresh output base isolates output-tree reuse while retaining shared caches. Disabling --disk_cache says nothing about the repository caches. bazel shutdown tests warm-server state, not whether action outputs are reusable. 2.4.2 Where Bazel Caches Things provides the earlier operator-level map of where the layers live. This reference adds the identity and lifecycle distinctions needed for diagnosis.