6.2.5 When a Cache Hit Is Slower

A remote cache hit can be correct and still make a build slower. The hit avoids execution, but it introduces lookup, network transfer, decompression, and local materialization work. Whether that trade is favorable depends on the action class, the client cohort, and whether the saved work was on the build's critical path—not on hit rate alone.

6.2.1 How Remote Cache Keys Work explains when two clients may safely reuse one action result. Here correctness remains a gate: compare performance only after the action identity and resulting bytes are known to be valid.

Compare paths, not labels

For one action request, separate the paths you are actually comparing:

recompute = identify inputs + local execution + local output handling

cache hit = identify inputs + remote lookup + download
          + decompression + local materialization

cache miss/write = identify inputs + remote lookup + local execution
                 + compression + upload

These are accounting shapes, not claims that every backend exposes identical stages. Some transfers overlap other work; some clients defer output downloads; and a miss can upload asynchronously. Record the interval and observation that represent each term instead of deriving them by subtracting unrelated totals.

A hit helps when avoided execution exceeds its added costs at the point that matters to the journey. A 20-second compile replaced by a 2-second fetch is an obvious candidate. A 100-millisecond action replaced by several serialized network exchanges may not be. Network round trips can dominate remote-cache workloads with many sequential requests, which is why client-to-service placement is part of the cohort definition.¹

Hold the denominator still

Apply the matched-cohort discipline from 6.1.2 Measuring Builds Fairly. Choose one journey and one action class, then keep these dimensions fixed across the recompute, miss/write, and hit observations:

• source revision, target scope, configuration, platform, and toolchain;
• Bazel, rules, rc files, remote client, and service versions;
• client CPU, memory, filesystem, and network location;
• cache instance, service load, compression, and output-download policy;
• cache temperature and local state; and
• success criteria and required outputs.

Use production-representative builds over time. A small synthetic benchmark can isolate one stage, but it cannot establish the effect on a real mix of actions, cache temperatures, service contention, and developer or CI journeys. Continuous shadow traffic can be bounded to a representative subset, with separate infrastructure accounting for each compared configuration.²

Do not compare a clean recompute on a busy laptop with a cache hit from a large CI runner near the service. That experiment changes local compute, network latency, placement, and cache state at once. It may measure a real end-to-end choice, but it cannot attribute the result to caching.

Build a stage-level comparison

Collect distributions, not one average. The minimum useful comparison looks like this:

Evidence	Recompute cohort	Miss/write cohort	Hit cohort
Eligible action requests, by class	count	count	count
Input discovery and content hashing	time and bytes read	time and bytes read	time and bytes read
Remote lookup	none	requests and latency	requests and latency
Execution avoided or performed	local duration	local duration	verified absent
Upload path	none	bytes, requests, compression CPU, completion latency	none
Download path	none	as applicable	bytes, requests, latency
Decompression and materialization	local output cost	local output cost	CPU, file count, bytes written, latency
End-to-end journey	latency distribution	latency distribution	latency distribution
Resource and money boundary	local compute	compute plus cache write	cache read plus avoided compute

Use the same denominator for every row. Per-action timing answers “is this action class cheaper to reuse?” Per-invocation latency answers “did this journey finish sooner?” Bytes per successful invocation answer a different question from bytes per cache hit. Do not divide one configuration's total egress by another configuration's hit count.

Include failed and missing observations explicitly. Dropping timeouts and fallbacks from the remote cohort makes its tail look better precisely when the service performed worst.

Follow the critical path and the tail

An off-path action can become much faster without changing when the invocation finishes. Conversely, one slow download needed by a dependent local action can hold the critical path even when thousands of other actions hit quickly. For each class, retain whether it appeared on the critical path and how much of its latency was exposed rather than overlapped.

Report at least a central and tail view appropriate to the journey—for example, median and a high percentile—plus the worst bounded cases you would page or roll back on. Do not invent a universal percentile or threshold. Remote service load is spiky, and systems that optimize scheduling explicitly treat tail latency as a separate objective from average utilization.³

Target selection is also part of the evidence boundary. Asking the cache about unaffected work still costs loading, analysis, requests, and possibly downloads. One production case used a pre-cache change analysis specifically to avoid hammering a small cache and reduce egress; that is evidence that “more hits” and “less total work” are different interventions, not a portable recommendation for that tool.⁴

Treat large files as a byte-flow problem

Large inputs and outputs magnify different stages, so “the cache is slow” is too coarse a diagnosis.

For large inputs, measure content hashing and bytes read before the lookup. That work may be required to establish the action identity even when the result is already remote. If hashing dominates both the hit and recompute paths, improving cache hit rate does not remove the bottleneck.

For large outputs, compare the execution that reuse avoids with compression, upload, storage, download, decompression, and materialization. Include file count and shape as well as total bytes: creating many local files can expose a different cost from transferring one blob. 6.2.7 Directory Outputs and Tree Artifacts continues that high-cardinality case.

Measure the outputs the journey actually needs. In Bazel 9.0, --remote_download_outputs accepts all, minimal, or toplevel, with toplevel documented as the default. minimal avoids downloading remote outputs except those needed by local actions; toplevel also downloads outputs of top-level targets.⁵ Compare these policies while holding the target, service, cache state, platform, and toolchain fixed. Verify that every output needed by a later local action or by the user remains available. This is an output-materialization experiment, not a reduction in required compilation or linking.

Do not replace content hashing with trusted filesystem metadata merely to make the benchmark green. A shortcut that supplies a stale or forged digest can turn a performance optimization into incorrect cache identity. The portable fallback is content hashing; any alternative needs its own maintained integrity, freshness, platform, and unavailable-service contract.

Make a bounded policy decision

Classify each action cohort from the evidence:

• Reuse: hits reduce end-to-end latency or resource cost without unacceptable tail, transfer, or reliability effects.
• Bypass: recomputation is consistently cheaper for this class, and bypassing does not break downstream consumers or correctness controls.
• Change output handling: execution is worth avoiding, but downloading or materializing unneeded outputs dominates.
• Undecided: measurements are incomplete, cohorts differ, or the result is sensitive to load and placement. Keep the canary bounded and gather the missing evidence.

Amortize write cost across useful subsequent reads, not nominal hits. Include request, storage, network or egress, client CPU, and operational failure costs within the experiment's declared boundary. Use that evidence to decide for one action class and journey; 6.6.8 Build Infrastructure Cost and Impact combines such results into a fleet-wide investment model.