6.2.2 Setting Up a Shared Remote Cache

A shared cache is not deployed when every client merely has a --remote_cache flag. It is deployed when a maintained service has a bounded storage and trust contract, and a controlled test proves that one clean Bazel client published an action result that a different clean client reused without executing the action locally.

3.6.5 Simple Remote Cache Setup gives the minimal client configuration. At infrastructure level, preserve enough evidence to distinguish the remote cache from every local reuse layer described in 2.4.2 Where Bazel Caches Things and to connect the observation to the action identity from 6.2.1 How Remote Cache Keys Work.

Choose one protocol contract

Bazel can use either its HTTP remote-cache protocol or the caching part of the Remote Execution API (REAPI). Both expose an action-cache (AC) lookup and content-addressable storage (CAS), but they are not interchangeable wire protocols.

With HTTP, Bazel sends GET and PUT requests beneath the configured base URL. Action-result metadata is addressed under /ac/, while output bytes are addressed under /cas/. An HTTP access log can therefore show the AC request and the separate CAS transfers directly.¹

With REAPI, the endpoint is a gRPC service. ActionCache returns or updates an ActionResult; CAS and ByteStream services find and transfer the referenced blobs. The service also has a capabilities contract. A backend's storage topology, authentication system, metrics names, and garbage collection are implementation choices, not properties supplied by REAPI.²

Choose one of these contracts for the first deployment and name it in the service record. Do not call an HTTP /ac/ request a GetActionResult RPC, and do not assume that a backend accepting HTTP also implements REAPI capabilities.

Make the service operable before connecting clients

For a concrete cache-only deployment, bazel-remote is a maintained standalone backend with HTTP and gRPC cache endpoints, bounded disk storage, status and metrics surfaces, TLS/authentication support, and optional object-store proxies. It is not a remote executor.³ Pin an immutable released artifact or image digest; do not deploy a floating tag.

The deployment record should answer these questions before a canary writes any data:

Decision	What to record and test
Protocol and endpoint	Exact https, grpc, or grpcs endpoint; backend version; Bazel version; advertised/documented protocol
Trust	Server identity and TLS trust roots; credential delivery; authenticated reader and writer identities; rejection of absent or wrong credentials
Namespace	REAPI instance name when the backend documents one, or an HTTP base-path deployment boundary; never treat the name itself as authorization
Storage	Persistent volume or proxy store, capacity limit, eviction behavior, restart persistence, and owner
Observability	Health/status check, request logs, AC/CAS counters, bytes, errors, latency, and a correlation window for the proof
Failure behavior	What the pinned client does when lookup, download, or upload fails; which fallback is acceptable for this journey; alert and rollback trigger

Start with a narrow instance or endpoint for one compatible client cohort. A service that has no authenticated writer boundary can publish a bad result to every consumer that trusts the namespace. Authentication, instance selection, and write authority are separate decisions; 6.2.3 Securing a Shared Remote Cache develops their threat model.

Standalone bazel-remote is sufficient for the storage and two-client reuse proof below, but its documented authentication does not provide a per-principal reader-versus-writer ACL. Put an authorization gateway in front of it, or choose a backend that documents operation-level permissions, before calling this a production writer boundary. Its AC instance separation is also backend-specific and must be explicitly enabled and tested; CAS blobs remain globally content-addressed, so an instance name is not a confidentiality boundary.³

Configure readers and writers separately

The following production template chooses a REAPI/gRPC endpoint. Its example hostname and instance name are deliberately not the loopback fixture used later: substitute the endpoint, instance, and credential mechanism recorded for your deployment, and keep secrets out of the checked-in file.

build:shared-cache --remote_cache=grpcs://cache.example.internal
build:shared-cache --remote_instance_name=team-ci-canary-v1

build:cache-writer --config=shared-cache

build:cache-reader --config=shared-cache
build:cache-reader --remote_upload_local_results=false

The writer may read existing entries and upload locally executed results. The reader may consume entries but cannot upload its local results through this configuration. Bazel documents --remote_upload_local_results=false as the read-only client control and warns operators to restrict who may write.¹ The service must independently enforce the matching permission; a client flag is not an access-control boundary.

For an HTTP deployment, change the endpoint to its documented https://... base URL and remove --remote_instance_name unless that backend explicitly maps it. The base path precedes the HTTP /ac/ and /cas/ resources. Do not carry a REAPI instance convention into an opaque HTTP store by assumption.

Keep this proof configuration separate from ordinary user configuration. In particular, omit --disk_cache, and audit imported rc files for a second remote endpoint, credentials, or policy overrides. The Bazel core repository's public cache documentation and remote-client regressions are the right place to verify the exact flags and client behavior for the pinned Bazel release.⁴

Prove reuse with two clean clients

Use the same immutable source revision, target, Bazel release, rules and rc files, platform and toolchain inputs, digest capabilities, endpoint, and instance. Select an action that is substantial enough to identify in the execution log. Provision two independent clients, or two containers that do not share an output base, disk cache, Bazel server, or output tree.

The CI cache baseline project turns that procedure into an executable test. From its workspace root, run:

bash tools/prove_shared_remote_cache.sh

The launcher downloads bazel-remote 2.6.1 for its upstream-supported Ubuntu 20.04+ or macOS 13+ hosts, verifies the published SHA-256 digest for the detected amd64 or arm64 asset, and delegates to the proof harness. That harness starts the HTTP endpoint on an available loopback port, writes a full access log, and probes /status with bounded connection and request timeouts. Every candidate backend PID is owned by the cleanup trap; an unready candidate is terminated and reaped before another port is attempted. The backend begins with an empty cache directory. The fixture does not claim compatibility with other Linux distributions or older operating-system releases.

The proof test then builds the deterministic //app:uppercase_message genrule three times. Client A uses an empty output base and uploads. Client B uses a different empty output base and --remote_upload_local_results=false. The bypass client uses a third empty output base, a second empty backend, and --remote_upload_local_results=false. Its lookup must miss and the backend must receive no writes; keeping an HTTP cache configured makes Bazel record the comparable action digest. All builds use --ignore_all_rc_files, so a system, workspace, or user disk-cache configuration cannot mask the experiment.

The decisive proof is the join across both sides, not the build's exit status:

Observation	Client A	Client B	Backend
Action identity	Execution log records the action digest	Same digest	GET 404 /ac/<digest>, then PUT 200, then GET 200
Returned output	Execution log records the output digest	Same digest	PUT 200 /cas/<digest>, then GET 200
Execution	cacheHit=false and a local runner	cacheHit=true, runner remote cache hit	AC metadata and the output blob are served to B
Control	Uploads enabled	Uploads disabled	Third client misses against a separate empty backend and returns to local execution

The fixture's assertions join the three client records to the backend access log and reject a digest, runner, hit-state, or read-only-bypass mismatch.

If the logs do not expose a field needed for that join, add a backend trace or cache event that does. Do not substitute cached stdout/stderr, equal final file names, or two successful builds: remote caches can store action stdout and stderr, and success does not reveal where an action ran.¹

On Linux x86_64 with Bazel 9.0.0, a captured run ended with this evidence (the digest and local runner name can change with platform or tool inputs):

SHARED REMOTE CACHE PROOF PASSED
client A: action 3bf8672d...715d92, runner 'processwrapper-sandbox', cacheHit=false
backend: AC miss -> AC/CAS writes -> AC/CAS reads for the same digests
client B: action 3bf8672d...715d92, runner 'remote cache hit', cacheHit=true
bypass:   runner 'processwrapper-sandbox', cacheHit=false (empty backend, uploads disabled)

The script exits nonzero unless the action and output digests match, A executes locally, B reports a remote-cache hit, the HTTP access log contains the matching AC/CAS status sequence, and the bypass executes locally. It retains all logs on failure, rather than replacing missing evidence with a green build or timing comparison.

The loopback fixture proves storage and cross-client reuse only. It intentionally does not model production TLS, authentication, authorization, persistence, or namespace isolation. Test wrong credentials, a deliberately different instance or HTTP base path, backend restart persistence, and documented unavailable-service behavior against the real canary deployment.

Promote only after the controlled miss, completed write, fresh-client hit, bypass, and negative credential/namespace tests all agree. Then use the bounded canary and rollback criteria in 6.1.5 Safe Infrastructure Rollouts before adding client cohorts or widening write authority.

After the service is qualified, H.3.2 Bootstrap can decide whether a pre-warmed first build is worth making part of the organization-wide new-developer journey. That policy depends on this cross-client proof; a fast first build alone cannot establish which cache layer supplied the result.