6.3.19 Local REAPI Development

extra

You do not need a worker fleet to exercise Bazel's Remote Execution API. A local REAPI executor lets the Bazel client upload an action to an endpoint, execute it in a separate worker environment, and retrieve the result while every component still lives on one development machine.

This makes the protocol traced in 6.3.4 Remote Action Tracing concrete without requiring a shared service.

That workflow is useful for three different reasons:

  • expose actions that accidentally depend on the developer host.
  • exercise a Linux execution boundary from a supported macOS, Windows, or Linux host.
  • share one persistent CAS and ActionCache between several local Bazel servers.

It is not a performance promise. It adds protocol, VM, and data-transfer work, and it cannot model the scheduler, network, contention, or failure behavior of a production cluster. Nor does it reproduce a production executor image: toolchains, runtime layers, environment variables, and execution properties must be compared with CI or RBE separately.

actiond's topology

actiond is a concrete implementation. Bazel connects to a host-side gRPC listener. The host starts a small Linux VM and forwards REAPI traffic to a guest process that implements Execution, CAS, ByteStream, ActionCache, and Capabilities services.1

Bazel client
    |
    | REAPI over local gRPC
    v
host actiond process
    |
    | vsock / Hyper-V socket bridge
    v
Linux guest: executor + CAS + ActionCache

The VM is long-lived. Actions do not boot one VM apiece. Each action receives a separate chroot with private mount and network namespaces. Compared with the local execution strategies in 2.3.2 Sandboxing, this places the action inside a separate Linux guest. An actiondfs filesystem exposes the Merkle input tree lazily from guest CAS blobs, so sandbox setup does not perform one host-side filesystem operation for every declared input.1

This is a stronger boundary than merely changing Bazel's local spawn strategy, but “runs in a VM” is not synonymous with “fully hermetic.” The action contract still depends on the selected toolchain, declared environment, execution properties, and any runtime layers supplied by the executor.

The published release matrix is narrower than “any host”: macOS ARM64, Windows ARM64/x86_64, and Linux ARM64/x86_64. Windows needs Hyper-V. Linux needs io_uring plus read/write access to KVM and vhost-vsock. The README contains the current checks and startup commands.1

Configure Bazel as an REAPI client

After installing the host-specific release and starting its documented VM server on 127.0.0.1:8980, first define and register an execution platform whose constraints describe the guest OS and CPU and whose toolchains produce Linux executables. The runnable local-reapi-executor snippet uses //:local_linux_exec for a Linux x86-64 guest and selects it with a named .bazelrc configuration:

build:local-reapi --remote_executor=grpc://127.0.0.1:8980
build:local-reapi --remote_cache=grpc://127.0.0.1:8980
build:local-reapi --spawn_strategy=remote
build:local-reapi --genrule_strategy=remote
build:local-reapi --remote_local_fallback=false
build:local-reapi --remote_upload_local_results=false
build:local-reapi --noremote_cache_compression
build:local-reapi --extra_execution_platforms=//:local_linux_exec

Then:

bazel build --config=local-reapi //:local_reapi_probe

The linked BUILD file defines both labels, so this command is concrete rather than a placeholder. For another workspace, the execution-platform label must describe the actiond guest and its registered Linux-executable toolchains. Cross-host builds may also need an explicit --host_platform when exec-configured tools still follow a host-platform contract, and an explicit --platforms when the requested output targets a different OS or CPU. The upstream LLVM smoke test selects all three surfaces.

Copy its intent, not its repository-specific labels.

This named configuration is one example of the repository-local workflows discussed in 3.5 Developer Experience & Local Tooling: developers opt into a different execution boundary without changing the target graph.

These flags describe the inspected actiond contract, not a universal local-RBE profile. In particular, compression was not supported at that revision, and disabling local fallback is a deliberate diagnostic choice: it prevents an incompatible action from silently succeeding on the host.1

Verify the boundary with upstream evidence

Do not infer remote execution merely from a green build. Use the upstream checks as observable evidence:

  • tools/e2e.sh vm covers macOS and Linux: it starts the host-specific VM and builds the test/ workspace with remote-only spawn strategies, cache acceptance disabled, and local fallback disabled. Its test/stress.bzl workload also checks that external networking is blocked while loopback and the packaged /etc/hosts work. Success therefore means those actions executed through the guest endpoint and satisfied the checked action-sandbox behavior. It is not a general security proof.
  • On Windows, use e2e/run_llvm_windows_vm_smoke.ps1. It starts the Hyper-V-backed executor, builds with remote-only strategies and local fallback disabled, and requires a remote process summary. A completed actiond measurement is the Windows evidence route. The bash route above does not support Windows.
  • e2e/llvm_tblgen_smoke.sh selects a Linux-musl host platform because generated exec tools run in the VM. A successful llvm-tblgen build with --remote_local_fallback=false, together with Bazel's process summary reporting remote processes, proves that the Linux exec-configured actions ran remotely. An exec-format or toolchain-resolution failure instead exposes a host-only tool assumption.
  • For cache reuse, keep the same actiond --root and endpoint but use a fresh Bazel --output_base for the second build, as the platform runners under e2e/ do. With cache acceptance enabled, remote cache hits in Bazel's process summary are the observable criterion that the new Bazel server reused the guest ActionCache/CAS. A second success without cache hits proves execution, not sharing.

These are upstream development routes, not lightweight prerequisites for every workspace. For your own target, the smallest useful check is a build with local fallback disabled plus an execution log or process summary that shows remote actions.

A repo-local proof with an explicit fixture boundary

Packaging actiond inside this snippet would make the asset host-specific and would still require VM support. Instead, the snippet keeps the live command available for an existing endpoint and makes the portable regression check deterministic. Its reduced execution record and cache-hit record were captured from separate Bazel 9.1.0 output bases using the same local Buildfarm endpoint. The first build rejected cache hits; the second used a fresh client. The capture manifest records normalized reproduction commands, distinct output bases, endpoint, versions, and fixture hashes. It also states that ephemeral paths and the Docker-published port were normalized; it is an inspectable provenance assertion, not independent attestation of the capture host. The verifier checks that contract, requires the same action digest, and distinguishes runner: remote, cacheHit: false from runner: remote cache hit, cacheHit: true.

Running the snippet's deterministic command builds //:local_reapi_probe locally, then checks those captured fields and prints:

PASS //:local_reapi_probe: first client selected remote execution (cacheHit=false)
PASS //:local_reapi_probe: fresh client selected a remote cache hit (cacheHit=true)
BOUNDARY checked-in records prove Bazel's client observations, not executor isolation

That output proves the fixture remains internally consistent and shows which Bazel-side observations to require. It does not prove that actiond is running now, that Buildfarm has actiond's VM boundary, or that either backend provided a particular isolation property. Run the named config against actiond and retain its upstream executor/process evidence when the VM boundary is the claim under test.

Your toolchains must run in the guest

On a macOS laptop, actiond does not make a Darwin compiler executable inside Linux. Bazel resolves toolchains before sending the action. The selected tools and all their inputs must be compatible with the Linux guest's CPU and declared execution constraints.

Keep the roles from 3.3.2 Platform Vocabulary distinct:

  • the host runs the Bazel client and actiond VM process.
  • the execution platform is the Linux guest that runs compiler/test actions.
  • the target platform describes the produced output.

A Linux executor can still cross-build a Windows or embedded target when the registered toolchain supports that pair. Conversely, a native macOS toolchain cannot execute remotely merely because the endpoint is local.

When resolution fails, inspect:

  1. the active execution platforms.
  2. the selected toolchain's exec_compatible_with.
  3. the executable format/CPU of every tool.
  4. hidden shell or system-library dependencies exposed by the VM.

Runtime mounts are explicit compatibility debt

The executor can mount packaged Bash or selected glibc versions when an action requests the corresponding execution property. This is useful for legacy tools, but the runtime is executor infrastructure rather than an ordinary input declared by the target.1

Use the property to make a boundary visible and unblock a migration. Do not use it to claim that the action is independent of the executor image. A truly portable tool should normally arrive through a declared toolchain or input closure.

What local REAPI proves

A successful run can prove...It does not prove...
The action's tools execute in the selected Linux environmentA production scheduler will route it correctly
The declared input root is sufficient for this worker runtimeNo hidden worker runtime is involved
REAPI/CAS/AC client paths work against this endpointConformance with every REAPI server or feature
Several local Bazel servers can reuse one CAS/ACNetworked or multi-region cache performance
VM isolation contains action execution better than host executionComplete protection against every hostile workload

Use it as a development and diagnostic layer between native local sandboxing and a shared RBE service. Promote findings into explicit toolchain inputs, platform constraints, or worker-image contracts instead of making the local executor a prerequisite for understanding the build.

key takeaway

A local REAPI executor is valuable because it changes the execution boundary without requiring a cluster. It catches host assumptions and exercises remote client paths, but the workspace still owns Linux-compatible toolchains and a production rollout still needs scheduler, capacity, isolation, and failure-mode validation.

Footnotes

  1. actiond repository map and upstream ARCHITECTURE.md — host/guest topology, REAPI subset, actiondfs input model, runtime properties, tests, and limitations. 1 2 3 4 5