6.4.1 Choosing Execution Strategies

An action is not “local” or “remote” merely because a flag names that mode. Bazel first selects an applicable execution strategy, and that strategy may still find a cache result, reuse a persistent process, race another branch, or fall back to a different executor. Operating a mixed fleet therefore requires two separate decisions: which strategy should be attempted for an action class, and what evidence proves what actually happened.

2.5.1 Strategies Abstraction introduced strategies as interchangeable ways to realize a spawn. At infrastructure scale, the useful unit is not one global default but a cohort of actions with the same mnemonic, tool behavior, platform, input and output shape, and service journey.

Compare complete execution paths

The main strategies expose different cost and isolation boundaries:

StrategyWhere the command runsWhat it can saveCost or risk to measure
localA subprocess on the Bazel hostRemote queueing and transferHost contention and ambient-state exposure
sandboxedA locally isolated subprocessRemote overhead while checking declared-input disciplineSandbox setup, filesystem work, and platform-specific overhead
workerA reused local tool processProcess startup, JIT, parsing, and tool initializationRetained memory, pool fragmentation, request-state leakage, and fallback
remoteA compatible remote executorShared capacity and host independenceCache lookup, input upload, queueing, execution, output upload, and materialization
dynamicCompeting local and remote branchesTail latency when either side occasionally winsDuplicate resource use, cancellation, and output coordination

Bazel's command documentation describes --spawn_strategy as the broad selection and --strategy=<mnemonic>=<strategy> as a per-mnemonic override. It also distinguishes local, sandboxed, worker, and remote; remote is applicable only when a remote executor is configured.1 A comma-separated strategy list is a priority order of applicable implementations, not a race. Dynamic execution is the explicit racing mechanism and is treated separately in 6.4.2 Dynamic Execution.2

Do not compare only command duration. For the same action cohort, measure the complete path from readiness to usable output:

  1. local resource admission or remote queue time;
  2. process or worker startup, sandbox preparation, and input transfer;
  3. command execution;
  4. output upload, download, and materialization; and
  5. the action's contribution to invocation critical path, resource pressure, and service cost.

This prevents a common false conclusion: a remote command can execute faster yet finish later for its consumer because queueing and data movement dominate. Conversely, a locally fast action class can make the invocation slower when it occupies CPU and memory needed to drive remote work. Profiles should be compared between matched clean or incremental cohorts because cache state changes the amount and kind of work performed.3

Persistent workers deserve the same end-to-end treatment. They help when repeated actions have substantial startup cost or benefit from cross-action caching, but only rules and tools that implement the worker contract can use them.4 Pool sizing, recycling, and fragmentation continue in 6.4.3 Operating Persistent Worker Pools.

Build an action-class strategy matrix

Start with representative mnemonics and split them further when one mnemonic hides materially different platforms, tools, or artifact shapes. For each row, record:

FieldQuestion it answers
Correctness gateDo candidate strategies produce equivalent required outputs under isolated reruns?
ApplicabilityDoes the action support sandboxing, workers, and the selected remote platform?
Latency distributionWhich path improves median and tail completion, including queue and transfer?
Resource demandWhich client CPU, memory, disk, network, and remote capacity does it consume?
ReuseDoes it obtain a cache result or reuse worker initialization, and how often?
Failure behaviorWhich fallback is permitted, and can it hide an infrastructure or action defect?
Placement evidenceWhich client and backend observations prove the selected path?

Use a narrow override for the canary rather than changing every spawn. For example, a pinned configuration may test a candidate strategy for one mnemonic:

build:strategy-canary --strategy=Javac=remote

The exact mnemonic and applicable strategies come from the analyzed actions and the pinned Bazel and ruleset versions; do not copy a mnemonic from another repository and assume it represents the same workload. A broad default remains useful as a fallback policy, but the high-value exceptions should be justified by measured rows in the matrix.

The canary must preserve target scope, revision, configuration, execution platform and toolchain, client and service versions, cache temperature, and output-download policy. Change one strategy decision at a time. Require output correctness and complete evidence before accepting a latency or cost win, and define a rollback that restores the previous strategy map without clearing shared state.

Configuration states intent. Joined evidence proves placement.
Follow one action from policy to the accepted result, then ask which observation can support each claim.

A four-step flow separates configured strategy intent, Bazel strategy selection, actual execution or reuse, and joined evidence. A matrix shows the distinct roles and limits of execution logs, profiles, Build Event Protocol data, and backend telemetry for cache hits, local execution, persistent workers, remote execution, and fallback.

Question: Did this action execute remotely, run locally, reuse a worker, fall back, or avoid execution through a cache hit?
1 · Intent
Configured strategy policy
Broad default and action-class override
2 · Selection
Applicable strategy chosen
Priority, capability, and fallback policy
3 · Actual path
Execution, reuse, or alternate path
The configured name is not the outcome
4 · Evidence
Joined action and attempt identity
Client context plus service observations
EVIDENCE BOUNDARIES
Claimed outcome
Execution log
Profile
BEP
Backend telemetry
Remote cache hit
Spawn reuse context
Lookup and transfer timing
Invocation and result context
Reused result and blobs
no joined executor attempt
Local process or sandbox
Local runner or strategy record
Local scheduling and process time
Invocation and output result
no joined remote operation
Persistent worker
Worker request and process evidence
Worker reuse and request timing
Invocation and result context
Worker-pool or process observation
Remote execution
Remote spawn context
Queue, transfer, and execution timing
Invocation and result context
Joined operation and executor attempt
Fallback
Preferred and alternate path records
Timing for both paths
Accepted invocation result
Preferred-path failure and accepted alternate
False inference: CAS traffic or a green build does not prove remote execution.
Evidence limit: Missing telemetry is not proof that no attempt occurred. Check completeness and identity joins.

Prove placement instead of inferring it

A successful build proves that Bazel obtained required outputs. It does not say whether an action executed locally, remotely, through a worker, or not at all. A remote cache hit is especially easy to mislabel: reusable remote bytes are not evidence of remote execution.

Join observations at three layers:

The runnable placement-evidence harness creates isolated cohorts for each outcome. Its verifier joins Bazel records to backend state and makes the negative assertions explicit, rather than treating a configured strategy as observed placement.

  • Bazel-side action identity: mnemonic, target context, command and input identity, plus an execution log or other spawn-level record. Bazel documents tool-friendly execution-log outputs for executed spawns.1
  • Invocation timing: a profile shows scheduling, queueing, worker, transfer, and critical-path effects. Treat it as timing evidence, not by itself as a backend executor identity.
  • Invocation result context: BEP supplies structured command, target, result, artifact, and completion evidence that lets fleet tooling retain the cohort's outcome. It does not replace spawn-level placement evidence. In particular, ActionExecuted events cover failed actions by default; publishing them for every action requires explicit configuration. 5.5.1 Build Event Protocol (BEP) develops the event graph and completeness rules.5
  • Service-side identity: cache lookup/result, remote operation, scheduler attempt, and executor or worker-pool observations joined to the same action.

The evidence expected for each case differs:

Claimed outcomeMinimum distinguishing evidence
Remote cache hitReused result and referenced blobs, with no executor attempt for that action
Local subprocess or sandboxBazel spawn record plus local strategy/process evidence, and no joined remote operation
Persistent workerBazel worker request/process evidence for the action, not merely the mnemonic
Remote executionA remote operation and executor attempt joined to the action, followed by a reachable result
FallbackBoth the failed or unavailable preferred path and the accepted alternate path, with the fallback reason
Dynamic winnerBoth branch attempts, winner and accepted outputs, loser cancellation or completion, and consumed resources

Absence of one backend record is not enough to prove local execution: telemetry may be incomplete or joined with the wrong identity. Conversely, a remote AC or CAS request only proves remote data access. 6.1.4 Action and Result Lifecycle provides the control, data, and result transitions needed to disambiguate these cases.

think

Diagnose: A canary configured with --strategy=CppCompile=remote is green, and backend logs show many CAS reads but no joined executor attempts for the sampled actions. Did the compiler run remotely?

Reveal

Not necessarily. The strongest current hypothesis is remote cache reuse: CAS traffic proves remote data access, not execution. Check the Bazel execution log for whether those spawns executed, join their action or result identities to AC and CAS observations, and run a controlled miss or cache-bypass cohort. If fallback is permitted, also retain the preferred-path failure and local attempt; the configured strategy alone cannot distinguish these paths.

Promote a strategy only with bounded fallback

Fallback trades availability for diagnostic clarity and sometimes for correctness. During qualification, disable or tightly bound it so a green result cannot hide remote incompatibility, worker rejection, or sandbox failure. In normal operation, document which action classes may fall back, the alternate strategy, the triggering failures, and the evidence retained from both paths.

Promote the canary only when the strategy matrix shows correct outputs, better end-to-end outcomes for its named journey, acceptable local and remote resource cost, and observable placement. Roll back on output divergence, missing placement evidence, unexpected fallback, or a protected latency, reliability, or capacity regression. Concurrency limits are a separate control developed in 6.4.6 Execution Concurrency; increasing them is not a repair for a poor strategy choice.

key takeaway

Choose execution strategy per measured action class, not as one fleet-wide preference. Compare local, sandboxed, worker, remote, and mixed paths from action readiness through usable output, including queueing, startup, transfer, materialization, resource pressure, correctness, and failure behavior.

Configuration expresses intent; it does not prove placement. Join Bazel's spawn and timing evidence with BEP invocation outcomes and cache, remote-operation, scheduler, worker, and executor observations. A cache hit is not remote execution, a green build is not placement proof, and fallback is safe only when its scope, trigger, alternate path, and evidence are explicit. Promote a bounded canary only after its action-class matrix passes correctness and end-to-end outcome gates with a reversible strategy map.

Check your understanding · 3 questions

1.A canary sets a per-mnemonic remote strategy and finishes successfully. What can the operator conclude immediately?

Select one answer

2.Match each observation to the strongest conclusion it supports:

Drag each answer onto the matching prompt, or click an answer and then click a prompt

Answers
A remote Action Cache result is reused, with no joined executor attempt
An execution log records a remote runner and the action digest joins to a backend operation and executor attempt
BEP records a successful BuildFinished event for the cohort
The preferred remote path fails and the accepted alternate path records a local runner

3.Which evidence should a controlled fallback canary retain before the alternate path is accepted as understood?

Select all that apply

0 of 3 answered

Footnotes

  1. Commands and Options — broad and per-mnemonic strategy selection, local/sandboxed/worker/remote distinctions, and execution-log options 1 2

  2. Dynamic Execution — explicit local/remote racing, applicability constraints, resource trade-offs, cancellation, and profile-based diagnosis

  3. Breaking down build performance — separating clean and incremental cohorts and interpreting execution and resource metrics

  4. Creating Persistent Workers — worker applicability, avoided startup cost, cross-action caching, and the worker protocol boundary

  5. Build Event Protocol Glossary — invocation result events and the default publication boundary for ActionExecuted