5.6.7 bazel dump — Internal State Inspection
recommendedbazel dump takes a snapshot of state retained by the current Bazel server. The
cache model behind that retained state is introduced in 2.4 Caching & Incrementality. A dump
is useful after an ordinary command has populated that state: you can count
Skyframe nodes, inspect the package or action cache, summarize rule instances,
or export tracked Starlark allocations. It is a developer debugging interface,
not a stable query language, so use it to form a hypothesis rather than as a
machine-readable source of truth.1
5.6.8 Skyscope — Skyframe Visualizer provides a complementary interactive view after bazel dump
has established which retained graph state deserves inspection.
Capture the server state that produced the symptom
Run the workload first, then run dump with the same startup options and output
base:
bazel build --config=ci //app:release
bazel dump --skyframe=summary
The ordering matters. Bazel's server retains the in-memory graph and caches across invocations. A different output base selects a different server. Changing startup options can also restart the server. Record the Bazel version, output base, command, targets, configuration, and whether the server was cold or warm beside a capture. A dump taken after a restart does not describe the server that showed the problem.
Current Bazel 9.0.0 exposes these principal views:1,2
| Option | What it exposes | A useful question |
|---|---|---|
--action_cache | Action-cache content | Which entries are retained locally? |
--packages | Package-cache content | Which packages remain loaded? |
--rule_classes | Registered rule classes | Which rule kinds are available? |
--rules | Rule and aspect summaries, with counts and action counts | Which classes dominate the analyzed graph? |
--skyframe=summary or count | Aggregate Skyframe graph information | Which SkyFunctions dominate the retained graph? |
--skyframe=value, deps, or rdeps | Values or dependency edges between internal nodes | Why is a particular internal node retained or connected? |
--skylark_memory=<path> | A pprof-compatible profile of tracked Starlark allocations | Which Starlark call paths account for tracked bytes? |
--skyframe also has version-specific modes such as function_graph,
active_directories, and active_directories_frontier_deps in Bazel 9.0.0.
Use bazel help dump from the exact producing binary before choosing a mode.
For the edge-oriented modes, --skykey_filter=<regex> limits emitted SkyKey
names. It applies to deps, rdeps, and function_graph in this release.2
Start with summaries and counts. Full values and edges can be enormous, may
contain workspace paths or other sensitive build metadata, and are easy to
misread without Bazel source context. If the real question is about declared
targets, configured targets, or actions, prefer query, cquery, or aquery.
Those commands provide supported models. dump exposes implementation state.
Separate three different memory questions
A large Bazel process does not identify which layer owns the bytes. Choose the measurement that matches the question:
dump --rulesgives a coarse rule/aspect summary. Memory figures appear only when rule memory tracking was enabled for that server.1dump --skylark_memory=<path>exports allocations attributed by Bazel's Starlark memory tracker. It is not a general JVM heap dump. The historical option spelling is stillskylark, even though the language is called Starlark.1,2- JVM tools inspect Java heap or native/off-heap categories. They cover memory
outside the Starlark attribution profile, but do not map it back to
.bzlcall paths.
The Starlark view requires a server started with the Java Allocation
Instrumenter and RULE_MEMORY_TRACKER enabled. 5.6.6 Starlark Memory Profiling
gives the complete capture and pprof workflow. bazel dump merely exports data
collected by that instrumentation. It cannot reconstruct Starlark attribution
after an uninstrumented build.
Do not treat the byte totals as interchangeable. A rule summary is aggregated,
a Starlark pprof profile is attributed tracked allocation, and a JVM heap or
native-memory report describes different populations. The BazelCon investigation
that compared these tools found dump --rules size estimates inaccurate for its
large Starlark strings, while a general Java heap analyzer found the objects but
could not identify the responsible Starlark function.3 Correlate views instead
of forcing them to agree.
Add JVM native-memory evidence when RSS exceeds heap
If operating-system resident memory is much larger than the Java heap explained by heap tools, start a dedicated Bazel server with the JVM's Native Memory Tracking enabled:
bazel --host_jvm_args=-XX:NativeMemoryTracking=summary \
build --config=ci //app:release
bazel --host_jvm_args=-XX:NativeMemoryTracking=summary info server_pid
Use the returned PID with the JDK attached to that server:
jcmd <pid> VM.native_memory summary
Native Memory Tracking must be enabled when the JVM starts. Attaching jcmd
later cannot recover an earlier untracked baseline. Keep the same startup option
on subsequent Bazel commands so they address the same instrumented server.
Compare like-for-like checkpoints—for example, after the same cold build and
after the same warm incremental build—and correlate changing categories with
Java heap, Starlark, and Skyframe evidence. NMT categorizes JVM-managed native
allocations. It is not a complete explanation of process RSS and it does not
attribute bytes to Bazel rules.
Turn a dump into a bounded experiment
Use one view to narrow the next measurement:
- If
--skyframe=countgrows across matched warm invocations, compare node types, then filter a focuseddepsorrdepscapture. Do not infer a leak from growth alone: a warm server is expected to retain useful incremental state. - If
--rulespoints to a rule or aspect class with high counts, inspect whether the requested graph legitimately contains that many configured instances and actions. Counts do not prove that the implementation wastes memory. - If the Starlark profile identifies a wide user-defined call stack, change one data-shape decision and repeat the same instrumented capture. Then remove the profiler and measure the real workflow.
- If NMT grows while Java heap and tracked Starlark allocations remain stable, investigate the changing native category rather than rewriting Starlark on suspicion.
Save the raw output only with its producing Bazel version and reproduction metadata. Output fields, node names, modes, and memory-accounting details can change between releases. Scripts that parse a dump silently turn an internal diagnostic snapshot into an unsupported compatibility dependency.
Populate one known Bazel server with the workload, then use the smallest
bazel dump view that answers the next diagnostic question. Summaries and
counts narrow the search. Detailed Skyframe edges, rule summaries, and tracked
Starlark allocations test different hypotheses and must not be conflated.
Use JVM Native Memory Tracking when process memory is not explained by heap evidence. Pin every capture to its Bazel version and server context, and treat all dump output as internal, version-sensitive evidence—not a stable format or proof of cause.
Check your understanding · 4 questions
1.Match each diagnostic question to the smallest useful bazel dump view:
Drag each answer onto the matching prompt, or click an answer and then click a prompt
2.A team wants to parse a detailed Skyframe dump in a dashboard that must keep working across Bazel upgrades. What is the best response?
Select one answer
3.A Bazel server's RSS is much larger than its explained Java heap. Which conclusions and next steps are sound?
Select all that apply
4.A memory-growth symptom appeared after a warm CI build. Which actions preserve a safe, comparable evidence capture?
Select all that apply
Footnotes
-
Commands and Options —
dumppurpose, supported views, rule summaries, and memory-tracking requirements ↩1 ↩2 ↩3 ↩4 -
Command-Line Reference — Bazel 9.0.0 dump option names, Skyframe modes, filters, and pprof output ↩1 ↩2 ↩3
-
Dude, where's my ram? An adventure in finding a ram thief in starlark land — observed limitations of rule-size reporting and general JVM heap attribution ↩