5.6.7 bazel dump — Internal State Inspection

recommended

bazel dump takes a snapshot of state retained by the current Bazel server. The cache model behind that retained state is introduced in 2.4 Caching & Incrementality. A dump is useful after an ordinary command has populated that state: you can count Skyframe nodes, inspect the package or action cache, summarize rule instances, or export tracked Starlark allocations. It is a developer debugging interface, not a stable query language, so use it to form a hypothesis rather than as a machine-readable source of truth.1

5.6.8 Skyscope — Skyframe Visualizer provides a complementary interactive view after bazel dump has established which retained graph state deserves inspection.

Capture the server state that produced the symptom

Run the workload first, then run dump with the same startup options and output base:

bazel build --config=ci //app:release
bazel dump --skyframe=summary

The ordering matters. Bazel's server retains the in-memory graph and caches across invocations. A different output base selects a different server. Changing startup options can also restart the server. Record the Bazel version, output base, command, targets, configuration, and whether the server was cold or warm beside a capture. A dump taken after a restart does not describe the server that showed the problem.

Current Bazel 9.0.0 exposes these principal views:1,2

OptionWhat it exposesA useful question
--action_cacheAction-cache contentWhich entries are retained locally?
--packagesPackage-cache contentWhich packages remain loaded?
--rule_classesRegistered rule classesWhich rule kinds are available?
--rulesRule and aspect summaries, with counts and action countsWhich classes dominate the analyzed graph?
--skyframe=summary or countAggregate Skyframe graph informationWhich SkyFunctions dominate the retained graph?
--skyframe=value, deps, or rdepsValues or dependency edges between internal nodesWhy is a particular internal node retained or connected?
--skylark_memory=<path>A pprof-compatible profile of tracked Starlark allocationsWhich Starlark call paths account for tracked bytes?

--skyframe also has version-specific modes such as function_graph, active_directories, and active_directories_frontier_deps in Bazel 9.0.0. Use bazel help dump from the exact producing binary before choosing a mode. For the edge-oriented modes, --skykey_filter=<regex> limits emitted SkyKey names. It applies to deps, rdeps, and function_graph in this release.2

Start with summaries and counts. Full values and edges can be enormous, may contain workspace paths or other sensitive build metadata, and are easy to misread without Bazel source context. If the real question is about declared targets, configured targets, or actions, prefer query, cquery, or aquery. Those commands provide supported models. dump exposes implementation state.

Separate three different memory questions

A large Bazel process does not identify which layer owns the bytes. Choose the measurement that matches the question:

  1. dump --rules gives a coarse rule/aspect summary. Memory figures appear only when rule memory tracking was enabled for that server.1
  2. dump --skylark_memory=<path> exports allocations attributed by Bazel's Starlark memory tracker. It is not a general JVM heap dump. The historical option spelling is still skylark, even though the language is called Starlark.1,2
  3. JVM tools inspect Java heap or native/off-heap categories. They cover memory outside the Starlark attribution profile, but do not map it back to .bzl call paths.

The Starlark view requires a server started with the Java Allocation Instrumenter and RULE_MEMORY_TRACKER enabled. 5.6.6 Starlark Memory Profiling gives the complete capture and pprof workflow. bazel dump merely exports data collected by that instrumentation. It cannot reconstruct Starlark attribution after an uninstrumented build.

Do not treat the byte totals as interchangeable. A rule summary is aggregated, a Starlark pprof profile is attributed tracked allocation, and a JVM heap or native-memory report describes different populations. The BazelCon investigation that compared these tools found dump --rules size estimates inaccurate for its large Starlark strings, while a general Java heap analyzer found the objects but could not identify the responsible Starlark function.3 Correlate views instead of forcing them to agree.

Add JVM native-memory evidence when RSS exceeds heap

If operating-system resident memory is much larger than the Java heap explained by heap tools, start a dedicated Bazel server with the JVM's Native Memory Tracking enabled:

bazel --host_jvm_args=-XX:NativeMemoryTracking=summary \
  build --config=ci //app:release
bazel --host_jvm_args=-XX:NativeMemoryTracking=summary info server_pid

Use the returned PID with the JDK attached to that server:

jcmd <pid> VM.native_memory summary

Native Memory Tracking must be enabled when the JVM starts. Attaching jcmd later cannot recover an earlier untracked baseline. Keep the same startup option on subsequent Bazel commands so they address the same instrumented server. Compare like-for-like checkpoints—for example, after the same cold build and after the same warm incremental build—and correlate changing categories with Java heap, Starlark, and Skyframe evidence. NMT categorizes JVM-managed native allocations. It is not a complete explanation of process RSS and it does not attribute bytes to Bazel rules.

Turn a dump into a bounded experiment

Use one view to narrow the next measurement:

  • If --skyframe=count grows across matched warm invocations, compare node types, then filter a focused deps or rdeps capture. Do not infer a leak from growth alone: a warm server is expected to retain useful incremental state.
  • If --rules points to a rule or aspect class with high counts, inspect whether the requested graph legitimately contains that many configured instances and actions. Counts do not prove that the implementation wastes memory.
  • If the Starlark profile identifies a wide user-defined call stack, change one data-shape decision and repeat the same instrumented capture. Then remove the profiler and measure the real workflow.
  • If NMT grows while Java heap and tracked Starlark allocations remain stable, investigate the changing native category rather than rewriting Starlark on suspicion.

Save the raw output only with its producing Bazel version and reproduction metadata. Output fields, node names, modes, and memory-accounting details can change between releases. Scripts that parse a dump silently turn an internal diagnostic snapshot into an unsupported compatibility dependency.

key takeaway

Populate one known Bazel server with the workload, then use the smallest bazel dump view that answers the next diagnostic question. Summaries and counts narrow the search. Detailed Skyframe edges, rule summaries, and tracked Starlark allocations test different hypotheses and must not be conflated.

Use JVM Native Memory Tracking when process memory is not explained by heap evidence. Pin every capture to its Bazel version and server context, and treat all dump output as internal, version-sensitive evidence—not a stable format or proof of cause.

Check your understanding · 4 questions

1.Match each diagnostic question to the smallest useful bazel dump view:

Drag each answer onto the matching prompt, or click an answer and then click a prompt

Answers
Which SkyFunctions dominate the retained graph?
Which rule or aspect classes dominate the analyzed graph?
Which packages remain loaded in this server?
Which Starlark call paths account for tracked allocations?

2.A team wants to parse a detailed Skyframe dump in a dashboard that must keep working across Bazel upgrades. What is the best response?

Select one answer

3.A Bazel server's RSS is much larger than its explained Java heap. Which conclusions and next steps are sound?

Select all that apply

4.A memory-growth symptom appeared after a warm CI build. Which actions preserve a safe, comparable evidence capture?

Select all that apply

0 of 4 answered

Footnotes

  1. Commands and Optionsdump purpose, supported views, rule summaries, and memory-tracking requirements 1 2 3 4

  2. Command-Line Reference — Bazel 9.0.0 dump option names, Skyframe modes, filters, and pprof output 1 2 3

  3. Dude, where's my ram? An adventure in finding a ram thief in starlark land — observed limitations of rule-size reporting and general JVM heap attribution