4.5.5 Test Execution Output as Build Dependencies

extra

Test reports are usually evidence from the test runner, not files in the normal build graph. This pattern is for the rarer case where a downstream build target must depend on execution-time evidence: coverage directories, logs, safety reports, or other proof that has to be packaged with a shipped artifact.1 It deliberately turns one report-producing test run into a declared build action, so the report is a build output instead of a side effect.

A test log is not automatically an input to another target
A normal bazel test run creates runner-managed logs, XML, and an exit result. Those side effects are not ordinary graph artifacts that a sibling report rule can depend on.
NORMAL TEST TARGET
bazel test //tests:api_test
Bazel builds the test executable, then its test runner executes it and manages test.log, XML, timing, and pass/fail status.
No graph edge: a sibling //reports:api_report cannot declare “the log from that separate test run” as an action input. Running both targets does not connect their executions.
OPTION AREPORT INSIDE THE ACTION GRAPH
NICHEUse only when the report must become a graph artifact.
Run the test program again as a tool
Input/toolbuilt test executablewith runfiles
Declared outputsreport.jsonsummary.html
The report rule owns a new action. It must recreate the test environment, arguments, and runfiles behavior that it needs. It does not reuse Bazel’s previous test execution.
OPTION BCOLLECT OUTSIDE THE GRAPH
DEFAULTPrefer this for normal test reporting.
Let CI consume test-runner results
An external collector reads Bazel’s produced test logs/XML after bazel test and publishes dashboards or combined reports.
This preserves normal test semantics and avoids executing the test program twice.
Choose explicitly: a report action declares a new run and its outputs. A CI collector instead consumes the normal test runner’s side effects.

The Missing Edge

The phase boundary from 2.2 Three Phases of a Build is the source of the surprise. A cc_test target is a build target whose default output is the test executable. bazel test builds that executable and then runs it under Bazel's test runner, but the files created while the test runs are not the same thing as the target's default build outputs.2

The difference is concrete. A pkg_tar that lists :foo_test as a source packages the compiled test executable, not the coverage reports and logs produced when that executable later runs as a test.3 Bazel also has a standard bazel coverage workflow, but the official coverage guide documents an important limitation: some coverage output files are not treated as normal graph inputs, which is why the coverage report combination action has special remote-execution caveats.4

The Test Encyclopedia provides another clue. Tests may write undeclared outputs under TEST_UNDECLARED_OUTPUTS_DIR, and Bazel zips those under bazel-testlogs. That is useful for test diagnostics, but it is not the same as declaring a file that another build target can consume through DefaultInfo.files.5 If another rule needs a report as an input, you need a target that declares that report.

Create A Report Target

The durable pattern is a sibling report target. Keep the ordinary test target for normal developer workflows, then add a custom rule target whose action runs the compiled test binary and declares the report files or report directory as outputs.6

def quality_cc_test(name, **kwargs):
    native.cc_test(
        name = name,
        **kwargs
    )

    _test_report(
        name = name + "_report",
        test_binary = ":" + name,
        testonly = True,
    )

The reporting rule is just a rule-authoring application of 4.2.2 Actions. During analysis it declares outputs such as coverage/, logs/, or a merged report file. During execution, its action launches the test binary and writes exactly those declared outputs. Returning them through DefaultInfo(files = ...) makes the report target buildable and consumable by ordinary targets.7

That changes the packaging edge:

pkg_tar(
    name = "foo_report",
    srcs = [":foo_test_report"],
    testonly = True,
)

Now foo_report depends on a build target that produces the report artifacts. The reports are no longer hidden behind a previous bazel test invocation or a path under bazel-testlogs. They are reachable artifacts in the requested build graph.8

Gate It Behind A Config

Do not make every bazel build run tests. A safe compromise is to activate the report path only under a quality configuration, such as bazel build --config=quality ..., while keeping default cc_test behavior unchanged for normal bazel test usage.9

That split matters for API design. The wrapper macro should make the report target discoverable and predictable, but not silently change the meaning of a normal test target. This is similar to the public-surface discipline from 4.5.3 Example Workspaces as Contract Tests and 4.5.4 Stardoc — API Documentation: users should be able to tell which target is the regular test and which target is the report-producing build artifact.

For larger systems, put the quality config in CI policy and release gating. H.7.2 Evidence Portfolio develops the monorepo-scale decision of which tests run before merge, after merge, or nightly. The narrower rule-authoring point here is: if a report must be a build dependency, model it as an output of a build rule.

Preserve The Test Contract Deliberately

Running a test binary as a build tool is not equivalent to bazel test. Direct execution of test binaries is allowed but not endorsed, because that invocation does not necessarily follow the full Bazel test environment contract.10 A report rule that needs test-like environment variables, writable directories, runfiles, or exit-code handling must provide that contract itself instead of assuming the test runner did it.

That is the main caveat. Using the test as a tool can be flaky.11 The rule author has to decide which environment variables are stable, where the test may write, how report directories are cleaned, how failures are surfaced, and whether the action can run under the selected execution strategy. If the answer is "we just need coverage dashboards," the standard bazel coverage path is usually a better fit.12

This pattern is most defensible when a build artifact needs auditable evidence beside it: regulated automotive software, medical software, aerospace systems, or other release processes where reports are part of the delivered package.13 For ordinary developer feedback, keep tests as tests.

key takeaway

Do not try to depend on the side effects of bazel test. If a report must be an input to another build target, create a report-producing rule that declares the report as an output.

This is a niche pattern. It buys graph-shaped packaging and CI gates by accepting a harder rule contract: the reporting action must recreate the parts of test execution it relies on.

Check your understanding · 3 questions

1.Why does pkg_tar(srcs = [":foo_test"]) not package reports produced by running the test?

Select one answer

2.Which choices match the report-target pattern described in the article?

Select all that apply

3.True or false: running a test binary from a report rule has the same contract as bazel test.

Choose True or False for each sentence

A report rule must recreate any test-like environment variables or writable directories it relies on.
Directly running the test binary automatically gives the full Bazel test runner contract.
This pattern is most appropriate when reports must be packaged or audited as build artifacts.
The report target should replace the normal test target for everyday developer testing.
0 of 3 answered

Footnotes

  1. Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer — safety-critical release reports as the motivating use case.

  2. Test encyclopedia — tests are run by bazel test. Direct binary execution does not necessarily follow the test runner contract.

  3. Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauerpkg_tar(srcs = [":foo_test"]) packages the executable rather than execution reports.

  4. Code coverage with Bazelbazel coverage --combined_report=lcov and remote-execution caveat for coverage outputs not being normal graph inputs.

  5. Test encyclopediaTEST_UNDECLARED_OUTPUTS_DIR is zipped under bazel-testlogs for test outputs.

  6. Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauerlum_cc_test wrapper macro creates a sibling _report target.

  7. Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer — custom reporting rule takes the compiled test binary as a tool and declares reports as outputs.

  8. Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer — packaging uses srcs = [":foo_test_report"] to consume report artifacts.

  9. Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer — report generation is activated through --config=quality, leaving normal test behavior unchanged.

  10. Test encyclopedia — direct execution of test binaries is allowed but not endorsed because it bypasses the specified test environment.

  11. Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer — caveat that test-as-tool can introduce flakiness.

  12. Code coverage with Bazel — standard coverage workflow for producing LCOV reports from bazel coverage.

  13. Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer — automotive compliance requires execution reports and coverage proof.