4.5.5 Test Execution Output as Build Dependencies
extraTest reports are usually evidence from the test runner, not files in the normal build graph. This pattern is for the rarer case where a downstream build target must depend on execution-time evidence: coverage directories, logs, safety reports, or other proof that has to be packaged with a shipped artifact.1 It deliberately turns one report-producing test run into a declared build action, so the report is a build output instead of a side effect.
bazel test run creates runner-managed logs, XML, and an exit result. Those side effects are not ordinary graph artifacts that a sibling report rule can depend on.bazel test //tests:api_testtest.log, XML, timing, and pass/fail status.//reports:api_report cannot declare “the log from that separate test run” as an action input. Running both targets does not connect their executions.The Missing Edge
The phase boundary from 2.2 Three Phases of a Build is the source of the surprise. A cc_test target is a build target whose default output is the test executable. bazel test builds that executable and then runs it under Bazel's test runner, but the files created while the test runs are not the same thing as the target's default build outputs.2
The difference is concrete. A pkg_tar that lists :foo_test as a source packages the compiled test executable, not the coverage reports and logs produced when that executable later runs as a test.3 Bazel also has a standard bazel coverage workflow, but the official coverage guide documents an important limitation: some coverage output files are not treated as normal graph inputs, which is why the coverage report combination action has special remote-execution caveats.4
The Test Encyclopedia provides another clue. Tests may write undeclared outputs under TEST_UNDECLARED_OUTPUTS_DIR, and Bazel zips those under bazel-testlogs. That is useful for test diagnostics, but it is not the same as declaring a file that another build target can consume through DefaultInfo.files.5 If another rule needs a report as an input, you need a target that declares that report.
Create A Report Target
The durable pattern is a sibling report target. Keep the ordinary test target for normal developer workflows, then add a custom rule target whose action runs the compiled test binary and declares the report files or report directory as outputs.6
def quality_cc_test(name, **kwargs):
native.cc_test(
name = name,
**kwargs
)
_test_report(
name = name + "_report",
test_binary = ":" + name,
testonly = True,
)
The reporting rule is just a rule-authoring application of 4.2.2 Actions. During analysis it declares outputs such as coverage/, logs/, or a merged report file. During execution, its action launches the test binary and writes exactly those declared outputs. Returning them through DefaultInfo(files = ...) makes the report target buildable and consumable by ordinary targets.7
That changes the packaging edge:
pkg_tar(
name = "foo_report",
srcs = [":foo_test_report"],
testonly = True,
)
Now foo_report depends on a build target that produces the report artifacts. The reports are no longer hidden behind a previous bazel test invocation or a path under bazel-testlogs. They are reachable artifacts in the requested build graph.8
Gate It Behind A Config
Do not make every bazel build run tests. A safe compromise is to activate the report path only under a quality configuration, such as bazel build --config=quality ..., while keeping default cc_test behavior unchanged for normal bazel test usage.9
That split matters for API design. The wrapper macro should make the report target discoverable and predictable, but not silently change the meaning of a normal test target. This is similar to the public-surface discipline from 4.5.3 Example Workspaces as Contract Tests and 4.5.4 Stardoc — API Documentation: users should be able to tell which target is the regular test and which target is the report-producing build artifact.
For larger systems, put the quality config in CI policy and release gating. H.7.2 Evidence Portfolio develops the monorepo-scale decision of which tests run before merge, after merge, or nightly. The narrower rule-authoring point here is: if a report must be a build dependency, model it as an output of a build rule.
Preserve The Test Contract Deliberately
Running a test binary as a build tool is not equivalent to bazel test. Direct execution of test binaries is allowed but not endorsed, because that invocation does not necessarily follow the full Bazel test environment contract.10 A report rule that needs test-like environment variables, writable directories, runfiles, or exit-code handling must provide that contract itself instead of assuming the test runner did it.
That is the main caveat. Using the test as a tool can be flaky.11 The rule author has to decide which environment variables are stable, where the test may write, how report directories are cleaned, how failures are surfaced, and whether the action can run under the selected execution strategy. If the answer is "we just need coverage dashboards," the standard bazel coverage path is usually a better fit.12
This pattern is most defensible when a build artifact needs auditable evidence beside it: regulated automotive software, medical software, aerospace systems, or other release processes where reports are part of the delivered package.13 For ordinary developer feedback, keep tests as tests.
Do not try to depend on the side effects of bazel test. If a report must be an input to another build target, create a report-producing rule that declares the report as an output.
This is a niche pattern. It buys graph-shaped packaging and CI gates by accepting a harder rule contract: the reporting action must recreate the parts of test execution it relies on.
Check your understanding · 3 questions
1.Why does pkg_tar(srcs = [":foo_test"]) not package reports produced by running the test?
Select one answer
2.Which choices match the report-target pattern described in the article?
Select all that apply
3.True or false: running a test binary from a report rule has the same contract as bazel test.
Choose True or False for each sentence
Footnotes
-
Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer — safety-critical release reports as the motivating use case. ↩
-
Test encyclopedia — tests are run by
bazel test. Direct binary execution does not necessarily follow the test runner contract. ↩ -
Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer —
pkg_tar(srcs = [":foo_test"])packages the executable rather than execution reports. ↩ -
Code coverage with Bazel —
bazel coverage --combined_report=lcovand remote-execution caveat for coverage outputs not being normal graph inputs. ↩ -
Test encyclopedia —
TEST_UNDECLARED_OUTPUTS_DIRis zipped underbazel-testlogsfor test outputs. ↩ -
Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer —
lum_cc_testwrapper macro creates a sibling_reporttarget. ↩ -
Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer — custom reporting rule takes the compiled test binary as a tool and declares reports as outputs. ↩
-
Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer — packaging uses
srcs = [":foo_test_report"]to consume report artifacts. ↩ -
Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer — report generation is activated through
--config=quality, leaving normal test behavior unchanged. ↩ -
Test encyclopedia — direct execution of test binaries is allowed but not endorsed because it bypasses the specified test environment. ↩
-
Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer — caveat that test-as-tool can introduce flakiness. ↩
-
Code coverage with Bazel — standard coverage workflow for producing LCOV reports from
bazel coverage. ↩ -
Why We Should Care About Test Execution Output in Safety-Critical Industries - Markus Hofbauer — automotive compliance requires execution reports and coverage proof. ↩