4.5.2 Ruleset Integration Tests

Ruleset integration tests answer a different question than analysis tests. An analysis test can inspect providers and registered actions for one target. An integration test asks whether a downstream workspace can load the ruleset, declare real BUILD targets, run Bazel, and get the behavior the ruleset promises.¹ That matters because the public API of a ruleset is not only a provider shape from 4.2 Custom Rules, Providers & Actions. It also includes .bzl entry points, macros, module setup, generated repositories, toolchain registration, runfiles, and the command lines users actually run.

Test Like A User

The fixture should look like a tiny user project, not like another internal unit test package. Give it its own MODULE.bazel or WORKSPACE, real BUILD.bazel files, sample source files, and the same public load statements that documentation tells users to copy. Ruleset tests double as both verification and valid examples of the rules' public API: a contributor should be able to change the implementation with confidence that the supported usage still works.²

That is the main boundary from 4.5.1 Analysis-Phase Testing. Analysis tests are excellent when you need to assert on providers, actions, failure messages, or configuration-dependent analysis behavior. The official testing guide calls them the best option for testing a rule's inner workings, while also noting that they are intentionally small and limited by analysis-test dependency limits.¹ Ruleset integration tests sit outside that boundary. They exercise the ruleset through Bazel itself.

A useful fixture normally contains the smallest workspace that proves one user-facing path:

test-scenarios/
  BUILD.bazel
  compile_smoke_test.sh
  compile_smoke/
    MODULE.bazel
    BUILD.bazel
    main.mock

The outer workspace defines the test target. The nested compile_smoke/ workspace is the user project. The runner script changes into the nested workspace and invokes Bazel against labels such as //:app or //:all. This shape is the core idea behind rules_bazel_integration_test: test cases are ordinary workspace directories, and an executable test runner receives environment variables for the Bazel binary and workspace directory.³

What To Cover

Start with flows that users will copy first. If the ruleset exposes mock_binary, the first fixture should build or run a small binary through the public .bzl entry point. If setup requires a module extension, include the extension tag and generated repo usage in the fixture's MODULE.bazel. If a rule requires a toolchain, make the fixture register or consume that toolchain the way the README says.

Then add fixtures from regressions. A bug that escaped analysis tests often lives in the space between pieces: a generated repository name, a runfiles path, an execution-platform assumption, or a macro default that only fails after a real Bazel command runs. Each fixed bug should leave behind one small workspace that would have failed before the fix. Over time, the integration suite becomes a map of the ruleset's supported surface.

Do not make every fixture a miniature monorepo. One fixture should prove one behavior. If a single scenario needs a module extension, toolchain resolution, generated files, and a runtime test, keep it because the behavior is genuinely integrated. If those are separate public promises, split them so a failure points to one contract.

Keep It Inside Bazel When You Can

The weakest pattern is "examples exist, and CI runs some ad hoc shell script over them." The plain examples-folder approach has three problems: Bazel is not itself invoked under Bazel's test runner, the Bazel version may not be pinned, and local files, shell state, or environment variables can leak into the run.⁴ It is still better than no coverage, but it loses the test runner behavior that users already get from bazel test.

A stronger shape is a Bazel test target whose job is to run Bazel on the fixture workspace. The runner can be a shell script, Go test, Python test, or another executable that your ruleset already supports. The simple baseline is buildable targets plus shell tests verifying execution. Failure testing with Bazel-in-Bazel is harder because it needs separate output directories, and integration-test helper rules exist to manage that setup.⁵

For success-path smoke tests, the runner is often very small:

cd "$BIT_WORKSPACE_DIR"
"$BIT_BAZEL_BINARY" build //:app
"$BIT_BAZEL_BINARY" test //:app_test

The exact environment variable names depend on the helper rule. The architectural point is stable: the test target declares the Bazel binary, fixture workspace, and expected command sequence. That lets the integration test appear in ordinary bazel test //... output and gives CI one target label to run.

Include Module And Release Fixtures

For a published Bzlmod ruleset, a fixture with its own MODULE.bazel is not optional decoration. The Bzlmod migration guide recommends that source archives include a test module in a subdirectory. That test module is a Bazel project with its own WORKSPACE and MODULE.bazel, depends on the module being published, and contains examples or integration tests for the most common APIs.⁶

The rules_testing repository keeps this route deliberately small: e2e/bzlmod/ redirects the released module name to the checkout with local_path_override and preserves a focused regression for unexpected downstream toolchain resolution.⁷ It is evidence for one consumer-visible failure boundary, not a template for turning every unit case into a nested Bazel invocation.

That test module is the same idea at release time: prove the module the way a downstream user will consume it. It should avoid private repository shortcuts. Use bazel_dep(), use_extension(), use_repo(), registration calls, and public loads exactly as user documentation describes them. The publishing and matrix automation around that check belongs later in 4.11.5 Ruleset CI/CD Patterns. The test shape starts here.

The official rules-deployment guide makes the repository-level version of the same recommendation. A ruleset repository should have a recognizable layout with tests/ and optionally examples/, and many rulesets use CI workflows to run tests on pull requests and main before release workflows run on tags.⁸ The next item, 4.5.3 Example Workspaces as Contract Tests, focuses on examples as executable documentation. In this item, the examples matter when they are wired into a real test signal.

Control The Matrix

Integration tests are expensive because each fixture may start a separate Bazel server, fetch repositories, configure toolchains, and execute real actions. That cost is the reason to keep local integration tests narrow and move broad compatibility to CI. Experience from rules_haskell shows the tension clearly: nested workspace tests were easier to read, but cache reuse and multiple Bazel versions required deliberate design to avoid minutes of setup per test.⁹

Use two layers:

local / presubmit:
  current supported Bazel version
  smallest fixtures for public API and recent regressions

release / scheduled CI:
  multiple Bazel versions
  supported platforms
  examples, docs generation, and publication gates

The local layer protects contributors from breaking the ruleset on ordinary changes. The release layer proves the compatibility promise that users read in the README. Do not hide a known compatibility gap by skipping the test silently. Tag it, document it, or move it to the matrix where it belongs.

The maintained rules_bazel_integration_test map makes both layers executable. Its examples/simple/ fixture runs a controlled Bazel binary against a child workspace, while the README's bazel_versions matrix separates ordinary semantic versions from rolling compatibility checks. This is a ruleset harness contract. It does not make every Bazelisk FORK/VERSION form a supported matrix entry.

Read Failures As API Feedback

A failing ruleset integration test is usually not "just CI." It says the public story does not work somewhere: setup instructions are incomplete, a macro hides a private assumption, a runfiles path is wrong, a toolchain is only registered in the main repo, or a generated repo name changed. The fix may be code, documentation, or a narrower compatibility claim.

That is why these tests are most valuable when the fixture stays readable. A reviewer should be able to open the nested workspace and see the same shape a user would write. If the fixture needs five helper layers before it reaches a public rule call, it is probably testing your test harness more than the ruleset.