4.5.3 Example Workspaces as Contract Tests

recommended

An example workspace is executable documentation for a ruleset's public API. It shows the exact MODULE.bazel, BUILD files, loads, setup calls, toolchain registration, and commands a real consumer should be able to copy. When CI builds and tests that workspace, the example becomes a contract test: a change that breaks the documented user path breaks before release, not after users follow the README.1

Examples Sit Between Docs And Tests

A reusable ruleset already has several test layers. 4.5.1 Analysis-Phase Testing inspects providers, actions, and analysis-time invariants. 4.5.2 Ruleset Integration Tests covers the harness shape for running Bazel against fixture workspaces. Example workspaces test a different promise: "can a downstream repository use this API the way we document it?" That connects directly to 4.4.1 Rule Public API Design. If a rule, macro, provider, or module extension is part of the supported surface, at least one example should exercise the supported entry point rather than a private helper.

The official deploying guide places tests/ and examples/ next to the public .bzl entry point in a ruleset repository, and describes examples/ as a place to show users basic ways to use the rules.2 That placement is important. Examples are not random demos hidden in a blog post. They live beside the ruleset, evolve with the API, and are discoverable from the README and generated docs. Stardoc can document the attributes and providers in .bzl files. Examples show the same API in a working repository shape.3 The publishing and repository-layout side of that story continues in 4.11.1 Ruleset Layout & Public Entry Points, while 4.5.4 Stardoc — API Documentation focuses on generated API reference.

What The Workspace Should Prove

An example should cover the path a user is supposed to trust. For a small ruleset, one workspace may be enough:

examples/basic/
  MODULE.bazel
  BUILD.bazel
  srcs...

For a mature ruleset, examples usually form a small portfolio:

  • a minimal setup that proves the first README snippet still works
  • a common dependency shape, such as a library consumed by a binary or test
  • the recommended toolchain or module-extension setup path
  • a realistic cross-package layout that uses public loads only
  • one compatibility smoke path for older supported Bazel or dependency versions, when that support is part of the release promise

The common mistake is to treat examples as only tutorial prose. Rule authors often keep an examples/ folder and build all example targets in CI, because each rule introduces an API and a good test suite gives a valid usage example.4 That instinct is right, but the example only becomes a useful contract when it is explicit about the behavior it protects. "Build //... in examples" is a start. "build and test the supported user paths we document" is a contract.

Keep The Consumer Boundary Honest

An example workspace should behave like an outside repository, not like an internal package with privileged access. It should load public entry points such as //foo:defs.bzl, use documented setup macros or Bzlmod extensions, and avoid reaching into internal/, generated repositories, private providers, or helper files that users are not supposed to load. If the example needs a private symbol to work, either the public API is missing a supported entry point or the example is teaching an unsupported shortcut.

Bzlmod makes this consumer boundary concrete. A nested test module can depend on the parent ruleset with local_path_override, carry its own MODULE.bazel, and exercise the ruleset from the perspective of a downstream module.5 In rules_scala, nested repositories and modules, including ones under examples/, are used to validate behavior from that consumer perspective. Each nested module has its own module/workspace files, .bazelrc, and .bazelversion, and the parent repo excludes those nested modules from broad top-level target patterns with .bazelignore.6

That pattern catches failures that analysis tests cannot see: missing bazel_dep() instructions, broken use_extension() / use_repo() setup, incorrect local_path_override assumptions, undocumented toolchain registration, or examples that only work because they live inside the ruleset repository. It also keeps README snippets honest. If the docs tell a user to paste a bazel_dep() block and build //:hello, there should be an example workspace doing exactly that.

Make Examples Executable In CI

The minimum contract is simple: CI should run the same command the docs recommend. If the README says:

cd examples/basic
bazel test //...

then the ruleset's presubmit should do that too. The official deploying guide points to CI workflows that run tests on pull requests and main-branch changes before release.7 For published Bazel modules, the Bzlmod migration guide goes further: a source archive submitted to the Bazel Central Registry should include a test module in a subdirectory that depends on the module being published and contains examples or integration tests covering common APIs.8

Compatibility promises need a slightly stronger shape. Compatibility testing can use nested modules and generated test modules that switch Bazel versions, Bzlmod/WORKSPACE modes, dependency versions, and selected test targets.9 The smoke-test idea is useful for examples: choose representative targets that exercise the supported public surface, then build and test them under the combinations the ruleset claims to support.10 You do not need every example to run every matrix entry. You do need at least one consumer-shaped workspace to fail quickly when a release no longer satisfies its stated compatibility policy.

When Examples Are Not Enough

Do not replace focused rule tests with example workspaces. The official rule-testing guide calls analysis tests the best option for testing rule internals such as providers and actions, and also points out their size and boilerplate limits.11 Use analysis tests for provider fields, action shapes, failure messages, and rule-specific invariants. Use example workspaces for setup paths, public loads, module integration, run/test behavior, and documentation snippets.

Also be careful with how examples are run. A shell script that invokes Bazel inside a test can be useful, but it is not free: nested Bazel runs need separate output directories, version control, cleanup, and platform discipline. The naive "examples folder plus CI build" pattern has real flaws: it may be non-hermetic, non-reproducible, and outside Bazel's normal test runner if implemented as an ad hoc script.12 Bazel's test encyclopedia also warns that directly executing test binaries is allowed but not endorsed because it does not provide the full bazel test environment.13 The better design is deliberate: either model the example as a real Bazel test fixture, or make the CI command explicit and controlled enough that users can reproduce it.

Try the mini-ruleset workspace as a consumer-shaped example: its examples/basic package uses only the public ruleset facade and is exercised by the project's declared commands.

key takeaway

An example workspace is part of the ruleset API. Keep it consumer-shaped, load only public entry points, pin or select the Bazel/dependency versions it claims to cover, and run it in CI with the same commands users see in the docs.

Use focused rule tests for internals. Use examples to prove that the documented public path still works from a user's repository.

Check your understanding · 3 questions

1.What makes an example workspace a contract test for a ruleset?

Select one answer

2.Which properties should a consumer-shaped example workspace have?

Select all that apply

3.True or false: keeping examples honest as a contract surface.

Choose True or False for each sentence

An example workspace can simulate a downstream consumer of an in-development module by using local_path_override from its own MODULE.bazel.
Documented commands in the README should match the labels and entry points the example actually builds and tests.
Reaching into internal/ loads or generated repo internals is fine in examples as long as it makes the demo shorter.
Examples should be matrixed across every Bazel version and platform, even those the ruleset does not claim to support.
0 of 3 answered

Footnotes

  1. Bazel rules to test Bazel rules — rules introduce public APIs, and test suites provide valid examples of their use.

  2. Deploying Rules — recommended ruleset repository layout includes top-level tests/ and examples/, with examples showing basic user flows.

  3. Stardoc Documentation — Stardoc extracts rule, provider, macro, and function documentation from .bzl files.

  4. Bazel rules to test Bazel rules — common ruleset practice of building example workspaces in CI, plus the public-API motivation.

  5. Bzlmod Migration Guidelocal_path_override models a local module dependency from a consuming module.

  6. Migrating to Bazel Modules - Maintaining Compatibility, Part 3 — nested repositories/modules in rules_scala, including examples, .bazelignore, .bazelrc, .bazelversion, and local overrides.

  7. Deploying Rules — CI/CD guidance for rulesets and reusable workflows that run tests on PRs and main.

  8. Bzlmod Migration Guide — BCR source archives should include a test module with examples or integration tests covering common APIs.

  9. Migrating to Bazel Modules - Maintaining Compatibility, Part 3 — tests across Bazel versions, Bzlmod and legacy WORKSPACE modes, nested modules, and generated modules.

  10. Migrating to Bazel Modules - Maintaining Compatibility, Part 4 — parameterized smoke tests, representative targets, exact dependency versions, and build/test execution.

  11. Testing — analysis tests for providers, actions, failure testing, and configuration-specific behavior, with size and boilerplate caveats.

  12. Bazel rules to test Bazel rules — limitations of naive examples-as-tests: hermeticity, reproducibility, and test-runner integration.

  13. Test encyclopedia — directly executing test binaries is not endorsed because it does not follow the specified bazel test environment.