4.5 Rule Testing & Documentation

A ruleset is trustworthy when its public surface is proven, not merely described. Drift is the usual failure: the README shows a setup path CI never runs, generated docs list an attr no example uses, or a provider field changes without a test that names the contract. This section is about choosing the smallest proof layer that would catch that drift before a user does.

Think of the section as a proof ladder. Each step sees a different kind of truth, and each step gets noisy when asked to prove the wrong thing.

The Proof Ladder

4.5.1 Analysis-Phase Testing is the bottom rung: cheap, sharp, and analysis-only. Provider fields, output groups, declared actions, configuration-specific behavior, and intentional analysis failures all happen before actions run, so analysis tests can assert on them directly with bazel_skylib's analysistest helpers. The result is still a real Bazel test target. It passes or fails based on whether the rule analyzed the way the contract said it would. Use this layer for the contract inside one rule, especially the surfaces introduced in 4.4 Production Rule Surface.

4.5.2 Ruleset Integration Tests is the real-Bazel rung. Once the question becomes "does the public .bzl entry point work when another workspace loads it, sets up a module extension, registers a toolchain, and runs bazel build or bazel test?", analysis tests run out. Integration fixtures are normal Bazel test targets that drive a nested fixture workspace through the same commands users run. They catch regressions in generated repository names, runfiles paths, module-extension setup, and public consumption flows. Every escaped bug should leave behind a small workspace that would have failed before the fix. The wider release gate for those fixtures belongs in 4.11.5 Ruleset CI/CD Patterns.

4.5.3 Example Workspaces as Contract Tests is the consumer-shaped rung. An example is the repository a user is supposed to be able to copy: it loads only public entry points, sets up Bzlmod the documented way, and uses local_path_override so it depends on the ruleset like an external consumer. Treat examples as executable documentation, not decorative demos. They are most useful when they pin down a specific user story such as the README snippet, recommended toolchain setup, or a representative cross-package layout, not when they become a generic "build everything in examples/" target nobody reviews.

4.5.4 Stardoc — API Documentation is the reference rung. Rule, attr, provider, field, function, and macro docstrings live next to the declarations they describe, and a stardoc() target turns them into reproducible Markdown. That keeps documentation drift visible: a public symbol with no docstring becomes a hole in the generated page, and a reorganized .bzl file shows up as a bzl_library dependency update before it becomes user confusion. The cheapness depends on the docstrings already being there. Stardoc without doc = ... on public rules, attrs, and providers can be reproducible and still substantively empty.

4.5.5 Test Execution Output as Build Dependencies is the unusual side branch. It covers the rare pattern where execution-time evidence such as coverage data, audit logs, or regulated reports must become a declared build dependency other targets consume. Most rulesets should skip it until packaging or compliance really needs those reports as artifacts. For ordinary developer feedback, standard bazel test and bazel coverage paths are usually the better choice.

Docs And Tests Should Say The Same Thing

The strongest idea in this section is that API contracts should be expressed where they are also demonstrated and verified. A doc = "..." string on attr.label_list(...) is documentation, but it is also the spec Stardoc renders and the spec an example workspace should satisfy. An analysis test that reads a provider field is verification, but it is also the most precise statement of what the rule promises to publish.

An integration fixture that runs the public setup story is a test, but it is also the second README maintainers cannot easily forget to update.

The trap is to treat each layer as a separate chore. Docs become Markdown someone writes after the code ships. Examples become folders unconnected to CI. Integration tests become catch-all "did anything break?" suites. Analysis tests become optional once the rule "works." When that happens, the layers stop reinforcing each other: docs mention an attr no example uses, examples load private helpers, integration tests depend on internals, and analysis tests pass while the public provider drifts.

Choose The Narrowest Proof

Use analysis tests when the claim lives in analysis. They are excellent for provider shape, action registration, output groups, and failure text. They are wrong for "does the binary link and run?" or "does this generated artifact contain the right bytes?" Artifact-content checks belong to execution-time tests.

Use integration tests when the claim depends on a real downstream workspace, real commands, setup APIs, module extensions, toolchain registration, runfiles behavior, or runtime contracts such as 4.4.2 RunEnvironmentInfo that must be exercised through bazel run or bazel test. They are too blunt for every provider field.

Use examples and Stardoc when the public API is stable enough to teach. Stardoc generates the reference. The example proves the reference works from a consumer's repository. Both connect back to 4.4.1 Rule Public API Design: if a symbol is part of the supported surface, it should appear in both places.

Skip 4.5.5 Test Execution Output as Build Dependencies on a first read. Return to it only when execution-time evidence itself has to become a build artifact, not merely because a test should print useful output.

Explore the mini-ruleset to see the proof ladder assembled in one workspace: analysis tests, consumer examples, generated API documentation, and release metadata.