5.10.2 Slow CI Consultant Practicum

This practicum asks you to investigate an unfamiliar slow CI workload without a guided sequence. You receive a synthetic dossier, a limited probe budget, and a deliverable contract. Your job is not to name every possible Bazel problem. It is to choose evidence that separates this dossier's competing explanations and write a diagnosis no broader than that evidence.

Complete the case before opening the reveal. 5.10.1 Unexpected Rebuild Walkthrough provides the integrated method, but this dossier deliberately changes the symptom and withholds a ready-made path through the tools.

Synthetic dossier: Atlas presubmit

All names, durations, counts, and observations below are synthetic teaching data, not captured Bazel output or performance baselines.

The Atlas repository runs this presubmit workload:

bazel test //services/payments/... //services/orders/...

The CI team reports that median elapsed time increased after a repository reorganization. Test results remain green. The same pinned Bazel version and worker image are used before and after the change.

You receive these records:

Evidence	Before	After
revision	before the repository reorganization	after the repository reorganization
repository structure	baseline shared BUILD macros and package boundaries	those macros and boundaries reorganized
elapsed time	9 min	15 min
loading + analysis	70 s	72 s
execution	7.5 min	13.5 min
remote action-cache hit rate	84%	83%
actions reported	11,800	18,900
longest single action	48 s	49 s

An engineer also supplies three notes:

• A controlled rerun of the same after revision on the same worker class remains near 15 minutes.
• CI command and rc provenance match across the comparison.
• No execution logs or narrowed graph queries have yet been collected.

These records make several explanations plausible: the reorganization may have expanded the target or action graph; it may have shifted work onto a broad dependency path; cache misses may have become more expensive despite a similar rate; or a small critical bottleneck may dominate the added time. A sandbox or undeclared-input problem is possible in the abstract, but the dossier contains no isolation symptom yet.

Constraints

You may request two additional probes before writing your first memo. Each probe must state:

1. the exact question it answers;
2. the comparison cohort or universe it uses;
3. the observation that would support one hypothesis over another;
4. the observation that would make you change branches.

Choose from the following, or propose an equally bounded Bazel probe:

• a scoped target-graph comparison;
• an action-plan comparison;
• paired execution logs;
• a matched timing-profile comparison;
• BEP-derived invocation data;
• focused remote-cache diagnostics;
• a sandbox or undeclared-input reproduction.

The optional branches described in 5.3.2 Practical Graph Analysis, 5.4.2 Critical Path, 5.5.3 Remote Cache Diagnostics, and 5.8.2 Diagnosing Sandbox Issues can deepen particular investigations. They are not hidden prerequisites: the dossier and the probe descriptions above contain enough information to select and justify two probes.

Deliverable

Write a one-page diagnosis memo with these headings:

• Incident boundary — workload, revisions, environment, and what “slow” means.
• Competing hypotheses — at least two explanations that predict different observations.
• Probe choices — two probes, in order, including why each is more discriminating than the alternatives.
• Evidence table — observation, inference, and rejected interpretation in separate columns.
• Bounded diagnosis — the strongest claim the supplied and requested evidence would justify.
• Evidence limits and next action — what remains unknown and the next falsifiable test or repair experiment.

Do not prescribe a fleet policy, shared-service SLO, worker-capacity program, or telemetry platform. This is one bounded workload across controlled invocations.

Rubric

Score each dimension from 0 to 2:

Dimension	0	1	2
Comparison control	mixes incomparable runs	records some controls	states workload, revision, configuration, worker, server, and cache boundaries
Hypotheses	lists causes without predictions	gives one discriminating prediction	keeps multiple falsifiable explanations and branch conditions
Probe economy	chooses tools by familiarity	one probe discriminates	both probes are sequenced by information gain
Interpretation	treats metrics as causes	separates some observations	distinguishes observation, inference, cascade, and absence of evidence
Diagnosis scope	makes a universal claim	adds a caveat	matches the exact cohort and states evidence limits
Next action	says “investigate more”	names a tool	predicts how a controlled repair or follow-up should change evidence

A strong submission scores at least 10 of 12 and has no zero in comparison control or diagnosis scope. There is no single mandatory tool pair. A scoped graph comparison followed by aquery, or an action-plan comparison followed by paired execution logs, can both be defensible if their branch conditions are explicit.