ADR-0141: Build hooks and pipeline gates ship their own proven-red regression test

Status: Accepted Date: 2026-07-08

Context

A single-character parser bug in pitlab-docs/hooks/wikilinks.py — the MkDocs build hook that renders and link-checks the whole docs corpus — aborted the entire mkdocs build and red-lined all docs publishing across three commits (Issue #1598). The hook is ~460 lines of load-bearing meta-tooling and, until that fix, had zero tests of its own. It is not alone: the docker-stacks post-deploy gate family (smoke_gate.py, loki_log_gate.py, testplan_gate.py), the docs generators (doc_gen/generate_*.py), and the assemble-time validators (validate_docs_published.py, validate_overview_hubs.py, validate_mermaid.mjs) are all code whose job is to guard or transform other artifacts, whose failure blast-radius is the whole corpus/deploy, and none carried a regression test asserting it still works.

The pattern was being re-derived per incident. Prime Directive 12 already says “a gate is only worth having if it would actually fail on the specific breakage it exists to catch” — but that discipline was applied to services (the Service Test Plan Standard, ADR-0102) and ad-hoc to the gates themselves. The gap: no standard required the gates and hooks to carry their own proven-red test. That is a recurring, in-scope missing-standard finding (RULE 9 / PD9), not a one-off.

Decision

Adopt the Build Hook & Pipeline-Gate Test Standard: any build hook or pipeline gate ships a committed regression test, co-located with it, that calls the real entry point over a fixture, carries a negative control, is proven red on the breakage it guards before green, and runs as a named CI step in the gate’s own environment before the protected artifact publishes.

• Scope is gate/transform tooling, not host scripts. In: code a pipeline invokes that can fail the build (a gate) or rewrites published artifacts (a transform/hook). Out: /usr/local/bin host cron/timer scripts, which the Script Standard and Pipeline-Gate Script Location Standard govern and which fail one host, not the corpus. Same execution-context pivot as the location standard.
• Real entry point, negative control, real artifact. The test drives the shipped code path (e.g. the hook’s on_files over a temp Files tree), asserts the true outcome (raises on bad input / passes on good, or checks rewritten output), and includes a case with genuine breakage to prove the gate still fires — closing the “an assertion satisfiable without the feature is not a test” trap (PD12 §4).
• Proven red-then-green. The test must be demonstrated failing on the broken code before it is trusted green, with the evidence recorded — the same red-before-green discipline PD12 demands of any gate.
• CI step in the gate’s own environment. Run in the identical image/interpreter the gate uses (a MkDocs hook test in the same squidfunk/mkdocs-material image mkdocs build uses) so no version skew lets the test pass while the gate fails; a discrete named step blocks the deploy legibly.

Established by the wikilinks-hook fix under Issue #1598: hooks/test_wikilinks.py drives the real on_files gate over a synthetic table-cell fixture plus a negative control, proven red on the pre-fix hook and green after, wired as the CI step “Test wikilinks hook”.

Alternatives considered

• Leave it as a per-incident habit. Rejected — the wikilinks bug shows an untested gate ships silently and fails whole-corpus; “we’ll add a test when it bites” means every gate is one typo from a red main. The recurring cost is exactly what a standard removes (PD9).
• Fold the requirement into the Script Standard. Rejected — build hooks (MkDocs hooks/) are not ansible/scripts/ scripts and never deploy to /usr/local/bin, so they fall outside that standard’s shape rules; and “must carry a proven-red test” is a meaty, cross-repo testing decision worth its own standard + ADR, not a clause bolted onto shape rules.
• Extend the Service Test Plan Standard to cover gates too. Rejected — that standard tests services’ functional behaviour via one shared runner and three service-deploy triggers; gates are tested by unit/integration harnesses in their own repo’s CI, a different mechanism. Keeping them separate but explicitly paired (this is the PD12 counterpart for tooling) is clearer than overloading one standard.
• Rely on mkdocs --strict / broader CI to catch hook regressions. Rejected — those catch content errors the gate reports, not defects in the gate’s own logic (a parser that stops resolving a valid form fails closed and looks like a content error). Only a test that exercises the gate over known-good input catches the gate breaking.

Consequences

• Every new or materially changed build hook / pipeline gate now owes a co-located, proven-red regression test wired into CI — a small authoring cost that makes the highest-blast-radius tooling self-defending.
• Existing untested gates (the docker-stacks gate family, the doc_gen generators, the assemble-time validators) are now non-conforming. Rather than migrate-on-contact, the backfill is run as a deliberate campaign under Epic #1599 (a scoping/convention child first, then per-repo tranches), so conformance is driven to completion and tracked, not left to happen incidentally.
• The docs pipeline gains a permanent “Test wikilinks hook” step; other gate pipelines gain an equivalent as their tests land.
• Reinforces PD12 structurally at the tooling layer, complementing the Service Test Plan Standard (ADR-0102) at the service layer.