The gate that had never been seen red

The short version

Bottom line. A CI gate spends its whole life green, which means a gate that has quietly stopped working looks exactly like one that is doing its job.

• What it cost — controls that reviewed cleanly and could not fire: a substring match that matched its own comment, and a conformance sweep that returned zero findings because its discovery query was wrong.
• What changed — every gate ships a poisoned fixture and must reject it, and a sweep that finds nothing must prove it could have found something before reporting a pass.
• What it costs — a test to write and maintain for every control. It is the cheapest insurance in the estate, because the alternative is a green light with nothing behind it.

The problem

A CI gate is a control that is almost always green. That is its normal state, its designed state, and the state in which nobody looks at it. Which means a gate that has stopped working — a regex that no longer matches, a path that moved, a discovery step returning nothing — looks exactly like a gate that is working.

Two shapes of this were found in the estate, and they fail for different reasons:

A check that cannot fail. A substring match that also matched its own explanatory comment. An OR branch satisfied by a neighbouring element. A string present in a fallback payload whether or not the feature rendered. Each of these is green with the feature removed, which makes it not a test.

A check that folds “found nothing” into the pass path. A gate that enumerates a set and asserts every member conforms returns zero findings when the set is empty. Zero findings on a healthy fleet and zero findings because the discovery query was wrong are byte-identical, and one of them is a green light with nothing behind it.

The design

Two rules, deliberately narrow, both mechanically enforced.

Every build hook and pipeline gate ships its own proven-red regression test (ADR-0141). The test drives the real entry point — not a refactored inner function — over a poisoned fixture per pattern the gate claims to catch, asserts it rejects each one, then runs it over a clean set and asserts it passes. CI runs the self-test immediately before running the gate, so a gate that has quietly stopped catching something reds the build itself rather than waving the next violation through.

Fixtures, not the live tree, because a fixture can contain a violation on purpose. And the real entry point, because a test against an inner function proves the logic and not the wiring — and the wiring is what breaks.

A gate aggregating over a discovered set must distinguish empty from satisfied (ADR-0382). Discovery gets its own floor and its own exit code: below the floor is EMPTY_SET, which is neither pass nor fail but “I could not look”. The distinction that matters throughout this estate is that “I could not look” and “I looked and it was fine” must never share an exit code.

The trade-off

The cost is real: every gate is now roughly twice the code, and the fixtures need maintaining alongside the patterns. A gate with six forbidden patterns carries at least seven fixtures, and each one has to be updated when a pattern changes.

The cost was accepted because the alternative is not “a simpler gate” but “a gate nobody can distinguish from a stub”. There is no cheaper way to know a control works than to have seen it refuse something.

The second cost is more subtle and is the reason for the proven-red wording specifically. It is tempting to test a gate against a synthetic fixture that is easy to detect. That proves the regex compiles. The requirement is a fixture that resembles the real violation — the pattern as it actually appears in the corpus, with its real surrounding text — because the failure mode being guarded against is a pattern that matches the test case and not the world.

The evidence

• The publish gate protecting this very site was proven red first, one fixture per forbidden pattern, before it was ever trusted green — and its self-test runs in CI immediately before it does.
• The build-time data snapshot behind this site’s set-pieces carries the same shape: its sanitiser is fed a graph in which every field carries an estate identifier, and the assertion is required to abort on each one.
• The empty-set rule has its own visible instance here: the renderer that selects publishable pages exits with a distinct code when fewer than a floor of pages carry the flag, rather than publishing an empty site and reporting success.

Both rules generalise into the standard enforcement standard, which tracks which of the estate’s written standards have a machine check behind them and which are still on trust. A standard with no gate is a suggestion.