CI gates

The short version

• Every estate claims to have tests. The question that separates them is which of those tests has ever been seen to fail.
• A check that has only ever run against a working system is indistinguishable from one that cannot fail — and the second is worse than none, because it is a green light nobody earned.
• This panel lists every script the pipelines actually run, what its own source says it asserts, and whether CI proves it red on a broken fixture before trusting it green. The proportion that are is stated as measured, not as intended.

The cheapest way to build a system that looks safe is to add checks nobody has ever watched fail. They cost minutes of pipeline time, they turn green on every run, and they are load-bearing in exactly one respect: everyone downstream believes them. The estate’s answer is a rule that a gate ships with a self-test that proves it red — a deliberately broken fixture the gate is required to reject — and that the self-test runs in CI, on the same run that trusts the gate.

CI gate map — 104 scripts across 102 pipelines

Snapshot taken 25 Aug 2026, 12:16 AEST — build-time, not live

• 102pipeline definitions
• 74are checks
• 26ship a self-test
• 24proven red by CI

Search gatesRepositoryallansiblearronpitman-siteclaude-configdocker-stackstf-azuretf-backblazetf-cloudflaretf-proxmoxtf-tailscaleProofallproven red by CIships a self-testno self-testKindeverything CI runschecks onlysupporting scripts

104 of 104

Proof	Script	What it asserts
self-test only	ado_pipeline_metrics.pyansible/scripts/ · arronpitman-site	ado_pipeline_metrics.py — the ONE collector for ADO pipeline queue/run timing.
proven red	ado_pipeline_run.pyansible/scripts/ · ansible-ci	Trigger or wait on an ADO pipeline run (Epic #376, #923, #1824).
proven red	ado_to_actions.pyansible/scripts/ · ansible-ci	Convert Azure DevOps pipeline YAML into GitHub Actions workflows, deterministically.
proven red	agent_access_conformance_gate.pyansible/scripts/ · ansible-ci	Conformance gate over the register of AGENT-OPERATED SYSTEMS — Epic #2488, Issue #2518.
proven red	ci_secret_parity_probe.pyansible/scripts/ · ansible-ci-secret-parity	ci_secret_parity_probe.py — prove a GitHub Actions job resolves the SAME credentials as an ADO agent job on the same runner (Epic #2282, Issue #2285, acceptance criterion 3).
proven red	check_docs_toolchain_pins.pyansible/scripts/doc_gen/ · pitlab-docs	check_docs_toolchain_pins.py — pitlab-docs MkDocs toolchain pin-parity gate (Issue #2186).
none	check_exposure_register.pyansible/scripts/doc_gen/ · pitlab-docs	check_exposure_register.py — fail the docs build if an un-gated (posture-d) row on the External Exposure Register points at a vanished link target (Issue #1705, Epic #938).
none	generate_cloudflare_docs.pyansible/scripts/doc_gen/ · pitlab-docs	Cloudflare Access Map generator.
none	generate_docker_docs.pyansible/scripts/doc_gen/ · pitlab-docs	Docker stacks documentation generator.
none	generate_room_automations.pyansible/scripts/doc_gen/ · hass-config	Per-room "day in the life" pages, generated from hass-config.
none	generate_schedule_docs.pyansible/scripts/doc_gen/ · pitlab-docs	Maintenance schedule documentation generator.
none	generate_service_deps.pyansible/scripts/doc_gen/ · pitlab-docs	Service dependency map generator — ONE derivation, THREE renders (Epic #2260, ADR-0355).
none	generate_tf_docs.pyansible/scripts/doc_gen/ · 7 pipelines	Terraform documentation generator.
proven red	public_scrub_gate.pyansible/scripts/doc_gen/ · 2 pipelines	Fail-closed scrub gate for pages published to the public showcase site (Epic #2557, Issue #2559).
none	standards_enforcement_gate.pyansible/scripts/doc_gen/ · 2 pipelines	Standard-Enforcement meta-gate (Epic #1713, Issue #1715, ADR-0176).
none	test_check_exposure_register.pyansible/scripts/doc_gen/ · pitlab-docs	Proven-red regression for check_exposure_register.py (ADR-0141).
none	test_check_standards_completeness.pyansible/scripts/doc_gen/ · pitlab-docs	Regression + negative-control test for check_standards_completeness.py (Epic #1996, Issue #1997 — Build-Hook & Pipeline-Gate Test Standard, ADR-0141, PD12).
none	test_validate_compliance_catalogue.pyansible/scripts/doc_gen/ · ansible-ci	Regression + negative-control test for validate_compliance_catalogue.py (Epic #2247, Issue #2249 — Build-Hook & Pipeline-Gate Test Standard, ADR-0141, PD12).
none	test_validate_script_observability.pyansible/scripts/doc_gen/ · ansible-ci	Committed regression test for validate_script_observability.py (Build-Hook & Pipeline-Gate Test Standard, ADR-0141 / ADR-0219).
none	test_validate_standards_taxonomy.pyansible/scripts/doc_gen/ · pitlab-docs	Regression + negative-control test for validate_standards_taxonomy.py (Epic #1996, Issue #1997 — Build-Hook & Pipeline-Gate Test Standard, ADR-0141, PD12).
proven red	validate_adr_numbers.pyansible/scripts/doc_gen/ · pitlab-docs	no module docstring
proven red	validate_bin_manifest.pyansible/scripts/doc_gen/ · ansible-ci	Validate the control01 /usr/local/bin deploy manifests (Epic #2386).
none	validate_compliance_catalogue.pyansible/scripts/doc_gen/ · ansible-ci	Compliance control-catalogue resolution gate (Epic #2247, Issue #2249, ADR-0352).
none	validate_docker01_cac.pyansible/scripts/doc_gen/ · ansible-standards-enforcement-reconciliation	docker01 config-as-code conformance gate (Epic #886, Issue #2002).
none	validate_docs_published.pyansible/scripts/doc_gen/ · pitlab-docs	Build-time guard: every hand-authored doc in pitlab-docs must reach the site.
none	validate_notification_catalog.pyansible/scripts/doc_gen/ · ansible-ci	Static Notification-Catalog guard (Epic #1275, Issue #1278, ADR-0062).
none	validate_overview_hubs.pyansible/scripts/doc_gen/ · pitlab-docs	Blocking structural lint for overview-hub docs (ADR-0113, Documentation Architecture Standard).
none	validate_pipeline_path_coverage.pyansible/scripts/doc_gen/ · ansible-ci	CI guard: every pipeline's trigger paths cover the files its playbook actually consumes.
none	validate_schedule_tz.pyansible/scripts/doc_gen/ · ansible-ci	Static Timezone-Standard guard for pitlab host schedules (Epic #1270, Issue #1272).
none	validate_script_observability.pyansible/scripts/doc_gen/ · ansible-ci	Ratcheting unattended-script observability gate (ADR-0219, Epic #1852, Issue #1848).
none	validate_standards_taxonomy.pyansible/scripts/doc_gen/ · pitlab-docs	Standards-taxonomy gate (Epic #1996, Issue #1997, ADR-0252).
none	validate_systemd_selfheal.pyansible/scripts/doc_gen/ · ansible-standards-enforcement-reconciliation	systemd self-heal clean-reap guard (Epic #1921, Issue #1925, ADR-0234).
proven red	validate_userparam_barrier.pyansible/scripts/doc_gen/ · ansible-ci	validate_userparam_barrier.py — a NEW Zabbix UserParameter must be written ABOVE the `meta: flush_handlers` barrier in playbooks/sync_control01.yml.
proven red	github_schedule_watchdog.pyansible/scripts/ · ansible-ci	Freshness watchdog for GitHub Actions scheduled workflows.
proven red	groom_backlog_debt.pyansible/scripts/ · ansible-control01-sync	groom_backlog_debt.py — the backlog-grooming debt detective (Epic #1865, Issue #2476, ADR-0399).
proven red	ha_entity_name_conformance.pyansible/scripts/ · ansible-control01-sync	ha_entity_name_conformance.py — assert every Home Assistant friendly_name is formatted so the Assist conversation agent can actually match it.
proven red	ha_press_detector.pyansible/scripts/ · ansible-control01-sync	ha_press_detector.py — catch a silent Zigbee button break within minutes of use.
none	infisical_env.pyansible/scripts/ · arronpitman-site	Fetch-on-login interactive env for control01 (Epic #376, Issue #925 — Phase 4).
none	infisical_fetch_env.pyansible/scripts/ · 3 pipelines	Fetch a stack's secrets from Infisical and emit them as .env lines (Epic #376).
none	infisical_tf_vars.pyansible/scripts/ · 6 pipelines	Inject Infisical secrets into a Terraform ADO pipeline job as TF_VAR_* (Epic #376, Issue #926 — Phase 5).
none	loki_push.pyansible/scripts/ · ansible-standards-enforcement-reconciliation	Push a single structured log line to Loki (Epic #376, #923).
none	patch_reboot_coordinator.pyansible/scripts/ · ansible-ci	patch_reboot_coordinator.py — pitlab fleet PATCH+reboot coordinator (Epic #1769, Issue #1771).
none	pitlab-onboarding.shansible/scripts/ · 2 pipelines	no module docstring
none	pushover_notify.pyansible/scripts/ · 3 pipelines	Send a Pushover notification (Epic #376, #925).
none	secret_scan.shansible/scripts/ · 10 pipelines	no module docstring
none	test_ado_pipeline_run.pyansible/scripts/ · ansible-control01-sync	Regression gate for ado_pipeline_run.py's wait-path hardening (Issue #2210, ADR-0339).
none	test_freescout_distill.pyansible/scripts/ · ansible-control01-sync	Self-test for freescout_distill.py — stdlib only, no network, no live LLM/Qdrant/FreeScout.
none	test_ha_ownership_anomaly.pyansible/scripts/ · ansible-control01-sync	Proven-red regression for ha_ownership_anomaly.py (ADR-0141 discipline, Issue #2328).
none	test_haos_loki_logger.pyansible/scripts/ · ansible-deploy-haos-loki	Offline liveness self-test for the HAOS loki_logger custom component (Issue #2064).
none	test_stalled_grab_watchdog.pyansible/scripts/ · ansible-control01-sync	Self-test for stalled_grab_watchdog.py — stdlib only, no network, no live *arr.
self-test only	test_stuck_alert_watchdog.pyansible/scripts/ · ansible-control01-sync	Discrimination proof for the stuck-alert watchdog (Epic #2107, Issue #2158).
none	test_zabbix_cron_heartbeats.pyansible/scripts/ · ansible-control01-sync	Self-test for zabbix_cron_heartbeats_monitoring.py — stdlib only, no network, no live Zabbix.
proven red	validate_ci_pool_parity.pyansible/scripts/ · ansible-ci	validate_ci_pool_parity.py — assert every agent in the pitlab pool carries the SAME, DECLARED toolchain (Epic #2187, Issue #2206; design docs/designs/2187-ci-runner-pool.md).
proven red	validate_deploy_tags.pyansible/scripts/ · ansible-ci	validate_deploy_tags.py — conformance gate for the stack-deploy tag taxonomy.
proven red	validate_dt_project_namespaces.pyansible/scripts/ · ansible-ci	validate_dt_project_namespaces.py — the conformance gate for per-host Dependency-Track project namespacing (Issue #2121, ADR-0363).
proven red	validate_guard_ledger.pyansible/scripts/ · ansible-ci	Aggregating verdict gate for a multi-guard CI job (incident #2470).
proven red	validate_handler_durability.pyansible/scripts/ · ansible-ci	validate_handler_durability.py — conformance gate for pending-handler durability.
proven red	validate_pipeline_agent_portability.pyansible/scripts/ · ansible-ci	validate_pipeline_agent_portability.py — the conformance gate for pipeline agent-portability (Epic #2187, Issue #2216, ADR-0337).
proven red	validate_repo_script_locality.pyansible/scripts/ · ansible-ci	validate_repo_script_locality.py — conformance gate for repo-script execution locality.
proven red	validate_secret_scan_coverage.pyansible/scripts/ · ansible-standards-enforcement-reconciliation	validate_secret_scan_coverage.py — fleet-wide secret-scanning conformance gate (ADR-0277).
self-test only	validate_target_lock_coverage.pyansible/scripts/ · ansible-ci	validate_target_lock_coverage.py — the conformance gate for target-host serialisation (Issue #2199 step 7, ADR-0330).
proven red	waitfor_poll.pyansible/scripts/ · ansible-ci	waitfor poller — evaluate a TYPED predicate to a terminal state, always leaving a marker.
none	zabbix_monitoring_reconciler.pyansible/scripts/ · ansible-standards-enforcement-reconciliation	Monitoring-as-Code drift audit — the live half of the Zabbix config-as-code loop (Epic #833, Issue #1523).
none	assert_rendered.mjsarronpitman-site/scripts/ · arronpitman-site	Rendered-DOM acceptance for the four set-pieces (Epic #2557, Issue #2560).
proven red	build_snapshots.pyarronpitman-site/scripts/ · arronpitman-site	Emit the build-time data snapshots the four set-pieces render (Epic #2557, Issue #2560).
none	legibility_gate.mjsarronpitman-site/scripts/ · arronpitman-site	Diagram legibility gate for the published site (Epic #2591, Issue #2593).
none	lighthouse_gate.mjsarronpitman-site/scripts/ · arronpitman-site	Lighthouse gate for the published site (Epic #2557, Issue #2560).
proven red	prerender_mermaid.mjsarronpitman-site/scripts/ · arronpitman-site	Pre-render every Mermaid diagram in the site's content to static SVG, at build time, in both colour schemes (Epic #2591, Issue #2593; ADR-0456, which supersedes ADR-0028's rejection of SVG pre-render).
self-test only	prove_setpiece_scrub.sharronpitman-site/scripts/ · arronpitman-site	no module docstring
proven red	render_corpus.pyarronpitman-site/scripts/ · arronpitman-site	Select and render the `public: true` slice of the pitlab-docs corpus as Starlight pages.
proven red	rule_inventory_gate.pyclaude-config/scripts/ · claude-config-ci	Rule-inventory gate for the claude-config constitution (Epic #2420, Issue #2421).
proven red	artifact_gate.pydocker-stacks/scripts/ · docker-stacks-observability	Artifact acceptance gate for RUN-TO-COMPLETION workloads (Issue #2181, ADR-0321).
proven red	changed_image_trivy_gate.pydocker-stacks/scripts/ · docker-stacks-secret-scan	changed_image_trivy_gate.py — block a docker-stacks push that bumps an image pin to a tag carrying a CRITICAL vulnerability.
proven red	counter_birth_guard.pydocker-stacks/scripts/ · docker-stacks-observability	counter_birth_guard.py — static CI gate against the counter-birth false-page class (ADR-0292).
none	eit_conformance_gate.pydocker-stacks/scripts/ · 6 pipelines	Encryption-in-Transit conformance gate — fails a deploy when a NEW plaintext cross-host listener appears on docker01 that is not accounted for on the exceptions register.
proven red	gate_common.pydocker-stacks/scripts/ · 5 pipelines	Shared helpers for the docker-stacks post-deploy gates (Issue #2185, ADR-0317).
none	homepage_icon_gate.pydocker-stacks/scripts/ · docker-stacks-infra	Pre-deploy icon gate — fails the homepage deploy if any tile icon does not resolve to a real image, so the start page can never ship a generic "?" placeholder.
none	homepage_regen_parity_gate.pydocker-stacks/scripts/ · docker-stacks-infra	homepage_regen_parity_gate.py — fail the deploy when stacks/homepage/config/services.yaml is not what its generator would produce from its sources (Epic #1594, Issue #1967; ADR-0143).
proven red	host_stack_source_gate.pydocker-stacks/scripts/ · docker-stacks-infra	Host->repo stack-source reconciliation gate — every workload running on docker01 must have source.
none	loki_log_gate.pydocker-stacks/scripts/ · 6 pipelines	Post-deploy Loki log-shipping gate — fails the deploy if a just-deployed stack ships NO logs to Loki.
proven red	pipeline_coverage_gate.pydocker-stacks/scripts/ · 6 pipelines	Pipeline stack-list completeness gate — every deployed stack must be gated and triggered.
none	quiet_hours_guard.pydocker-stacks/scripts/ · docker-stacks-observability	quiet_hours_guard.py — static stuck-mute CI gate for the sleep quiet-hours window (ADR-0264).
none	refresh_consumer_roots.shdocker-stacks/scripts/ · 6 pipelines	no module docstring
proven red	renovate_config_gate.pydocker-stacks/scripts/ · 3 pipelines	renovate_config_gate.py — semantic CI gate over renovate.json (Issue #2150).
none	smoke_gate.pydocker-stacks/scripts/ · 6 pipelines	Pre-deploy smoke gate — fails the deploy if a just-deployed container does not start & serve.
none	test_eit_conformance_gate.pydocker-stacks/scripts/ · ansible-eit-reconciliation	Self-test for eit_conformance_gate.py — stdlib only, no network, no ssh.
proven red	testplan_gate.pydocker-stacks/scripts/ · 7 pipelines	Post-deploy service test-plan gate — runs each deployed stack's test plan, fails the pipeline on a functional regression, and (mode-gated) AUTO-ROLLS-BACK the offending deploy to last-good (Service Test Plan Standard, ADR-0102; Issues #1478 gate + #1479 auto-rollback).
self-test only	testplan_run.pydocker-stacks/scripts/ · 7 pipelines	Shared service test-plan runner — the one runner behind the Service Test Plan Standard (ADR-0102).
proven red	testplan_synthetic_gate.pydocker-stacks/scripts/ · docker-stacks-infra	Scheduled synthetic test-plan gate for host/LXC services (Service Test Plan Standard, ADR-0102; host/LXC tranche, Issue #1489).
none	check_dashboard_conformance.pydocker-stacks/stacks/observability/scripts/ · docker-stacks-observability	Dashboard conformance gate — enforces Pillar 2 + the bidirectional catalog at deploy time.
none	plan_classifier_gate.pytf-azure/scripts/ · 2 pipelines	plan_classifier_gate.py — decide whether a tf-azure apply may run UNATTENDED or must pause for Pit's approval.
none	test_plan_classifier_gate.pytf-azure/scripts/ · 2 pipelines	Regression test for plan_classifier_gate.py (ADR-0141 — build-hook/gate test standard).
none	plan_classifier_gate.pytf-backblaze/scripts/ · tf-backblaze	plan_classifier_gate.py — decide whether a tf-backblaze apply may run UNATTENDED or must pause for Pit's approval.
none	test_plan_classifier_gate.pytf-backblaze/scripts/ · tf-backblaze	Regression test for plan_classifier_gate.py (ADR-0141 — build-hook/gate test standard).
none	config_completeness_gate.pytf-cloudflare/scripts/ · tf-cloudflare	config_completeness_gate.py — fail the tf-cloudflare plan when a LIVE Cloudflare object is absent from terraform state and not on the ratified completeness allowlist (Issue #2040, ADR-0259 follow-up).
none	exposure_conformance_gate.pytf-cloudflare/scripts/ · tf-cloudflare	exposure_conformance_gate.py — fail the tf-cloudflare plan on an un-gated, unregistered public hostname (RULE 13 / ADR-0150).
none	plan_classifier_gate.pytf-cloudflare/scripts/ · tf-cloudflare	plan_classifier_gate.py — decide whether a tf-cloudflare apply may run UNATTENDED or must pause for Arron's manual approval (Epic #2057, RULE 14 scoped carve-out / ADR-0268).
none	test_config_completeness_gate.pytf-cloudflare/scripts/ · tf-cloudflare	Regression test for config_completeness_gate.py (ADR-0141 — build-hook/gate test standard).
none	test_exposure_conformance_gate.pytf-cloudflare/scripts/ · tf-cloudflare	Regression test for exposure_conformance_gate.py (ADR-0141 — build-hook/gate test standard).
none	test_plan_classifier_gate.pytf-cloudflare/scripts/ · tf-cloudflare	Regression test for plan_classifier_gate.py (ADR-0141 — build-hook/gate test standard).
none	plan_classifier_gate.pytf-proxmox/scripts/ · 2 pipelines	plan_classifier_gate.py — decide whether a tf-proxmox apply may run UNATTENDED or must pause for Pit's approval.
none	test_plan_classifier_gate.pytf-proxmox/scripts/ · 2 pipelines	Regression test for plan_classifier_gate.py (ADR-0141 — build-hook/gate test standard).
none	plan_classifier_gate.pytf-tailscale/scripts/ · tf-tailscale	plan_classifier_gate.py — decide whether a tf-tailscale apply may run UNATTENDED or must pause for Pit's approval.
none	test_plan_classifier_gate.pytf-tailscale/scripts/ · tf-tailscale	Regression test for plan_classifier_gate.py (ADR-0141 — build-hook/gate test standard).

32% of the checks are proven red by CI — that is,24 of 74 are run against a deliberately broken fixture on the same run that trusts them, and are required to fail. The rest are not, and that is the honest state of it rather than a rounded-up one. A gate nobody has seen go red is a green light nobody earned.

Derived, not listed: all 102 pipeline definitions were read from the build API, 60 of them run at least one repository script, and every one of those scripts was fetched and read for its own description. Every one resolved to a file.A script counts as a check when its filename declares it one —validate_, assert_, verify_, check_,test_, or a _gate/_guard/_conformancesuffix — which is this estate's own convention. A builder or a credential helper is listed and not counted.

Why “proven red” is a different column from “has a self-test”

They are deliberately separate, because collapsing them hides the interesting state. A self-test that exists and is never run is a gate nobody has seen go red; it has the shape of proof and none of the effect. Splitting the columns makes that state visible instead of rounding it up into the reassuring one.

The number that is not 100% is the honest part of this panel. Some of these checks are proved by hand at the time they are written and never again; that is a real gap, it is visible here, and publishing it is cheaper than the alternative, which is discovering it during an incident.

How this list is derived

Nothing here is typed in. All the project’s pipeline definitions are read from the build API, each one’s YAML is read from the repository it builds from, and every script that YAML runs is read for its own opening description. If a single definition cannot be resolved, the build fails rather than publishing a partial census — a smaller number presented with the same confidence is worse than no number, because nothing on the page could tell you which you were looking at. A hand-written list would be the specific failure this estate names authored, not observed: every entry correct, and the set silently short of whatever was added after the list was written.

Three things had to be got right before the list was trustworthy, and each was wrong first:

• The credential could read three of the hundred pipeline definitions. Reading everything through the API looked like the thorough choice, and the build identity’s authorisation is scoped to the repositories its own pipeline references — so the first published version of this panel covered 30 scripts and presented them with exactly the same confidence as the full set. The page’s own numbers agreed with the data behind it perfectly; only an absolute floor in the acceptance gate caught it, because an equality check between two consistent wrong numbers cannot. It now reads from the build agent’s own checkouts, which carry every repository, and refuses to publish at all if any definition is unreadable.
• Anchoring on an interpreter keyword — python3 …, node … — found the tidy invocations and missed the strictest gates on this very site, which are handed to a container image’s own entry point. Their absence looked exactly like “this pipeline has no rendered-DOM gate”.
• A step’s display name comes after the shell block it names, so attributing scripts to the last label seen shifts every step by one. It does not fail; it produces a complete, plausible, wrong map.

What counts as a check

A script counts when its filename declares it one — validate_, assert_, verify_, check_, test_, or a _gate / _guard / _conformance suffix. That is this estate’s own naming convention rather than a judgement made here, and stating the rule on the page is what makes a miscount a visible disagreement with something published rather than an invisible opinion. A builder or a credential helper is listed and not counted.