Logging Standard

This is the single domain standard for how pitlab captures every workload’s logs and keeps them for the right window — one home for the two decisions that used to live as separate micro-standards: that logs reach Loki at all (log capture — required destination, content path, stream labels, and a proven “logs land in Loki” gate), and how long each stream is retained (named retention tiers, an override-justification rule, and a bounded storage budget). Each is preserved below as a self-contained, subject-named rule section with its conformance checklist verbatim; the consolidated Enforcement table carries every machine and review obligation across both. This domain-unit consolidation is governed by ADR-0252; the per-decision rationale lives in the ADRs cited inline in each section (ADR-0075, ADR-0184).

How this standard is organised — two rule sections, one domain

This standard folds two former standards into one domain per the domain-unit rule (the unit of a standard is the domain, ADR-0113 / ADR-0252). Read the section for the concern you have; each is self-contained with its own conformance checklist:

• Log capture — every workload ships logs to Loki — the single log destination, the three capture gaps, stdout/tail content rules, stream-label taxonomy, ingestion paths, the proven “logs land in Loki” gate, and log-content alerting (ADR-0075).
• Loki log retention — named retention tiers, the override-justification and ADR-reference rules, the tier→stream registry, and the bounded storage budget (ADR-0184).

---

Log capture — every workload ships logs to Loki

This is the log-capture standard for pitlab: every service, container, and host must ship its logs to Loki, and the logs that ship must include the diagnostic ones — not just a startup banner. It exists because “Loki log shipping” in Pillar 2 was unmeasurable, so a third of running containers shipped nothing and the most decisive application logs (written to files) never reached the platform at all. This standard replaces “ships some logs” with a checkable shape — required destination, required content path, required stream labels, and a proven “logs land in Loki” gate — that closes the gap and keeps it closed. The incident that surfaced it and the rationale live in ADR-0075; it is the RULE 9 / Pillar 2 companion to the Dashboard Standard.

Every workload ships logs to Loki — Loki is the single log platform

There is one log destination in pitlab: Loki (obs-loki:3100). Every service, container, VM, LXC, host, and network device ships its logs there — there is no second log store, no “check the box on the host” exception, and no workload that is allowed to keep its logs only inside itself. A log that is not in Loki is, for operational purposes, invisible: it cannot be searched, correlated, alerted on, or retrieved by the agent. The motivating failure was a FreeScout licence error that sat in a container file for three days, unsearchable, while a downstream symptom alerted instead of the cause (Issue #1338).

The three capture gaps this standard closes

A workload can fail log capture in three distinct ways. The standard names each so a review can check for all three rather than only the obvious one:

Gap class	Symptom	Fix this standard requires
Coverage	The container/host ships nothing to Loki — no stream exists for it at all.	Wire it into an Alloy source (docker discovery, journal, file tail, or syslog relay) so a stream appears.
Content	A stream exists (e.g. via stdout) but the diagnostic logs are written to a file the stdout path never sees — FreeScout-class.	Redirect the app’s file logging to stdout/stderr, or agent-tail the file. The error lines must be searchable in Loki, not just the banner.
Host	A non-Docker VM/LXC/host ships container logs but not its own journald/syslog, or has no Alloy agent at all.	Run the Alloy agent (or the syslog-relay path for non-Alloy hosts) so the host journal reaches Loki.

Coverage is the floor; content is where the FreeScout incident lived; host is the long tail. All three are in scope for “logs land in Loki” — a workload is not done until none of the three applies to it.

Apps that log to files must redirect to stdout/stderr or be agent-tailed

The Docker→Loki path (loki.source.docker) only sees a container’s stdout/stderr. An app that writes its real logs to a file inside the container is invisible on that path even though a stream exists for it (the Content gap). The required fixes, in priority order:

1. Redirect the app to stdout/stderr — the preferred fix, because it needs no extra mount and no Alloy change. Most frameworks support it via config:
   • Laravel: LOG_CHANNEL=stderr (this is the exact FreeScout fix — storage/logs/laravel-*.log is a file the stdout path never reads).
   • Python: a StreamHandler to sys.stderr, or logging.basicConfig() default.
   • Nginx: access_log /dev/stdout; error_log /dev/stderr;.
   • Many LinuxServer.io / s6 images log only an init banner to stdout and the real app log to a file — check, don’t assume.
2. Agent-tail the file — when the app genuinely cannot log to stdout. Bind-mount the log file to a host path and add a loki.source.file target in the Alloy config (the model sab01 and qbittorrent already use). This carries the same required stream labels (host, env, job="file", service, device_type).

Redirect is preferred over tail: a tail adds a bind mount, an Alloy component, and a second failure mode (the file moves/rotates and the tail goes stale). Reach for the tail only when stdout is not an option.

Required stream labels conform to the authoritative Loki taxonomy

Every stream shipped to Loki carries the stream-label set defined as code in ansible/standards/loki-labels.yml (the authoritative taxonomy, v1.4) — this is the tagging standard (RULE 10) applied to logs. Labels must be low-cardinality: each unique label combination is a Loki index key and a new stream, so a high-cardinality value (a request ID, a timestamp, a user) belongs in structured metadata, never a stream label.

The journal SYSLOG_IDENTIFIER (the per-unit/per-script tag) is the canonical example: since ADR-0219 moved unattended scripts to journald (job=journal), a script’s own lines are selected by its identifier. That identifier is structured metadata syslog_identifier (taxonomy v1.4), not a stream label — there is one value per systemd unit/script (systemd, sshd, CRON, kb-draft-agent, …), far too many to index. Query a script’s run log with {job="journal", host="control01"} | syslog_identifier=`kb-draft-agent` — replacing the brittle CONTENT-anchor filters used before the promotion (Issue #1868). It is promoted at ingest by a loki.relabel → loki.source.journal → loki.process (stage.structured_metadata + stage.label_drop) chain in both Alloy configs; the internal source label is __journal_syslog_identifier (single underscore — an application field, unlike the trusted _SYSTEMD_UNIT → __journal__systemd_unit).

Label	Meaning	Applies to
env	Deployment environment — always homelab.	every stream
job	Source/pipeline type: journal, docker, file, unifi-syslog, ha-syslog, pihole-dns, linux-syslog.	every stream
host	Originating device — Ansible inventory hostname, or the syslog HOSTNAME field.	every stream
group	Infra tier: physical, vms, lxc, network.	every stream
device_type	Functional role: media, app, infra, monitoring, security, dns-resolver, …	every stream
compose_project	The per-service join key (matches cAdvisor’s compose-project label) — joins a service’s logs and metrics for the Dashboard Standard.	job=docker only
service	The service name — matches the dashboard UID and the blackbox probe_success{service} label.	file-tailed app streams
level	Normalised severity: error, warn, info, debug. Grafana colour-codes it.	syslog / ha-syslog (derived)
vendor	Hardware vendor — ubiquiti for network gear only.	network gear
facility	Syslog facility name.	unifi-syslog only

Adding a new label value is a deliberate edit to loki-labels.yml, the same guardrail-against-sprawl model as the docs tag vocabulary — not an ad-hoc string in one Alloy block.

The five ingestion paths each have a fixed onboarding route

How a workload reaches Loki depends on what it is. There is exactly one correct path per source kind — use it, don’t invent a new one:

Source	Path	Where it is configured
Docker container (stdout)	loki.source.docker auto-discovers every container on docker01.	docker-stacks/stacks/observability/templates/alloy/config.alloy — discovery is automatic; a new container ships with no config change (if it logs to stdout).
Docker container (file log)	loki.source.file tailing a bind-mounted host path.	Same config.alloy (the sab01/qbittorrent pattern) + the bind mount in docker-compose.yml.j2.
Linux host/VM/LXC journal	Alloy agent loki.source.journal.	ansible/roles/alloy/templates/config.alloy.j2, deployed by deploy_alloy.yml.
Non-Alloy Linux host (no native binary, e.g. armv7l)	journal → rsyslog imjournal → obs-syslog-relay → Alloy.	ansible/playbooks/deploy_syslog_forwarder.yml (the mesh01 pattern).
Network gear / Home Assistant	UniFi → UDP 514 → obs-syslog-relay (3164→5424) → Alloy TCP 1515; HA → loki_logger HTTP push direct to Loki.	The relay container + HA custom component — see syslog-loki-pipeline.

A new Docker workload that already logs to stdout is the only zero-config case: discovery picks it up. Everything else is a code change in one of the files above, committed (RULE 6 / RULE 7).

“Logs land in Loki” is a Pillar 2 definition-of-done, proven not assumed

Wiring up a log path is not the same as logs arriving. A service does not satisfy Pillar 2 until log arrival is verified against the live Loki API — the gap existed precisely because shipping was assumed, never checked. The verification is a LogQL query that returns recent lines for the service’s stream:

# Replace <name> with the container_name / service. Expect a non-empty result
# with a recent timestamp. Empty = the workload is dark (Coverage gap).
logcli query '{container_name="<name>"}' --limit 5 --since 15m   # or via the Loki HTTP API
curl -sG 'http://obs-loki:3100/loki/api/v1/query_range' \
  --data-urlencode 'query={container_name="<name>"}' --data-urlencode 'limit=5' | jq '.data.result | length'

For a Content-gap fix, the assertion is stronger: query for a known diagnostic line (an error the app actually emits), not just any line — a banner-only stream passes a naive presence check but fails the real test. The estate-wide version of this check — set-differencing running containers against Loki streams, and the time-window trap that makes a healthy estate look one-third dark — is the Loki log-coverage audit. This verification is now codified as the structural RULE 12 post-deploy gate: docker-stacks/scripts/loki_log_gate.py runs in every deploy pipeline’s Smoke stage and fails the run red if a just-deployed stack ships nothing to Loki (ADR-0086, Epic #1339 / Issue #1344, building on the pre-deploy smoke gate). The gate matches {compose_project="<stack>"}∪{service="<stack>"} so file-tailed services count, and uses a two-tier window — an informational 1h liveness window and a 7d dark fail-threshold (the audit-proven all-ship window) — so an idle-but-wired stack is reported quiet and passes while a genuinely dark stack fails closed (quiet ≠ dark). The gate catches never ships; a transient shipping break for an already-active workload is the in-prod Loki absence/ruler alert’s job, not the deploy gate’s. Remediation items close only on shown evidence the logs actually arrive.

Log-content alerting catches causes, not just downstream symptoms

A log capture is only as good as what it lets you catch. Where a service emits a known error signature that is the root cause of an incident class, a Loki ruler rule on that signature must exist so the cause pages directly — not a downstream heartbeat-staleness symptom two systems away (the FreeScout MTTD-vs-time-to-root-cause failure). Loki ruler rules live as code under docker-stacks/stacks/observability/templates/loki/rules/. This is Pillar 3 (Alerting) applied to log content: if a log line could have caught an incident earlier, an alert on it must exist before the work closes. The canonical example is FreeScout’s production.ERROR / site_inactive line — invisible for three days, now an alertable signature once the Content-gap fix ships it to Loki.

A log-derived alert must not be able to fire on re-ingested history

Loki timestamps a line when it is ingested, not when the application wrote it. So any path that causes an agent to re-read a file it has already shipped will replay historical lines into Loki stamped as though they were happening now — and a Loki ruler rule cannot tell the difference. The rule is not broken, the log is not corrupt, and the service is healthy; the alert is simply describing a past event as a present one. Every live check the responder runs comes back clean, which reads as a flaky alert rather than a shipping defect.

This estate has hit the class three times, from three unrelated causes, which is why it is a standard and not a service note:

Cause	Mechanism	Evidence
Lost read positions	A compose command: override dropped Alloy’s --storage.path, so positions lived in an ephemeral container path. Every recreate re-tailed every file from offset 0.	Six stale FreeScoutApplicationErrors from replayed laravel-2026-07-*.log; undetected 2026-05-30 → 2026-07-26 (Issue #2020)
Glob matches rotated generations	An app rotating as App.log → App.1.log re-presents each rotation as a new file under a bare *.log glob, so the whole file ships again.	PlexTranscodeErrors fired on 3 Aug errors as a 6,384-line “burst” dated 9 Aug (2026-08-13)
Cumulative tail in a script	A fixed tail -n N of a cumulative log re-publishes previous runs’ lines on every run.	A fixed HTTP 401 reappearing in journald after the fix landed (Epic #2129)

The rule. When adding or changing a file-based log source, or authoring a log-content alert, the capture path MUST be replay-safe:

• Read positions must survive a restart or recreate. For Alloy, a command: override MUST repeat --storage.path and that path MUST be a persistent mount. Losing positions is the highest-blast-radius form of this defect because it replays every tailed file at once.
• A file glob MUST NOT match rotated generations. Establish how the app rotates before writing the glob. Number-after-extension (app.log.1, standard logrotate) is safe against *.log; number-before-extension (App.1.log) is not and requires an exclusion (path_exclude → Alloy’s __path_exclude__).
• A script shipping its own diagnostics MUST ship only the current run’s bytes — record the offset before the run and emit from it, or write the run to a per-run temp file. Never a fixed tail of a cumulative file.
• When diagnosing from a log-derived alert, verify the frame before believing the event. Compare the line’s own embedded timestamp against the query window, and check the filename stream label. A mismatch beyond ingest lag means replay, not incident.

Duplicate ingest is the quieter half of the same defect and is worth checking independently: it inflates stream count and retention cost with no alert to announce it. Nine real Plex log files were producing 28 Loki streams, one rotated copy out-shipping the live file 1,066 lines to 811 over 7 days.

Custom exporters must log scrape failures to stderr, not only a <name>_up=0 metric

A custom Prometheus exporter (a hand-written stdlib /metrics server) that catches every scrape exception and returns only <name>_up 0 is non-conforming — the failure is detectable (the metric flips) but not diagnosable (no line says whether it was a 401, a timeout, or a DNS failure). Every exception branch that degrades a scrape MUST emit one ERROR line to sys.stderr with flush=True (which Alloy ships to Loki), naming the target and the exception type/message; the exporter also emits a one-line startup banner so its Loki stream exists, and stays otherwise silent so a healthy scrape is not per-scrape noise. This is the Content gap applied to exporters: <name>_up=0 with an empty Loki stream reads identically to “exporter idle” and leaves an on-call operator with nothing to act on. The canonical form is the _log() helper in paperless_exporter.py / n8n_workflow_exporter.py / plex_tautulli_exporter.py (ADR-0213). A swallowed exception surfacing only <name>_up=0 fails this standard.

Conformance checklist for a workload’s log capture

A workload’s logging conforms when all of these hold — this is the checklist the /wrapup gate and reviewers apply, alongside the Dashboard Standard checklist:

• A Loki stream exists for the workload (no Coverage gap) — verified by a live LogQL query returning recent lines.
• Diagnostic/error logs are present in Loki, not only a startup banner (no Content gap) — file-loggers redirected to stdout/stderr (LOG_CHANNEL=stderr etc.) or agent-tailed.
• For non-Docker hosts: the host’s own journald/syslog reaches Loki (no Host gap).
• Stream labels conform to ansible/standards/loki-labels.yml — env, job, host, group, device_type present; compose_project on job=docker; values low-cardinality.
• The capture path is committed as code (Alloy config / Ansible role / compose mount) — no manual-only log wiring (RULE 6 / RULE 7).
• Log arrival is proven against the live Loki API before the work closes (the RULE 12 “logs reach Loki” assertion) — not assumed. For a docker-stacks workload this is enforced structurally by the loki_log_gate.py deploy gate (ADR-0086); a stack must be in its pipeline’s gate list.
• Any known root-cause error signature has a Loki ruler alert so the cause pages, not just a downstream symptom (Pillar 3).
• The capture path is replay-safe: read positions persist across a recreate (Alloy --storage.path on a persistent mount, repeated in any command: override), file globs exclude rotated generations where the app numbers before the extension, and any script shipping its own diagnostics emits only the current run’s bytes. A log-derived alert that can fire on re-ingested history is non-conforming.
• Custom Prometheus exporters log every scrape-failure branch to stderr (target + exception), not only a <name>_up=0 metric — a swallowed exception with an empty Loki stream is non-conforming.

---

Loki log retention

This is the log-retention standard for pitlab: every Loki stream keeps its logs for a named retention tier, a longer-than-default window is a deliberate, ADR-backed per-stream override rather than a copy-paste of the last one, and total retention growth is bounded by a storage budget and watched by an alert. It exists because Loki retention was an implicit single global window (retention_period: 720h / 30 days) with no policy, and ADR-0172 then added the first per-stream override ({job="ha-syslog"} → 90 days) with no standard behind it — the exact “one-off without the standard” trap RULE 9 warns about, where the next service needing longer retention copies that block and the overrides silently drift. This standard replaces “everything is kept for 30 days, except the exceptions someone remembers” with a checkable shape — named tiers, an override-justification rule, an ADR-reference rule, and a bounded storage budget. It is the Logging Standard’s retention companion (that standard governs that logs reach Loki; this one governs how long they stay), the Loki counterpart to the Backup & Retention Standard (which governs PBS backups, not logs), and the rationale + rejected alternatives live in 0184-loki-log-retention-standard.

Retention is one global floor plus priority-ordered per-stream overrides

Loki has exactly one retention mechanism, and this standard is expressed entirely in its terms. In stacks/observability/templates/loki/loki.yaml, limits_config.retention_period sets a single global window that applies to every stream, and limits_config.retention_stream is a priority-ordered list of {selector, priority, period} rules that override the global window for the streams a selector matches. The compactor (with retention_enabled: true, retention_delete_delay: 2h) is what actually enforces both — it deletes chunks past their effective window. There is no per-tenant or per-label retention beyond this; a tier is therefore either the global floor or a retention_stream entry.

The three named retention tiers are short-7d, default-30d, and forensic-90d

Retention is expressed as a named tier, never an ad-hoc per-stream hour count, so a stream declares which tier it sits in and the tier fixes the Loki period. The current live config ratifies exactly two of these — the global retention_period: 720h (default-30d) and the single {job="ha-syslog"} → 2160h override (forensic-90d); short-7d is defined for the high-volume-low-value case and currently has no members. Tiers are deliberately few — three is the whole vocabulary:

Tier	Loki period	Window	When to use
short-7d	168h	7 days	High-volume, low-forensic-value streams nobody queries past a week (verbose debug, chatty access logs). Trims cost below the default. No current members.
default-30d (floor)	720h (retention_period)	30 days	The global floor every stream gets unless overridden — general operational and diagnostic logs. This is the presumption, not a choice to justify.
forensic-90d	2160h	90 days	Low-volume streams that must be correlated across a long arc — release-to-release forensics on a monthly-cadence component, or slow-burn incident investigation. Current member: {job="ha-syslog"} (ADR-0172).

A tier is the only retention vocabulary a stream may use. Introducing a fourth tier or changing a tier’s period is a deliberate edit to this table plus its own ADR — reviewed like any config change — not a one-off period: value buried in a single retention_stream entry.

The two axes are forensic reach and volume — longer retention needs high value AND low cost

Every tier assignment is decided on two axes, and they pull against each other:

• Forensic reach — how far back you genuinely need to query this stream. This is the benefit of longer retention. Name the concrete question (“in which HA release did this automation start misfiring?”, “what did this service log the week before last month’s incident?”). “We might want it someday” is not a forensic question and does not clear the default.
• Volume — how many bytes per day the stream ingests. This is the cost of longer retention: retained bytes ≈ daily volume × window, so a high-volume stream at 90 days costs ~3× its 30-day footprint.

The rule falls straight out of the two axes: default-30d is the balanced presumption; a longer window (forensic-90d) is justified only when forensic reach is high AND volume is low so the cost stays proportional; a shorter window (short-7d) is justified when volume is high AND forensic reach is low. High-volume and high-reach is the case a per-stream override cannot solve — it blows the budget — and is an escalation (sample the stream, split out the forensic subset, or accept 30 days), not an override.

A stream stays on default-30d unless it clears a tier’s criterion in writing

The override-justification rule is the heart of this standard — it decides when a retention_stream entry is warranted versus when a stream stays on the floor:

• The default is the presumption. A stream sits on default-30d with no retention_stream entry and no ADR. The overwhelming majority of streams stay here. Absence of an override is the normal, conforming state — not a gap.
• An override to forensic-90d requires all three: (1) a written forensic question that 30 days demonstrably cannot answer (the reach axis), (2) a volume assessment showing the stream is small enough that 90 days fits the Loki storage budget (the cost axis), and (3) a governing ADR (below). Miss any one and the stream stays on default-30d.
• An override to short-7d requires a demonstrated high daily volume and no query need past 7 days — a stream that is pure noise after a week. Note reject_old_samples_max_age: 168h already bounds ingest of late samples to 7 days; short-7d aligns retention to the same horizon for such streams.
• “Longer is safer” is not a justification. Every override is a standing storage cost and a line item on the audit surface below. The presumption defends the budget; the burden of proof is on the override, never on the default.

Every non-default retention_stream entry MUST reference an ADR — no silent overrides

A retention_stream entry is a per-stream policy exception, and every exception carries its reasoning as code. Each entry’s comment in loki.yaml MUST cite a governing ADR — exactly as the live {job="ha-syslog"} block cites ADR-0172. The ADR records the forensic question, the volume assessment, and the tier chosen, so the why survives the next editor. An override without an ADR reference is a conformance violation, full stop:

• The ADR is the override’s justification made durable. loki.yaml shows what is retained and for how long; the ADR shows why this stream earned a non-default window and why the volume fits the budget — the two things a bare period: 2160h cannot convey.
• The tier→stream registry below is the audit surface. Every retention_stream entry in loki.yaml maps to exactly one row here: a named tier and a governing ADR. A retention_stream selector present in loki.yaml but absent from this table — or present with no ADR — is drift to be reconciled, the same discipline the Backup & Retention Standard applies to backup tiers.
• Changing a stream’s window is a diff to loki.yaml + an ADR, deployed by the observability pipeline — never a live hand-edit on docker01 (RULE 7). Note that any change to loki.yaml triggers a full observability-stack restart (the deploy.yml handler), so batch retention edits rather than churning the file per-comment.

The tier→stream registry is the audit surface — every override maps to a tier and an ADR

This table is the source of truth for “which streams have a non-default retention, and why”. It reconciles 1:1 with the retention_stream list in loki.yaml: every override there is a row here; every row here is an override there. The global floor is listed for completeness but is not a retention_stream entry.

Stream selector	Tier	Loki period	Governing ADR	Justification (forensic question · volume)
(all streams, unset)	default-30d	retention_period: 720h	— (the floor needs no ADR)	The balanced presumption for general operational + diagnostic logs.
{job="ha-syslog"}	forensic-90d	2160h	ADR-0172	“In which HA Core release did this behaviour change?” — HA releases are ~monthly, so 30 days equals the release cadence and the previous release’s logs age out exactly when its successor is investigated. Stream is low-volume (WARNING+ plus targeted INFO), so 90 days costs little.

As of this standard, {job="ha-syslog"} is the only retention_stream override in loki.yaml — the PD9 cross-check for crept-in overrides came back clean, so no undocumented exceptions had to be folded in. When the next override lands, it adds one row here and one ADR, and this table stays the complete picture.

The storage budget bounds Loki to 15 GiB of the shared observability share

Retention growth must be bounded and observable, so total retained log bytes have a ceiling. Loki’s chunks live on the shared arch01-obs virtiofs share on docker01 (/mnt/arch01/observability, 40 GiB), which it co-tenants with the Prometheus TSDB:

• The budget: Loki ≤ ~15 GiB of the 40 GiB share. At authoring, Loki holds 4.4 GiB (chunks 4.4 GiB, index/WAL negligible) and the whole share is 6.7 GiB / 40 GiB used (17%) — roughly 3× headroom under the budget, and the budget itself leaves ~25 GiB for the Prometheus TSDB and slack. The budget is what a new forensic-90d override must fit inside: an override whose daily-volume × 90 days would push Loki past ~15 GiB is the high-volume case that must escalate, not override.
• It is observable at the share level today. The ObservabilityStorageLow Prometheus alert (stacks/observability/templates/prometheus/rules/homelab.rules.yml) fires warning → Pushover when node_filesystem_avail_bytes{mountpoint="/mnt/arch01/observability"} drops below 20% (i.e. the share exceeds 32 GiB used), with a runbook that explicitly names “check Prometheus TSDB + Loki retention sizes; prune old data if needed.” So runaway retention growth already pages — Pillar 3 is satisfied for the failure mode that matters (the disk filling).
• Per-stream ingest volume is visible on the committed loki-log-volume-errors Grafana dashboard — the input to the volume half of every tier decision.

The share-level alert can’t attribute growth to Loki vs Prometheus — a bounded observability gap

ObservabilityStorageLow watches the combined Loki + Prometheus TSDB footprint on arch01-obs, not Loki alone. It reliably catches the disk filling, but it cannot by itself say which co-tenant grew or which stream × tier drove it. Closing that finer gap — a per-tier retained-bytes recording rule (sum by (tier) (...)) that makes the 15 GiB budget self-policing and attributable — is a bounded Pillar 3 enhancement, not a blocking gap (the runaway signal exists). It is noted here so the next retention change that meaningfully grows the footprint knows the attribution tooling is worth adding rather than re-deriving the gap.

Conformance checklist for a Loki retention change

A retention change conforms when all of these hold — the checklist /code-review and /wrapup apply, and the shape the audit surface above is reconciled against:

• Every stream’s retention is a named tier (short-7d / default-30d / forensic-90d), not an ad-hoc period: value.
• A stream keeping longer than default-30d has a written forensic question 30 days can’t answer and a volume assessment showing it fits the ~15 GiB Loki budget.
• Every non-default retention_stream entry’s loki.yaml comment cites a governing ADR (as {job="ha-syslog"} cites ADR-0172).
• The tier→stream registry reconciles 1:1 with the retention_stream list in loki.yaml — no override without a row, no row without an override.
• The change is a diff to loki.yaml deployed by the observability pipeline, not a live hand-edit on docker01 (RULE 7), and retention edits are batched (any loki.yaml change restarts the observability stack).
• Total Loki footprint stays within the ~15 GiB budget; ObservabilityStorageLow covers the share (Pillar 3), and a footprint-growing change considers the per-tier attribution enhancement.
• A new tier (beyond the three) is a deliberate edit to the tiers table plus its own ADR — never a fourth window smuggled into a single entry.

---

Enforcement

This is the consolidated enforcement declaration for the whole Logging domain — the log-capture and log-retention rule sections above resolve to one table (per the Standard-Enforcement Standard, the ## Enforcement section is the single source of truth the meta-gate resolves against). Log capture carries one standing machine gate (the docker-stacks loki_log_gate.py deploy check) with the rest decided by conformance-checklist review; retention is a mixed standard whose structural obligations are review-with-a-machine-path and whose budget bound is a live Prometheus alert. Every named machine mechanism resolves against the live repos; the review clauses are applied by /wrapup and /code-review.

Obligation	Class	Layer	Mechanism	Dead-man
Every workload ships logs to Loki and arrival is proven against the live Loki API before close (no Coverage gap)	machine	deploy	loki_log_gate.py (docker-stacks deploy Smoke stage, ADR-0086) matches {compose_project}∪{service} and fails a dark stack red	n/a
Apps logging to files redirect to stdout/stderr or are agent-tailed so diagnostic lines — not just a banner — reach Loki (no Content gap)	review	—	/wrapup logging conformance checklist item “diagnostic/error logs present in Loki, not only a startup banner” + /code-review of the Alloy/compose change	n/a
Non-Docker VM/LXC/host ships its own journald/syslog to Loki (no Host gap)	review	—	/wrapup conformance checklist item “host’s own journald/syslog reaches Loki” verified by a live LogQL query (capture path deployed by deploy_alloy.yml / deploy_syslog_forwarder.yml)	n/a
Stream labels conform to the loki-labels.yml taxonomy and stay low-cardinality	review	—	/wrapup conformance checklist label line + /code-review of loki-labels.yml / Alloy config edits (no standing machine validator resolves shipped labels against the taxonomy)	n/a
The log-capture path is committed as code — no manual-only wiring (RULE 6 / RULE 7)	review	—	/wrapup conformance checklist item “capture path committed as code” + /code-review	n/a
Known root-cause error signatures carry a Loki ruler alert so the cause pages, not a downstream symptom (Pillar 3)	review	—	/wrapup Pillar 3 alerting review / conformance checklist final item, backed by ruler rules under docker-stacks/stacks/observability/templates/loki/rules/	n/a
The capture path is replay-safe — Alloy read positions persist a recreate (--storage.path on a persistent mount, repeated in any command: override), file globs exclude rotated generations where the app numbers before the extension, and scripts ship only the current run’s bytes	review	—	/code-review of the Alloy config / compose command: / alloy_extra_log_paths diff + /wrapup conformance checklist replay-safety item; candidate machine gate: assert every command: overriding Alloy’s CMD repeats --storage.path, and flag a bare *.log glob on a known number-before-extension rotator	n/a
Every retention_stream entry uses a named tier period value (168h / 2160h; 720h reserved for the floor) — no ad-hoc window	review	—	/code-review (a non-tier period: in a loki.yaml diff is visible); candidate for a check_loki_retention.py deploy gate parsing retention_stream	—
Every non-default retention_stream entry’s comment cites a governing ADR	review	—	/code-review on the loki.yaml diff + /wrapup; candidate machine gate: assert each retention_stream block has an ADR-\d{4} comment	—
The tier→stream registry reconciles 1:1 with loki.yaml’s retention_stream list (no override without a row; no row without an override)	review	—	/code-review cross-reads both files; candidate machine gate resolving the doc table against loki.yaml (config-reference-validation shape, ADR-0142)	—
A stream longer than default-30d has a written forensic question + volume assessment	review	—	ADR review + /code-review (the ADR is the artifact; no script judges forensic merit)	—
Total Loki footprint stays within the ~15 GiB budget on arch01-obs	machine	runtime	ObservabilityStorageLow Prometheus alert (homelab.rules.yml) — warning → Pushover at <20% share free; share-level, not Loki-specific (noted gap)	the alert is the runtime detector; freshness rides Prometheus/Alertmanager liveness (InstanceDown)
A new tier (beyond the three) is a deliberate tiers-table edit plus its own ADR	review	—	/code-review + ADR review (a new tier is a standard change, not a config change)	—
Retention edits are batched (any loki.yaml change restarts the observability stack)	advisory	—	justification: an efficiency/blast-radius preference, not a correctness gate — the deploy.yml restart handler is correct behaviour, not a failure	—