Design: Agent lesson corpus — shared, verified operational memory

The agent’s hard-won operational lessons — the traps, the silent-failure modes, the “this looks like success but isn’t” gotchas — are today scattered across eight machine-local memory stores with zero overlap, so any one session can read about 14% of what the estate has actually learned. This design moves those lessons into the version-controlled docs corpus where every session and every future agent can retrieve them, pushes a small set of generalised failure classes into every session so the warning arrives before the mistake, and repairs the nine directives in soul.md/CLAUDE.md that currently cite evidence which does not resolve. It amends ADR-0018, whose reference/feedback split was correct but assumed native memory was a single store.

The problem — 297 lessons, 8 stores, zero overlap

Claude Code keys its memory store by project path. The cc-pool launches each session with its working directory at ~/sessions/cc-N, a plain folder that contains worktrees rather than being a repo itself, so the harness falls back to path-keying and each session slot gets a private store.

Measured 2026-08-12:

Store	Lessons
/home/pit	86
sessions/cc-1	60
sessions/cc-2	45
sessions/cc-3	42
sessions/cc-4	42
sessions/cc-5	18
/home/pit/.claude	5
/home/pit/ansible	3
Total	297 files, 297 distinct names — no duplication

Superseded by the delivery measurement — this table is wrong

Re-measured at migration time (Issue #2384, same day): 299 files across NINE stores, 277 of them lessons. This table counted files including each store’s MEMORY.md index, omitted the store keyed -, and its own rows sum to 301 rather than the 297 stated. The other 22 files are project/reference/user notes, which are not lessons and stayed native. Zero duplicate basenames was confirmed correct. Treat the delivered figures as authoritative: 277 lessons migrated, 252 pages after consolidation.

Zero duplication is the diagnostic finding: nothing is redundantly stored, so every lesson exists in exactly one silo. A trap learned expensively in one session is available to that session alone, and the same mistake can be paid for up to five times. Nine feedback-* citations in the operating doctrine resolve to nothing from a given session; two of them (reference-pit-memory-search, feedback-pve-needs-sudo) exist in no store at all — cited but never written.

Roughly a quarter of the corpus describes one recurring failure class: an empty, zero, or quiet result treated as a fact.

Decision tree — the resolved architecture

Decisions

Decision	Choice made	Rationale	Rejected alternatives
Store of record	Corpus holds bodies; a class index is pushed to every session	Only option where a lesson learned in one session reaches the others and survives a control01 rebuild	Consolidate native stores only (stays machine-local, un-versioned, fights upstream path-keying churn); corpus-only (converts every gotcha from a push warning to a pull lookup); do nothing
Push payload	~12 named failure classes, not 297 pointers	Generalises to traps not yet made; keeps always-loaded context ~1–2k tokens instead of 15–20k growing unbounded	All 297 pointers (bloats the always-loaded file, the documented way an agent stops prioritising it); recency window (arbitrary — high-severity lessons age out); classes + domain slice (best precision, but needs scope inference that fails quietly toward under-loading)
Class file	Hand-authored doctrine in claude-config	The class list changes ~monthly while lessons arrive ~6/week; generating it would add a scheduled job, a dead-man, and a bot with commit rights to the agent’s own operating instructions	Nightly generation; generate-and-commit-by-hand (no trigger — run twice then never); fold into soul.md (mixes evolving list with stable doctrine)
Capture path	Re-route memory_write(kind=feedback) to the corpus	Friction is what kills knowledge systems; 297 lessons exist because capture is one call. memory_write already routes by kind, so this repoints an existing route	Docs-as-code commit per lesson (edit-commit-push-wait mid-task, five sessions racing one index — realistically ends capture); nightly harvest; promote at /wrapup only
Corpus home	docs/feedback/, indexed, nav-excluded, own .pages	Stable greppable home and a citable URL, without putting terse agent notes in front of a human reader	Reuse _drafts/ (conflates two lifecycles — a draft awaits promotion, a lesson never gets promoted); publish to the site (editorial cost kills capture); fold into the incident corpus (most are technique traps, not incidents — degrades incident search precision)
Durability	Indexed on write, committed at /wrapup	Sibling sessions retrieve within seconds; commit rides a gate that already runs and already commits docs. No service account with push rights to estate documentation	Commit per write (needs a container with push rights; five sessions racing commits); batch commit on a schedule (reintroduces the job + dead-man just deleted); shared un-versioned directory
Migration scope	All 297, tiered proof, evidence line on each	Preserves the whole asset. Full live re-verification is not literally achievable — several lessons were learned by breaking production	Verify-what’s-safe-and-drop-the-rest (systematically deletes the highest-severity lessons because they were expensive); lab reproduction (own Epic); re-trigger everything (deliberately re-breaks the estate)
Consolidation	Merge duplicates + retire-if-guarded, guard proven red first	The guard becomes the memory — toil elimination applied to knowledge. The proof requirement stops this silently deleting live knowledge	Merge-only (leaves notes for structurally impossible traps); rewrite to 12 class docs (specifics are what make a lesson actionable — wise and useless); retire anything not hit twice (most are one-shot by nature; “never recurred” usually means the note worked)
Doctrine citations	Cite the class, link the instance	The class is already in context, so the rule arrives with its reasoning attached and costs no retrieval	Plain URLs only (every citation is a pull that does not happen); keep wikilinks (private format, not clickable, not valid markdown); strip citations (a rule with no incident behind it reads as arbitrary and gets dropped)
Freshness	Verify-on-use, re-stamp the evidence date	Effort follows actual usage; re-verifying dormant lessons is patching a machine nobody uses	Scheduled sweep (toil over a mostly-dormant store); expiry by class (shelf lives are guesses; false staleness trains the warning to be ignored); nothing
Secret control	Existing gitleaks CI gate only	Arron’s call — one control, no second ruleset to drift	Shift-left scan at write time (rejected: see residual risk)
Golden path	Cross-context canary in pit-memory’s existing testplan.yaml	Only an assertion that crosses contexts can fail if fragmentation returns	Same-context write/read (passes today, before anything is built — satisfiable without the feature); separate test plan (needs its own scheduled run + dead-man); verify once at handover

NFR resolution

Requirement	How this design satisfies it
Monitoring (Pillar 1)	Inherited — pit-memory is an existing monitored service. No new host or container is introduced.
Observability (Pillar 2)	Inherited dashboard and Loki shipping. The one genuinely new signal is cross-session retrievability, made observable by the canary rather than by a metric nobody reads.
Alerting (Pillar 3)	Deliberately none added. The failure mode — silent capture failure — is caught in-band by asserting the written artifact, and again by the /wrapup reconcile. A capture-rate dead-man was rejected: capture rate is behavioural, so the threshold is either too loose to fire or too tight and gets muted.
SBOM / vulnerability (Pillar 4)	Inherited — pit-memory ships a Dockerfile, so its base image and requirements.txt are already Renovate-tracked and Trivy/Dependency-Track scanned. No new image.
Documentation (Pillar 5)	This design doc, the ADR amending ADR-0018, the docs/feedback/ tier definition, and the front-matter schema.
Software update & patch lifecycle	Pattern: the pit-memory service follows the built-image path — base image + requirements.txt pinned in-repo, bumped by Renovate, deployed by the docker-stacks-* pipeline with the pinned tag in Git as the rollback handle. Trigger/cadence: Renovate update PRs, continuous. Gate: the stack’s post-deploy testplan_gate.py run — now including the cross-context canary, so a bad bump fails on the feature and not merely on readiness. Staleness detection: Trivy scan + the Dependency-Track project already covering the image. The knowledge-domain analogue — a lesson going stale — is handled by verify-on-use rather than a scheduled sweep: a retrieved lesson past its age threshold is re-confirmed and re-stamped before being acted on.
Config as code (RULE 7)	Every artifact is in a git repo: lessons and the tier in pitlab-docs, the class file and doctrine in claude-config, the service change in docker-stacks. Nothing is manual-only.
Planned maintenance	Not applicable — no service outage is planned. A pit-memory redeploy is a rolling container restart on docker01 with no dependent alerting.
Notifications	None added. This design emits no alerts and sends no email.
Tagging (RULE 10)	Front-matter kind: feedback, failure class, evidence tier and evidence date on every lesson; docs tag vocabulary extended; the grilled tag on the Epic.
Enterprise pattern	This is knowledge management as code — the discipline a mature SRE org realises through a shared postmortem corpus and runbook library rather than tribal memory. The push/pull split mirrors a hot doctrine layer over a cold retrieval layer; verify-on-use is demand-driven revalidation rather than scheduled revalidation.
DORA / SRE delivery performance	Deployment frequency and lead time come from the docker-stacks pipeline data already collected for pit-memory. Change failure rate is the post-deploy gate’s red rate, now sharpened because the canary tests the feature rather than readiness. MTTR is not newly instrumented — accepted, since no user-facing outage class is introduced. Reliability contract is inherited from pit-memory’s existing SLO; this design defines no new error budget, which is a conscious gap recorded below.
Minimal decomposition	3 child Issues → ~3–4 prompts. Splits only where a hard dependency forces ordering.

Compliance & control mapping

Control / decision	Framework mapping	How tested / evidenced	Residual risk
Lessons held in an Access-gated corpus (docs.pitbun.com, Require EntraID policy)	CPS 234 §26 control implementation · NIST CSF Protect (PR.AA) · ISO 27001 A.5.15 access control	Verified against tf-cloudflare apps.tf; continuously re-tested by the external attack-surface scan	None material — the corpus is internal-only
gitleaks secret scan on the docs repo	CPS 234 control testing · NIST CSF Protect (PR.DS) · ISO 27001 A.8.12 data leakage prevention	Runs in the docs pipeline and in the pre-commit hook; proven on every push	Accepted (Arron, 2026-08-12): detection happens at CI, not at the keyboard. A leak reddens the shared docs pipeline, blocking concurrent sessions’ publish until fixed. Shift-left scanning at write time was considered and declined to avoid a second ruleset
Artifact asserted on write; /wrapup reconcile	CPS 234 control testing · NIST CSF Detect (DE.CM)	Fails loudly at capture; reconcile compares captured-vs-staged	A session ending abnormally loses that session’s uncommitted lessons — bounded to one session, and visible as uncommitted files
Version control + PBS coverage of the corpus	CPS 230 business continuity · NIST CSF Recover (RC.RP) · ISO 27001 A.8.13 backup	Git history plus the existing pitlab-docs backup path	Restore is proven-by-construction, not drilled — recorded as a known gap
Cross-context canary in the service test plan	CPS 234 §26 systematic control testing · NIST CSF Detect · ISO 27001 A.8.29 security testing	Runs on every deploy through testplan_gate.py; proven red before trusted green	None — this is the strongest control in the design
No new external dependency	CPS 230 material service provider risk	All components are self-hosted; no third party is introduced	None — no exit plan required
Design authority: grilled + approved before build	NIST CSF Govern (GV.OC) · ISO 27001 A.5.1 policy	This document, the grilled tag on Epic #2382, and the ADR-0372 carve-out that authorised it	None

Data classification. Lessons are internal operational knowledge — no personal data, no customer data, no residency constraint. They may legitimately reference internal hostnames and API shapes, which is acceptable inside an Access-gated corpus. Secret values are prohibited by authoring rule; a lesson references where a secret lives, never its value.

Operational readiness & handover

Service record. No new service. This extends pit-memory (existing docs page, existing NetBox record, existing criticality tier). Dependencies: pitlab-docs (source of truth), Qdrant and the SQLite FTS index (retrieval), docker01 (runtime). No new material service provider.

Reliability contract. Inherited from pit-memory. The new SLI worth naming is cross-session lesson retrievability — a lesson written under one project context is retrievable under another. It is asserted by the canary on every deploy rather than continuously measured; no separate error budget is defined. This is a conscious gap: the failure is not user-facing and degrades silently rather than dangerously.

Runbook. The failure modes a cold reader must be able to handle: capture refuses (assert the artifact, check the corpus path is writable); a lesson is not retrievable from a sibling session (check the reindex ran, check the file committed); the class file has drifted from the corpus (caught at /wrapup, re-author the class). These land in the pit-memory operations page rather than a new runbook.

Test strategy.

• Throughout delivery — the mechanism issue closes only when a lesson written under one project context is demonstrably retrieved under another, shown live. The migration issue closes only when every migrated lesson carries a class and an evidence line, and every retirement names a guard proven to fire. No sub-issue closes on “applied successfully”.
• End-to-end — the cross-context canary added to stacks/pit-memory/testplan.yaml. Golden path: write via memory_write under context A, assert retrievable by content under context B, assert the file exists on disk. Failure action: the stack has a revertable pinned-tag deploy, so auto-rollback.
• Triggers — inherited from the existing plan: post-deploy pipeline gate, the /vulnscan post-remediation close-gate, and on-demand /verify.
• Maintenance obligation — any future work touching pit-memory or the feedback tier runs this plan first as a regression baseline and updates it in the same change when behaviour changes.

Day-2 BAU and toil. Deliberately near-zero: no cron, no watchdog, no scheduled sweep, no bot commits. The only recurring obligations are verify-on-use (paid inside work already happening) and the /wrapup class-gap review (inside a gate that already runs). This handover does not increase Arron’s manual load.

Backup and restore. The corpus is git-backed and covered by the existing pitlab-docs backup path. Restore is a clone. Recorded honestly as proven-by-construction, not drilled.

Standing owner. pit-memory is already picked up by /health, /vulnscan (Dependency-Track project + Trivy target), Renovate (base image + requirements.txt), and the synthetic test-plan gate. The class taxonomy is owned by /wrapup, which reviews whether a session’s lessons revealed a class that does not yet exist. Nothing in this design is unowned.

Exit. If the corpus tier is ever retired, lessons remain readable markdown in git — there is no proprietary store to escape. The native-memory path stays available as a fallback.

Delivery plan

3 child Issues → ~3–4 prompts. Ordering is forced twice: migration needs the classes and the corpus tier to exist, and the doctrine rewrite needs the migrated URLs.

#	Issue	Contents
1	Mechanism (#2383)	Author the ~12 failure classes; create docs/feedback/ + .pages + front-matter schema; re-route memory_write(kind=feedback) with artifact assertion; /wrapup commit, reconcile and class-gap review; cross-context canary in testplan.yaml, proven red; ADR amending ADR-0018
2	Migration + doctrine (#2384)	Migrate all 297 at tiered proof with an evidence line each; merge duplicates; retire only against a proven guard; rewrite soul.md/CLAUDE.md citations to cite-class-link-instance; resolve the 9 dangling citations
3	Handover / PRR (#2385)	Canary green through its real trigger; capture path exercised end to end; silent-capture-failure proven to fail loudly; verify-on-use demonstrated; standing owners named; early-life-support window defined and the Epic parked in Hypercare

Known thin spots, carried into delivery: the reliability contract is inherited rather than defined; the restore path is proven-by-construction rather than drilled; and the twelve failure classes do not exist yet — authoring them is the judgement-heavy first task, and if the taxonomy is wrong the push payload is wrong with it.

Deviations (as-delivered)

Stamped 2026-08-12 at Epic close (#2382). Twelve of the fifteen design decisions were implemented exactly as agreed. Three changed during delivery, and two things were delivered that the design never scoped. Every one is recorded below with its reason and its governing record.

Decision as designed	What was built	Why	Record
Migrate 297 lessons from 8 stores	Migrated 277 lessons from 9 stores → 255 pages after consolidating 28 duplicates into 24	The planning-time inventory counted files including each store’s MEMORY.md, missed the store keyed -, and its own rows summed to 301. Re-measured at migration time. The decision is unaffected — fragmentation and zero overlap both hold	Correction notes in this doc and ADR-0392
The class field carries only failure-class slugs	Two non-trap kinds added — standing-directive (54) and environment-invariant (52)	106 of the migrated pages describe no self-deception at all. Forcing a trap slug onto 43% of the corpus would have made the field meaningless. Both are deliberately kept OUT of the always-loaded file	Lesson corpus tier definition
Push payload is failure classes only	Also carries 12 standing directives, one line + link each	Arron’s ruling at close-out. A directive that lives only in the pull layer cannot work — the agent will not search for a rule it does not know exists, which is ADR-0018’s own argument for a push layer, applied to directives rather than gotchas	ADR-0392 Consequences; claude-config efb0084
Reliability contract inherited, no SLI or error budget — recorded as a conscious gap	Two SLIs named with an error-budget policy, on the existing dashboard panel	The gap was acceptable at design time and cheap to close at handover. Closing it is what makes the hypercare exit criteria testable	pit-memory production readiness
(not scoped)	Retrieval-index chunk cap + catalog exclusion	Acceptance queries surfaced a single 636,832-character chunk degrading recall corpus-wide, not just for lessons. Fixed rather than carried, because every session depends on it	Corpus Retrieval Standard
(not scoped)	Manual-gate-run rule in the Service Test Plan Standard	Proving the canary red required running the gate as CI does, which paged and attempted a live revert of another session’s commit. The gap was in the standard, not the instance	ADR-0102 Consequences; standard #manual-gate-run

The three thin spots the design carried into delivery, resolved: the reliability contract is now defined rather than inherited (above); the restore path was drilled, not assumed — a fresh git clone --depth 1 yielded 255 lesson pages, content spot-checked; and the taxonomy landed at 14 classes rather than the estimated twelve, with the two non-trap kinds held in the pull layer.

Nothing was dropped. No design decision went unimplemented.