The Evidence Chronicle

What the progression shows, by source class

EMPIRICAL is undefeated. No observation of the machine was ever overturned in this record — not the in-session hook failures, not the live filesystem reads, not the headless probes, not the scratch-DB reproductions. Everything else was overturned at least once.

BILL is the continuity layer, and that — not omniscience — is why the challenges landed. The user was navigating an unfamiliar tool, not adjudicating it; but as the only participant present across every session, he was also the only place cross-session contradictions could surface. The domain-bleed catch created the contexts layer (Exhibit F); the attribution pushback beat model inference (K); the settings.local.json challenge exposed a model self-contradiction (D); "you'll excuse me if I'm skeptical" overturned two fact-check verdicts (C, D); and even the shorthand of Exhibit H was a correct reading of the observable symptom. In a system where every other author is amnesiac, the human's accumulated friction is the institutional memory.

ANTHROPIC docs are necessary but describe a subset of the product. They correctly killed @import and the flat hook schema — and were contradicted by the machine on ~/.mcp.json and user-level settings.local.json. Observed behavior is a superset of documentation.

REPO artifacts are consensus snapshots, not truth — and they're Claude's writing too. The config doc and clib's code (both model-authored, Bill-curated) faithfully preserved the good empirical lessons (flattening, env sync) alongside the untested model assertions (@import) — and added drift of their own (flat hooks, skill @imports) where a later Claude session, writing without the earlier session's context, contradicted the earlier Claude's correct work.

MODEL and REVIEW assertions are hypotheses until tested. April's model was right about ~/.mcp.json and wrong about ${VAR} expansion; July's reviewers were right about hooks and wrong about ~/.mcp.json. Same class, same failure mode, opposite directions — which is why the class, not the date, is what the chips track.

The uncomfortable census: of the six classes, four — MODEL, REVIEW, REPO, and even the config doc this audit began by fact-checking — are Claude's own output at different moments, checking and contradicting itself across sessions that share no memory. The only independent witnesses in the entire record are BILL, ANTHROPIC, and the machine itself. That is the structural argument for the evidence hierarchy: when four of your six sources are the same fallible author wearing different timestamps, you privilege the two that aren't — and the machine outranks everyone.

Cautions for teams adopting Anthropic tooling

What this record generalizes to. Each caution cites the exhibits that earned it.

1. Treat model-asserted configuration as a hypothesis, and test it the moment it's written. The most expensive errors here were schemas Claude stated confidently and nobody ran: an import keyword that was never real (A), a hook shape no version ever accepted (B). Both were cheap to verify at generation time — a one-line probe, a read of the file another tool actually consumes. The cost of not verifying was months of a silently dead feature. (Exhibits A, B, E2)

2. Don't trust the docs as a ceiling either — the product's observed behavior is a superset of its documentation. Two "the docs say this doesn't exist" verdicts died on contact with the live machine (C, D). Before ripping out something that works, or building on something documented, check what the machine actually does: read the real config files, run a headless probe, watch what loads. (Exhibits C, D, G)

3. Session amnesia is the root hazard of model-driven development. Every session starts with no memory of the last one. This repo's git log is the fossil record: 24 commits, each a discrete Claude session, with diffs that flip prior decisions and messages that chart the frustration. A later session contradicted an earlier session's correct design (F) and its correct hook examples (B) because nothing carried the context forward. Mitigations: persist decisions in repo docs the next session will read, review diffs specifically for silent reversals, and keep one human continuously in the loop — their accumulated friction is the only institutional memory the system has. (Exhibits B, F; the git log)

4. Expect failures to be silent, and build fail-loud checks into anything that writes config another system consumes. Claude Code ignores invalid config rather than rejecting it: bad hooks are skipped, inert import lines render as text, malformed stores provision zero assets without a word. Every breakage in this record announced nothing, which is why each one read as "the product broke." If your tool writes settings, hooks, or assets for another system, ship a doctor command that asserts the contract end-to-end. (Exhibits A, B, H, I)

5. Prefer the platform's native primitives over re-implementing them. The single largest blast radius here (F) came from re-implementing loading that the platform had made native — auto-discovery and progressive disclosure. The founding design got this right; a later implementation session drifted into fighting the platform. When Claude Code ships a primitive, migrate to it; re-probe your assumptions after product updates with quick empirical checks rather than assuming either breakage or stability. (Exhibits F, G)

6. Read user reports as symptom descriptions, not literal mechanism claims. "It dropped my agents" meant "the deployed catalog no longer matches the contract, so it's absent from context" — accurate observation, compressed. Triage the observable symptom (what's missing from context, what file diverged from what schema) before litigating the attribution. (Exhibit H)