Day 20: Why every important AI claim needs provenance
Why important AI claims need receipts that users and teams can inspect.
An answer without sources is a rumour with good grammar.
When your system tells a customer "you are eligible for a refund," somebody will eventually ask: says who? If the honest answer is "the model felt strongly about it," you do not have a product yet. You have a liability with a chat interface.
Citations are closer to logging than to formatting
Teams file provenance under polish, something to add once the core experience works. It behaves much more like observability: the thing you desperately want during an incident and cannot retrofit after one.
Because when an AI answer causes a problem, the questions are always the same three. What exactly did the system claim? What evidence did it use? Was that evidence wrong, stale, out of scope, or simply ignored?
With provenance, that is a five-minute investigation. Without it, it is archaeology: re-running queries against an index that has since changed, trying to reconstruct what retrieval returned last Tuesday.
What a real receipt carries
Printing a document title under the answer is not provenance. This is closer to the minimum:
{
"claim": "Refunds are allowed within 30 days for unopened items.",
"source_id": "policy-refunds-v4",
"source_version": "2026-04-02",
"source_owner": "commerce_policy",
"retrieved_at": "2026-06-26T10:30:00Z",
"quote_span": { "start": 418, "end": 512 },
"permission_checked": true
}
The details that make it useful rather than decorative:
- A span, not a document. "It is somewhere in this 40-page PDF" fails the under-a-minute verification test, which means in practice nobody verifies.
- A version. Policies change. A citation that resolves to "the current version" cannot explain an answer given three months ago under different rules.
- A timestamp. Distinguishes "the system read the wrong thing" from "the system read the right thing, which has since been updated." Completely different fixes.
The metric nobody tracks
Measure citation coverage: what share of material claims in an answer are backed by a specific source span?
It is quietly diagnostic. Uncited claims cluster precisely where the model is improvising, filling gaps between retrieved facts with plausible connective tissue. Coverage dropping on a class of questions is an early hallucination signal, visible before any user complains.
Provenance turns debugging into an audit. Wrong answer? Follow the receipts. Bad retrieval, stale document, or a model that ignored good evidence, each points at a different component and a different owner.
Trust is not a tone of voice. It is a paper trail.
Day 20 of 60 Days of Production AI Systems.
Would your system survive one hard "says who?" from your most sceptical stakeholder?
Previous: Day 19: Why reranking is where retrieval becomes useful
Next: Day 21: Why RAG needs both broad context and precise evidence
Day 20 of 60 · Evidence-Driven RAG Patterns (chapter 4 of 10)
