Design the knowledge before the AI

Lesson 04.1

Retrieval is not authority

In a RAG system, the model may receive selected passages from a search system. Those passages can be chosen for semantic relevance, metadata, access permissions, ranking or reranking. A passage being retrieved does not make it current, approved or correct for the user's situation.

A highly relevant old draft can compete with a newer approved policy if the system lacks explicit authority and freshness signals. The model may also combine individually real passages that were never intended to be used together.

A reliable design treats coverage, retrieval and answer generation as distinct layers. Do not solve all three by simply uploading more files.

Lesson 04.2

The memory soup problem

Memory soup is what happens when a knowledge space accumulates documents faster than it accumulates structure and accountability.

The dangerous output is not always an obvious fabrication. It can be a plausible conclusion assembled from real fragments used in the wrong context.

The repository gets bigger while the answer quality gets worse. More information is not the same as better governed, more retrievable information.

• Contradictory documents coexist with no known winner.
• Stale guidance is retrieved with the same apparent confidence as current guidance.
• Draft notes and approved sources are indistinguishable.
• Nobody can describe what the space knows without opening every file.
• The AI stitches unrelated fragments into a polished but unsupported answer.
• New documents arrive, yet obsolete material is rarely retired.

Lesson 04.3

Give every knowledge space a contract

Useful metadata may include document ID, status, owner, version, effective date, review date, supersedes and topic. Metadata only improves outcomes when ingestion, retrieval or policy enforcement actually uses it.

Make authoritative content easy to identify and non-authoritative content hard to accidentally use. If the AI cannot decide which source wins, the right result may be an escalation rather than an answer.

• Purpose and scope — what questions belongs here?
• Exclusions — what does not belong?
• Authority — which source wins when documents disagree?
• Lifecycle — which items are draft, approved, superseded or archived?
• Ownership — who validates this information?
• Freshness — when is it reviewed, and what events trigger an earlier review?
• Supersession — how is the old version withdrawn or linked?
• Failure behaviour — what should retrieval and generation do with conflicts or missing evidence?

Lesson 04.4

The staleness–trust spiral

Unmanaged content creates noisy retrieval. Noisy retrieval creates inconsistent answers. Inconsistent answers reduce trust. People use the system less and report fewer problems. Without those signals, defects persist and knowledge gets even staler.

A decline in adoption can therefore indicate a reliability problem, not simply resistance to AI. The failure can continue invisibly because the system still runs and still produces fluent responses.