Part of my job is reading conversations where an AI support bot got it completely wrong: hallucinated policy, confident nonsense, perfect grammar.
The failures almost never come from the AI being stupid. They come from the system being incomplete. Once you know what to look for, the failure type announces itself.
Three failures, three fixes
A retrieval failure means the right article existed but was not found. A content failure means the right article was retrieved but the answer was wrong. A coverage gap means no article exists for the intent. These are different problems with different owners.
I use root-cause codes such as CONTENT_WRONG, CONTENT_MISSING, KEYWORD_GAP, RETRIEVAL_FAIL, HALLUCINATION, AUTOMATION_MISFIRE, and KNOWN_BUG. A model should flag non-fits rather than invent a category.
There are two kinds of hallucination too. Retrieval hallucination misrepresents retrieved content. Generation hallucination invents information without support. Conflating them sends the fix to the wrong place.
Reconstruct the chain: query generated, articles retrieved, customer data pulled, response generated. Then write an escalation brief with a plain problem statement, log evidence, reproduction pattern, frequency, user impact, business risk, hypothesis, requested action, and priority.
The skill makes me faster because it handles reconstruction, elimination, and the first draft. I spend my time on the judgment calls that require a human.
This is part of the Parenting Your AI series, a practitioner's guide to building AI skills that are safe, effective, and worth trusting. Written from inside enterprise AI systems by someone who has spent years diagnosing what goes wrong when AI meets real work at scale.
Read the full series at KnowledgeManagement.ie