Lesson 06.1
A good demo is not proof of reliability
AI output varies with inputs, context and system behaviour. Start with 12–20 representative cases and expand the set with actual failure patterns. Some cases should have an exact expected answer; others should grade behaviour such as refusing, escalating or citing a conflict.
- Normal requests the skill should handle well.
- Ambiguous or incomplete records.
- Contradictory, stale or missing sources.
- Out-of-scope and permission-boundary requests.
- High-impact edge cases and adversarial instructions.
- Previously failed cases that must not regress.
Lesson 06.2
Score what actually matters
An automated judge can scale evaluation, but should be compared with human decisions and reviewed for disagreement. Watch failure categories and severe regressions, not only one average score.
- Factual support — are material claims backed by approved evidence?
- Coverage — does the answer include essential information and conditions?
- Correct application — did it use evidence for the right situation?
- Boundary handling — does it recognise uncertainty and stop appropriately?
- Traceability — can a reviewer follow findings to the source?
- Action safety — were permission and approval rules respected?
- Usability — can the intended user act on the result?
Lesson 06.3
Diagnose the right layer
Do not respond to every failure by adding another sentence to the prompt. Find the broken layer, make one targeted change and rerun the baseline cases.
- Missing source → knowledge coverage problem.
- Wrong source retrieved → ranking, metadata or retrieval problem.
- Correct source retrieved but misused → generation or instruction problem.
- Tool did the wrong thing → workflow or integration problem.
- Correct answer, inappropriate action → permission or decision-boundary problem.
The idea to keep
Test the system that produces the answer—not merely the wording of the answer.
Try this
Create five test cases today, including one where “insufficient evidence” is the only correct result.