Production AI Failure-Mode Register
Use this before launching an AI workflow or agent.
Register
| Failure mode | Example | Control | Test | Metric | Owner | Stop condition |
|---|---|---|---|---|---|---|
| Plausible false output | Unsupported factual claim | Sources/citations + review | Known false-answer case | Unsupported-claim rate | ||
| Stale context | Old policy returned | Freshness metadata | Old/new policy query | Stale-source answer rate | ||
| Prompt injection | Retrieved content gives instructions | Trust boundary + tool limits | Injection fixture | Injection success rate | ||
| Unsafe tool use | Wrong CRM update | Argument validation + approval | Wrong-contact case | Wrong-action rate | ||
| Schema drift | Invalid JSON | Output validation | Malformed output case | Validation failure rate | ||
| Weak fallback | Silent failure | Human queue + user message | Tool outage case | Fallback completion rate |
Minimum Controls
- Prompt/template versioning.
- Source IDs or citations.
- Output validation.
- Tool argument validation.
- Human approval for high-impact writes.
- Fallback path.
- Logs with redaction.
- Kill switch.
Launch Decision
- Highest-risk failure mode:
- Blocking tests:
- Owner:
- First review date:
- Rollback path: