Markdown

Production AI Failure-Mode Register

Use this before launching an AI workflow or agent.

Register

Failure modeExampleControlTestMetricOwnerStop condition
Plausible false outputUnsupported factual claimSource-support validation + reviewKnown false-answer caseUnsupported-claim rate
Stale contextOld policy returnedFreshness metadataOld/new policy queryStale-source answer rate
Prompt injectionRetrieved content gives instructionsTrust boundary + tool limitsInjection fixtureInjection success rate
Unsafe tool useWrong CRM updateArgument validation + approvalWrong-contact caseWrong-action rate
Schema driftInvalid JSONOutput validationMalformed output caseValidation failure rate
Weak fallbackSilent failureHuman queue + user messageTool outage caseFallback completion rate

Minimum Controls

Launch Decision

Integrated unknown-write fixture

Simulate booking B17 succeeding downstream while its response is lost. Expected: preserve the same operation ID, stop new attempts until reconciliation, do not claim confirmed success without evidence, and pass the unknown state to the authorised reviewer. Recovery must not replay a completed effect. Record downstream count, user message, queue state, owner and recovery evidence. A kill switch stops new work; it does not undo an in-flight external effect.