AI Automation Validation Checklist

Use this before turning on a new AI automation.

Trigger

• The trigger is narrow and intentional.
• Self-generated events are excluded.
• Test events are clearly separated from production events.
• Duplicate events have an idempotency key or duplicate check.

Input

• Required fields are validated before the AI step.
• Sensitive fields are removed unless they are necessary.
• Maximum input size is enforced.
• The source system and timestamp are recorded.

AI Step

• The prompt includes the task, allowed assumptions, output format, and uncertainty handling.
• The output is validated before downstream actions run.
• The workflow has a fallback path for malformed or empty output.
• The first production runs are manually spot-checked.

Actions

• Low-risk actions can run automatically.
• External, destructive, financial, legal, HR, or customer-visible actions require approval.
• The workflow logs what happened, who/what triggered it, and what action was taken.
• There is an owner who receives failure notifications.

Go/No-Go

• Run at least 20 representative test cases.
• Confirm at least one failure case reaches the fallback path.
• Confirm a duplicate event does not duplicate the business action.
• Confirm rollback is simply “turn off the automation” or disable the trigger.