AI Build vs Buy Scorecard
Define the evidence, meaning, and weight for each score before evaluating options. The directions below are hypotheses to test, not a formula that determines “buy” or “build.” Security, privacy, legal, safety, accessibility, data-residency, and operational vetoes override any total.
Mandatory gates before scoring
| Requirement | Candidate evidence | Pass/fail/unresolved | Remediation owner |
|---|---|---|---|
| Data, privacy, legal and safety boundary | |||
| Core task quality and accessibility | |||
| Permissions, duplicate/unknown-result recovery | |||
| Records, export and deletion | |||
| Funded operational ownership |
For candidates that pass, compare workflow fit, user effort, integration/change burden, complete cost and strategic value with predeclared meanings and weights. Data sensitivity is not a score pointing automatically toward custom development.
Candidate interpretation to validate
- Buy/configure when a representative trial shows the product satisfies the workflow, controls, integration, evidence, and exit requirements at acceptable full cost.
- Extend when a vendor covers the commodity core and a thin, owned layer can safely supply the differentiating workflow without creating an unsupported control plane.
- Build when verified requirements cannot be met by candidates and the organisation can fund and operate the complete lifecycle.
- Keep the existing manual/no-AI process when it performs better on value, risk, or operability.
Full-Cost Check
- License/API cost:
- Implementation cost:
- Maintenance owner:
- Security review:
- Integration cost:
- Monitoring/evals:
- Exit cost:
- Human review/rework and incident cost:
- Dedicated capacity or metered inference cost (avoid double counting unless both are actually used):
- Migration, decommissioning, and data-export verification:
Representative-slice trial
- Same task set and acceptance criteria for every candidate:
- Failure recovery and human effort:
- Permission, audit, export, and deletion behavior:
- Latency and cost under representative load:
- Evidence attached to recalculated scores:
Decision
- Buy:
- Configure:
- Extend:
- Build:
- Revisit date:
Illustrative hold decisions
Vendor A lacks required source/version export; extension B duplicates a ticket on retry; build C lacks an operations owner. Each remains on hold for a different mandatory gap. Keep the manual baseline while resolving and retesting those gaps. No weighted total cancels a veto.