The playbook

Six stages, each with a way out

A stage is not a phase gate for its own sake — it is a commitment to stop spending if the evidence does not arrive. Use the exit criteria as written; they are deliberately uncomfortable.

Frame

1–2 weeks

Turn an ambition into a falsifiable problem statement.

Core activities

Write the problem statement as a job-to-be-done, not a technology wish
Quantify today's baseline: volume, cycle time, error rate, cost
Name the accountable business owner and the decision they will make
Decide the kill criteria before anyone writes code

Artefacts: Opportunity canvasBaseline metric sheetKill criteria memo
Exit criteria: A single sentence stating what changes, for whom, by how much, by when.
The trap: Starting with a model choice. The model is the last decision, not the first.

Prove

2–4 weeks

Establish technical feasibility against a real evaluation set.

Core activities

Build a golden evaluation set with domain experts before prototyping
Benchmark the AI approach against a boring deterministic baseline
Probe the hardest 10% of cases first, not the demo-friendly ones
Cost-model inference at realistic production volume

Artefacts: Golden eval setBaseline bake-off reportUnit economics model
Exit criteria: Measured quality clears the threshold the business owner signed up to.
The trap: A dazzling demo built on five cherry-picked examples.

Pilot

4–8 weeks

Put it in front of real users under real constraints, with containment.

Core activities

Run shadow mode first where the consequence of error is material
Instrument task completion, escalation and override rates
Hold weekly failure-review sessions with frontline users
Red-team for prompt injection and data exfiltration

Artefacts: Pilot charterFailure mode logRed-team report
Exit criteria: Users choose to keep using it when nobody is watching.
The trap: Measuring usage instead of outcome. Clicks are not value.

Harden

3–6 weeks

Make it boring: observable, reversible and supportable.

Core activities

Automate evals in CI with release-blocking thresholds
Define latency, cost and quality SLOs with alerting
Rehearse rollback and provider-outage scenarios
Complete the responsible AI impact assessment and sign-off

Artefacts: Eval pipelineSLO & error budgetRunbooksImpact assessment
Exit criteria: You can change the model on a Friday without fear.
The trap: Treating governance as paperwork at the end instead of design input.

Scale

Ongoing

Grow adoption deliberately, protecting quality and cost.

Core activities

Expand by cohort with explicit entry criteria per wave
Resource enablement, comms and support ahead of each wave
Track cost per successful task as a first-class metric
Review drift and eval results on a fixed monthly cadence

Artefacts: Rollout wave planEnablement kitAdoption dashboard
Exit criteria: Adoption curve and quality curve both hold as volume rises.
The trap: Scaling access faster than you scale support and governance.

Sustain

Ongoing

Run it as a product, and retire it honestly when value ends.

Core activities

Re-baseline value realisation against the original business case
Review model/provider roadmap and deprecation risk quarterly
Refresh the golden eval set as the domain changes
Decide openly whether to invest, hold or retire

Artefacts: Benefits realisation reviewModel lifecycle registerRetirement plan
Exit criteria: Value is evidenced, or the capability is retired without drama.
The trap: Zombie AI: still running, nobody using it, still costing money.

How to run it

Gate on evidence, not dates

A stage ends when its exit criterion is met. If the date arrives first, the honest options are extend, re-scope or stop — never "proceed and catch up".

One accountable owner per decision

Use the RACI artefact. Shared accountability for AI quality thresholds is how programmes end up with none.

Compress, don't skip

A low-consequence internal tool can run Frame to Pilot in three weeks. It still needs a baseline, a golden set and kill criteria.

Get the stage artefacts → Score readiness for the next gate