AI delivery · 9 July 2026 · 9 min read
From AI pilot to production: seven evidence gates.
A pilot proves that a team can make something interesting happen. Production requires a harder case: the workflow is useful, testable, owned, recoverable, and worth operating.
A gate may result in advance, redesign, or stop. Stopping a weak idea before rollout is a useful outcome.
Why pilots stall
The unanswered questions live outside the model.
Prototype work naturally focuses on the striking moment: a document becomes structured data, a question receives a useful answer, or a draft appears in seconds. The difficult production questions concern everything around that moment.
Responsibility
Who owns the process, handles exceptions, approves changes, and can stop the workflow?
Evidence
Which cases were tested, what counts as acceptable, and where does performance break down?
Operations
How are permissions, queues, costs, logs, failures, corrections, and changing inputs managed?
The gate model
Seven questions before production.
Each gate needs an owner, an artefact, and a decision rule agreed before results are known. That prevents enthusiasm from quietly redefining success after the test.
Problem and ownership gate
Name one bounded workflow, its start and finish, the accountable business owner, the affected users, and the decision the work supports.
Pass evidence: a current-state map and an owner who can make scope and rollout decisions.
Baseline and value gate
Measure current handling time, waiting, rework, quality, backlog, or another business metric on representative cases. Estimate burden without presenting hypothetical savings as fact.
Pass evidence: a documented baseline, target, sample, assumptions, and measurement method.
Data and access gate
Identify authoritative sources, permission boundaries, missing fields, sensitive content, retention needs, and how representative test data will be obtained.
Pass evidence: an approved test path with realistic data and known limitations.
Working-slice gate
Build the smallest complete loop: intake, processing, evidence, human review, action, and recorded outcome. A disconnected prompt demo does not exercise the workflow.
Pass evidence: representative cases travel end to end in a controlled environment.
Evaluation gate
Create normal, edge, adversarial, incomplete, and expected-failure cases. Define quality by the job: required fields, grounded evidence, correct routing, acceptable edits, or another observable standard.
Pass evidence: reproducible results against a pre-agreed threshold, with failures retained.
Human-control gate
Decide what a person must approve, what information they need, which actions they can take, how quickly they must respond, and where escalation goes.
Pass evidence: named roles, usable review states, and a measured checking burden.
Operations and economics gate
Define monitoring, logs, alerts, support ownership, rollback, model or rule changes, expected volume, latency, operating cost, and the next measurement window.
Pass evidence: an operating plan and a costed rollout that the business and technical owners accept.
The decision
Advance when the evidence clears the gates. Redesign when a bounded change can close a known gap. Stop when value, data, control, or recoverability remains too weak.
A stop decision should document what was learned so the same unsupported idea does not return under a new label.
The shared artefact
Keep a proof ledger, not a victory deck.
The ledger is a compact record that business, user, technical, and risk owners can challenge. It connects every claim to evidence and every piece of evidence to a decision rule.
Claim
“The workflow reduces routine report assembly while preserving human approval of commentary.” Make it specific and bounded.
Evidence
Baseline sample, test set, observed results, reviewer edits, exception log, cost, and limitations. Keep versions and dates.
Decision rule
The threshold for advancing, redesigning, or stopping—agreed before the test and owned by a named role.
- Scope: what the workflow does, and what remains explicitly outside it.
- Baseline: source, sample period, assumptions, and current performance.
- Build: data path, components, permissions, and working-slice boundary.
- Evaluation: cases, metrics, thresholds, observed results, and known gaps.
- Control: approval, correction, escalation, shutdown, and ownership.
- Status: controlled test, limited release, broader rollout, paused, or stopped.
Release in layers
Production is a sequence, not a switch.
A sensible release can begin in shadow mode, where the workflow produces outputs without acting. Next, reviewers may use it for a narrow case class. Volume and permissions expand only when evidence remains acceptable. Every stage needs a rollback path and an explicit owner.
Shadow
Compare outputs with current work; take no automated action.
Assisted
Users review every output and record corrections and exceptions.
Bounded
Allow narrow low-impact actions; route uncertainty to a person.
Expanded
Increase scope only after fresh evidence and owner approval.
Common gate failures
Watch for evidence that only looks convincing.
A polished interface can hide a weak workflow. Average quality can hide a dangerous exception class. A fast draft can create more checking than it removes. A low model price can distract from integration, review, and support effort.
Weak proof
- A handful of hand-picked examples
- Success described only as user enthusiasm
- No record of rejected or corrected outputs
- No named owner after the project team leaves
Stronger proof
- A representative and versioned evaluation set
- Business, quality, and review-effort measures
- Visible failure classes and tested recovery
- A dated decision signed off by responsible roles
Build the evidence before the estate
Start with one measurable workflow.
Estimate the current burden, audit the proof foundations, and test the riskiest assumptions in a controlled ten-working-day sprint before deciding on production rollout.