AI workflow design · 9 July 2026 · 9 min read
Human in the loop is a workflow.
“A human checks it” is not a control design. A real review step has a trigger, an accountable role, enough evidence, permitted actions, a response time, an escalation path, and a recorded outcome.
Human review can reduce risk, but it also consumes time and can fail. Design and measure both sides.
The checkbox problem
An approve button does not create meaningful oversight.
A reviewer may receive too little context, too many cases, or no authority to correct the underlying issue. They may not know what the AI did, which sources it used, or what happens after rejection. Under time pressure, “review” can become routine confirmation.
Rubber-stamp risk
The interface makes approval easy but careful comparison slow, so attention declines as the queue grows.
Hidden labour
A draft appears quickly, but checking sources, repairing omissions, and restoring context takes longer than the old process.
Orphaned exceptions
Rejected cases have no destination, owner, or deadline, leaving the hardest work stuck outside the happy path.
The operational blueprint
Design eight connected parts.
The details vary by risk and workflow, but each part should be explicit before expanding beyond a controlled test.
Trigger
Define which cases require review: every output, a specific action, missing evidence, rule violations, unusual inputs, sampled cases, or an explicit abstention. Avoid treating an uncalibrated confidence score as the only trigger.
Accountable role
Name the role, not “the team.” Specify required expertise, backup coverage, authority, queue ownership, and who can change or pause the workflow.
Review packet
Bring the original input, proposed output, relevant source passages, applied rules, warnings, prior decisions, and material uncertainty into one review surface. Do not force the reviewer to reconstruct the case across tabs.
Decision actions
Support the real choices: approve, edit, reject, request information, reroute, abstain, or escalate. Record a useful reason when it helps quality and operations.
Service level
State when review is due, how cases are prioritised, what happens outside staffed hours, and when a growing queue triggers a pause or fallback.
Escalation and recovery
Route policy questions, sensitive cases, system failures, and repeated error classes to the right owner. Define how an action is stopped, corrected, or reversed.
Audit record
Retain the versions needed to explain the event: input, output, evidence, reviewer, timestamps, edits, final decision, action, and system version—within the organisation’s retention rules.
Improvement loop
Aggregate corrections and rejection reasons into failure classes. Use them to improve instructions, rules, sources, interfaces, tests, and scope. A correction is operational evidence, not automatically training data.
Place the human deliberately
Review timing should match the consequence.
Not every workflow needs the same boundary. Choose it by considering reversibility, impact, evidence quality, error detectability, and the reviewer’s real capacity.
Before action
A person approves the proposed output before anything is sent, published, committed, or used for a consequential decision. Appropriate when effects are material or hard to reverse.
During the work
The system pauses on ambiguity, missing evidence, policy boundaries, or tool choices. A person supplies a decision, then the controlled workflow continues.
After action
Sampling or exception review may suit low-impact, reversible actions with reliable detection. It is weak protection when harm occurs before anyone can respond.
A practical rule
Move review later only with evidence.
Begin with shadow outputs or review before action. Expand a clearly bounded class only after representative tests and live review data show that quality, exceptions, recovery, and workload remain acceptable. Keep the ability to return to the safer state.
Measure the human system
A fast model can still create a slow workflow.
Measure end-to-end performance, not generation speed alone. Review time belongs in the business case, and queue health belongs in operating monitoring.
- Review rate: the share of cases sent to a person, by reason and case class.
- Touch time: active minutes needed to inspect, compare, edit, decide, and document.
- Queue time: elapsed time before a reviewer begins, including peaks and off-hours.
- Edit distance: how much the proposed output changes before acceptance, using a measure suited to the output.
- Rejection and escalation: frequency, reason, destination, and resolution time.
- False comfort: sampled approvals later found to contain a material issue.
- Abstention quality: whether the workflow sends the right uncertain cases to people.
- Outcome measure: cycle time, quality, backlog, rework, cost, or another pre-agreed business metric.
A 45-minute design exercise
Prototype the review before the model.
Take three normal cases, two awkward cases, and one known failure. For each, sketch the exact packet a reviewer receives and the actions available. Then walk through what happens after every action.
Ask the reviewer
- Can you tell which source supports each important claim?
- What would make you reject or escalate this case?
- Which context is missing from this screen?
- How many of these cases could you review during a busy hour?
Ask the operator
- Where does a rejected case go next?
- Who responds when the queue breaches its limit?
- Can you reconstruct and correct a past action?
- Which repeated failure would make you pause the workflow?
What good looks like
The reviewer improves control without becoming invisible infrastructure.
A well-designed loop makes uncertainty obvious, concentrates attention on meaningful decisions, and records outcomes that help the workflow improve. It respects capacity: when demand exceeds what reviewers can safely handle, the system slows, narrows, or falls back rather than silently lowering the standard.
Healthy review
Clear roles, compact evidence, meaningful choices, routable exceptions, known capacity, measured effort, and a tested pause path.
Review theatre
A generic warning, one approve button, no source context, no queue owner, no escalation, and no record of what reviewers changed.
Prove the whole loop
Test the review workload before rollout.
Estimate the current workflow burden, inspect ownership and evidence foundations, then build one controlled end-to-end slice that includes the human decision—not a placeholder for it.