Build the deferral path before the AI happy path

A prototype usually demonstrates the happy path: a request arrives, the model interprets it and a useful response appears. Production introduces the missing cases. The request may lack evidence, combine several intentions, conflict with policy or require knowledge held only by a particular team. The model still produces something, but the organisation must decide whether that output can change a record, reach a customer, alter code or trigger a payment.

Many teams treat this decision as a thin guardrail around the model. A confidence score falls below a threshold, so the item is sent to “human review”. That label hides a process transformation. Someone must receive the item, understand why it was routed, recover its context, make a decision, record the reason and return the result to the workflow. If those steps are not designed, the new feature has not removed work. It has converted visible work into interruptions, queue searches and repeated investigation.

The practical design unit is therefore not the model call. It is the complete path from request to accountable outcome, including the routes where the system should not finish the task. This changes daily work for product owners and engineers: exception handling moves into the initial design, reviewers become users of a deliberate interface, and deployment decisions include review capacity as well as model quality. The central trade-off is no longer automation versus manual work. It is where each decision can be made with the best combination of speed, evidence and control.

A model score describes the model, imperfectly; it does not describe the whole decision. The same predicted intent can justify different routes depending on what follows. Misclassifying an internal knowledge-search query may be reversible after one click. Misclassifying a request that changes a customer’s entitlement may create a financial, contractual or relationship cost. Routing should combine evidence about likely correctness with the consequence of being wrong.

Four questions make that policy concrete. What action will the output enable? Can that action be reversed cleanly? What evidence is available at the moment of decision? Is an identifiable reviewer likely to perform better on this type of case? Research on learning to defer formalises a related accuracy-cost trade-off: a deferral mechanism should consider expected model error and the cost or value of involving an expert, rather than treating abstention as a model-only property.

This changes the engineering design. Thresholds become versioned business rules tied to an action class, not constants buried in prompt code. A team might allow a system to draft every response, permit automatic sending only for a narrow set of informational requests, and require approval when an answer changes an account or commits the organisation. The trade-off is explicit: more controls add latency, while too little control transfers the cost of errors downstream. Teams can act by writing routing rules in operational language—action, evidence, consequence, reviewer—before tuning numerical thresholds.

Once deferral enters production, its queue becomes part of the service. Someone must own routing accuracy, reviewer availability, ageing items and recovery when a downstream system fails. A named reviewer role is not enough if the role has no allocated capacity. Queue volume can rise after a model update, a policy change, a new customer segment or a deterioration in source data. Without monitoring, that demand appears as personal overload rather than a product incident.

Track arrival rate, age, time to decision, route changes, requests for missing evidence, reviewer amendments and unresolved items by reason. Separate immediate interventions from work that can wait. Google’s SRE guidance distinguishes outputs that require action now, action within days and logging for later analysis; the same discipline prevents every AI exception from becoming an urgent interruption. The queue also needs a service objective appropriate to the business process, plus an explicit response when the objective is missed.

Capacity is a release constraint. If a proposed change increases deferrals beyond what reviewers can absorb, the team can narrow the launch, add evidence collection, improve routing or delay the change. It should not silently ask staff to compensate. Google’s account of automation in SRE describes automation as a force multiplier rather than a panacea; applying that idea here means measuring where the force lands. The operational trade-off is coverage against sustainable review load, and the decision belongs to product, engineering and operations together.

Model accuracy, task success and response quality remain useful engineering measures, but they do not show whether production work reaches a sound conclusion. A system can score well offline while routing its hardest cases to the wrong specialists, overwhelming a queue or presenting reviewers with evidence that anchors them on a bad proposal. Conversely, a deliberately narrow system may automate less yet produce a better overall service because its boundaries are clear and its exceptions are resolved well.

Measure final outcome quality across automated and reviewed routes, not just model correctness. Add avoidable-deferral rate, incorrect-automation rate, reviewer amendment rate, decision latency, queue age, missing-evidence frequency and the proportion of repeated exceptions converted into durable fixes. Where possible, retain enough information to compare what the model proposed with what the reviewer decided; research on deferring systems notes that evaluating the combined strategy requires attention to both model and human outcomes.

The practical goal is not zero deferral. It is a system in which every route has a purpose, an owner and an observable result. When teams build that path before polishing the happy path, uncertainty stops being an embarrassing edge case. It becomes structured operational information: evidence about where automation is appropriate, where the process is underspecified and where the organisation should improve next.