The approval point in an AI agent is not set by the model confidence score. It is set by two variables only: how reversible the action is, and how much damage it causes if it is wrong. An irreversible action with real damage — an email to a customer, a refund, a record deletion — requires approval before execution. A cheaply reversible action does not need approval at all, only sampling. The common failure is not too few approvals but the opposite: too many, which within weeks turn into a rubber stamp and lose the entire value they were built for.
Three mistakes that repeat in almost every implementation
First: approval on everything. An organisation nervous about the agent puts a human in front of every action. The result is that the agent saves no time and adds a step, while the approver — who sees dozens of near-identical requests a day — stops reading. The approval stays in the flowchart but no longer filters anything.
Second: approval driven by the model confidence score. A confidence score measures how plausible the output looks to the model, not how correct it is and not what it costs if it turns out wrong. An agent can be extremely confident about an email address it invented. A variable uncorrelated with damage cannot be the trigger for a decision about damage.
Third: approval without authority. An approver who lacks the authority to reject, or for whom rejecting creates extra work, will approve. An approval point without real organisational authority behind it is documentation, not control.
The equation that decides where the approval point goes
Two variables, four cells. That is the whole theory, and every other decision follows from it.
| Reversibility of the action | Low damage if wrong | High damage if wrong |
|---|---|---|
| Instantly reversible (draft, internal tag, summary) | No approval, weekly sampling | Daily sampling + full log |
| Reversible at a cost (record update, status change) | Post-hoc approval with an undo window | Approval before execution |
| Irreversible (customer email, refund, deletion) | Approval before execution | Approval before execution + second approver |
Note what is missing from the table: the model confidence level. It can be used to decide how closely to inspect within a cell, but never to move an action from one cell to another. The classification of the actions themselves should be derived from the agent permissions, as detailed in the permissions guide.
Four approval patterns and the operational difference between them
| Pattern | How it works | When it fits |
|---|---|---|
| Blocking approval | The agent stops and waits for a human before executing | Irreversible actions involving a third party or money |
| Undo window | The action executes after a short delay during which it can be cancelled | Cheaply reversible, high-volume actions |
| Sampling | Everything executes; a fixed percentage is reviewed afterwards | Internal actions at large volume |
| Exception-based escalation | The agent acts alone and stops only under predefined conditions | Mature processes with measurement history |
The undo window is the most underused pattern and usually the most correct one. It preserves full speed on the normal path and shifts the cost of control only to cases where somebody actually noticed a problem. For actions that can be cancelled within minutes, it delivers almost all the protection of a blocking approval with none of its costs.
The real problem: approval that became a rubber stamp
The bias that destroys approval points is known in the research literature as automation bias — the tendency to accept a system output without verification, and sometimes to abandon a correct human judgement in favour of an incorrect recommendation. A study in computational pathology found that presenting an incorrect recommendation raised the rate of commission errors by roughly 66% on low-complexity tasks compared with a group that received no recommendation at all.
Load makes the bias worse. In information security, where alerting and approval mechanisms have existed for decades, 2026 industry reports describe roughly 3,000 alerts per day in the average organisation, of which around 63% are never handled at all and around 46% turn out to be false positives. That is precisely the path an AI agent with too many approval points creates, only faster.
The design conclusion runs against intuition: for approval to work you need fewer approvals, not more. Every approval you add cheapens the value of all the others.
What has to appear on the approval screen
An approver handed a yes-or-no question without context will say yes. An approval screen that works shows five things, in this order.
- The exact action. Not a general description but the final value: the address, the amount, the field and the new value alongside the old one.
- The source. Which record, document or conversation this was based on, with a direct link to it.
- What was considered and rejected. The alternative the agent weighed and did not choose. This is the detail that turns automatic approval into a real check.
- The cost of being wrong. Whether the action is reversible, and if so within what time and at what cost.
- One-click rejection with a reason. The reason is stored and feeds into the test set, otherwise the same mistake will come back.
The fifth item is the difference between control and paperwork. A rejection that never returns to the system is work repeated over and over on the same error. How to feed rejections into the test set is covered in the evals guide.
The stop button is not a nicety — it is a requirement
The EU AI Act defines human oversight in Article 14 as a structural requirement for high-risk systems. The text requires that a person be able to decide not to use the system, to disregard, override or reverse its output, and to interrupt the system through a stop mechanism that brings it to a safe state.
Three practical implications follow, including for organisations outside the high-risk scope. First, the stop mechanism has to be implemented rather than merely documented. Second, somebody needs to know how to trigger it without consulting anyone. Third, triggering it has to be organisationally acceptable — a stop that invites an internal investigation will not be used in time.
A simple test: ask a random person on the team to stop the agent right now. If they do not know how, or if they ask who needs to be notified first — you do not have a stop button.
How to tell whether the approval point is in the right place
An approval point is a design decision that has to be measured, not assumed. Three metrics are enough.
| Metric | What it says | Healthy range |
|---|---|---|
| Correction rate | How often a human changed what the agent proposed | Neither zero nor most cases |
| Average decision time | How much time the approver actually invested | Enough to read the action and the source |
| Escalation rate | How often the agent stopped on its own initiative | Stable over time |
A correction rate approaching zero means one of two things: either the agent is ready for the approval to be removed, or the approval is no longer being read. The only way to tell them apart is a blind sample — take fifty approved actions and review them from scratch. If the review finds errors that were approved, the problem is not the agent.
A recommended rollout path
- First two weeks: the agent only proposes, a human executes. You collect a real correction rate.
- Once the correction rate drops below a predefined bar: move to execution with an undo window, on reversible actions only.
- Irreversible actions stay under blocking approval until there is at least a month of measurement history.
- Any change to the prompt, the tools or the model version moves the agent back one stage until the test set has been re-run.
The structural background — what actually runs inside the agent and where the approval point sits relative to the decision loop — is detailed in Anatomy of a Business AI Agent.
Frequently Asked Questions
Where should the human approval point sit in an AI agent?
It is determined by two variables: how reversible the action is and how much damage it causes if wrong. Irreversible actions with real damage require approval before execution; cheaply reversible actions are adequately covered by an undo window or post-hoc sampling. The model confidence score is not a valid variable for setting the approval point.
Why does approving every action actually cause harm?
Because it creates a load that leads to automatic approval. An approver who sees dozens of near-identical requests a day stops reading them, and every approval added cheapens the value of the others. Fewer approval points, focused on irreversible actions, filter more.
What is the difference between a blocking approval and an undo window?
A blocking approval halts the agent until a person decides, which suits irreversible actions. An undo window executes the action after a short delay in which it can be cancelled, preserving full speed and suiting reversible actions at high volume.
How do you know the approval point is in the right place?
By the correction rate. If almost nobody corrects anything, approval has become a rubber stamp or is unnecessary. If a large share of actions gets corrected, the agent is not yet ready for the autonomy it was given. Average decision time is another signal: a decision taking a few seconds is not a check.
Is there a legal requirement for human oversight?
Under the EU AI Act, Article 14 requires high-risk systems to be capable of effective human oversight, including the ability to disregard or reverse the output and to stop the system in a safe state. Even outside that scope, it is a reasonable design benchmark for any agent operating on customers or money.
Continue reading
Running MCP in the Enterprise: Gateways, Least Privilege and 200,000 Exposed Servers – The infrastructure layer that makes an approval point enforceable rather than advisory.