Human Approval in AI Automation - Put the Pause Before the Side Effect

An AI application that drafts an email and one that sends the email may use the same model, but they do not carry the same kind of authority. The first produces a proposal. The second changes the outside world. Where, exactly, should a person be asked to approve that change?

“Keep a human in the loop” sounds like an answer, yet it leaves the difficult parts unspecified. Which actions pause? What does the reviewer see? What happens if the target or amount changes after approval? Can the model route around the prompt through another tool? A confirmation dialog can become ceremony rather than control if the application cannot answer those questions.

This article develops a practical design pattern for small AI automations. It is a synthesis of security and risk-management guidance, not a universal standard: separate proposal from execution, classify actions by their consequences, bind approval to one exact proposal, and let ordinary application code enforce the boundary.

Approval belongs outside the model

Tools turn model output into capability. They may retrieve data, but they may also send messages, change records, publish pages, make purchases, or delete files. The OWASP GenAI Security Project separates three causes of excessive agency: too much functionality, too much permission, and too much autonomy. Human approval addresses part of the autonomy problem. It does not remove the other two.

That distinction matters because the model is not an authorization service. It can propose send_message, explain why the call seems appropriate, or even classify the call as low risk. The application must still decide whether the authenticated user may send to that recipient, whether the operation requires approval, and whether the approved parameters are the ones eventually executed.

OWASP consequently recommends enforcing authorization in downstream systems rather than relying on an LLM to decide whether an action is allowed. OpenAI's agent-building guide makes a compatible point: guardrails should be layered with authentication, authorization, strict access control, and standard software security. Approval is one layer in that stack, not a substitute for the stack.

Pause according to consequence, not because a tool exists

Prompting for every call is not necessarily safer. If a reviewer must approve dozens of harmless reads, the consequential request can become one more button in a long queue. A more useful starting point is to classify an action using several questions:

• Does it only read, or does it create a side effect? Looking up a public document differs from publishing one.
• Is the effect reversible? Saving a private draft differs from sending it to a mailing list; moving a file to recoverable trash differs from permanently erasing it.
• What is its scope? Editing one record differs from changing every matching record.
• What authority does it use? A tool limited to one user's drafts differs from a shared administrative credential.
• What could be exposed or transferred? A call may leak data or commit money even when it does not look destructive.

This is a practical taxonomy, not one prescribed list. NIST's Generative AI Profile says oversight arrangements may need to vary with context and risk. OpenAI's guide similarly suggests rating tools using factors such as read versus write access, reversibility, account permissions, and financial impact. For a small application, those factors can produce a simple policy: allow narrow reads, automatically perform low-impact reversible writes, and pause before external, sensitive, broad, expensive, privileged, or irreversible effects.

Some actions should remain unavailable rather than merely sit behind a prompt. If an automation only needs to summarize mail, do not give its tool the ability to send or delete mail. If it needs to update one type of record, an unrestricted SQL or shell tool is a much wider capability than the task requires. Removing unnecessary power is stronger than repeatedly asking a person whether that power should be used.

Make the proposal reviewable

A reviewer cannot meaningfully approve “continue.” The interface needs to describe the proposed side effect in terms a person can inspect. Depending on the operation, that may include:

• the action and the account or identity that will perform it;
• the target, such as a recipient, repository, file path, record, or publication channel;
• material parameters, such as the message body, patch, amount, permission, or deletion mode;
• the expected scope and whether the result can be reversed;
• the relevant source context and any untrusted content that influenced the proposal;
• the limits: what this approval will not authorize.

The Model Context Protocol tool specification offers a concrete example of this direction. It recommends clear indicators when tools are invoked, confirmation for operations, and showing tool inputs before sensitive calls. It also says the protocol does not mandate a particular interaction model. In other words, exposing a tool-call mechanism does not by itself design a trustworthy review experience.

A preview should favor the effect over the model's reasoning. For an email, show the actual recipients, subject, body, and attachments. For a code change, show the repository, branch, and diff. For publication, show the final title, destination, visibility, and content. A confident paragraph about why the action is sensible is not a replacement for the bytes and destinations that will change.

Bind “approve” to one exact action

Here is the central engineering inference: approval should refer to an immutable, or at least integrity-protected, proposal. The proposal receives an identifier and records the material action data. The reviewer approves that identifier. The executor then consumes that same proposal rather than accepting a fresh set of model-generated arguments.

Without this binding, the interface may display one recipient while the execution request contains another. A model could revise a draft after review, a background process could alter a record, or an ordinary race condition could make the preview stale. The problem is not uniquely “AI”; it is a familiar gap between checking something and using it.

A small state machine makes the boundary visible:

proposed -> pending_review -> approved -> executing -> completed
                           \-> rejected
                           \-> expired
                                          \-> failed

Material changes create a new proposal and require new approval. Pending approvals expire. An approved proposal is consumed once, or execution is made idempotent where retries are necessary. The exact mechanism can be a canonical serialization plus a cryptographic digest, a versioned database row that cannot be edited after review, or another design suited to the application's threat model. The sources do not prescribe one storage format; the invariant is the important part: the reviewed action and the executed action must not silently diverge.

Recheck authority at execution time

An approval says that a particular reviewer accepted a particular proposal at a particular time. It does not prove that the action is still permitted. Roles may change, a record may already be closed, a spending limit may have been reached, or the target may no longer exist.

The executor should therefore authenticate the actor, re-evaluate current authorization, validate all inputs, and check business invariants immediately before the side effect. It should use the narrowest downstream credential available and execute in the user's context where that is possible. These controls follow the least-permission and complete-mediation guidance in OWASP's excessive-agency chapter. The MCP specification likewise requires servers to validate tool inputs and implement access controls.

This separation also blocks an attractive but unsafe shortcut: letting the model call a broad tool after a boolean approved=true appears in its context. The approval service and executor, not the conversation, should own the transition. A new prompt injection, model response, or tool result cannot grant itself permission.

Keep evidence without turning logs into another leak

The lifecycle should leave enough evidence to reconstruct what happened: who requested the operation, which proposal was shown, who approved or rejected it, when it expired or executed, which policy version applied, and what result or error followed. NIST recommends defined oversight roles, monitoring and documenting overrides, and post-deployment processes for appeal, incident response, recovery, and change management. MCP also recommends logging tool use for audit purposes.

More logging is not automatically better. Tokens, passwords, private message bodies, and unnecessary personal data do not belong in an audit record merely because the system can store them. A useful log may keep identifiers, digests, classifications, timestamps, and redacted summaries while placing sensitive payloads under separate access and retention rules.

Failure paths deserve the same design attention as approval. What happens after a timeout? Can the user distinguish “not sent” from “response lost after sending”? Can two reviewers execute the same proposal twice? Is a partially completed multi-step action recoverable? Human review does not answer these transaction questions.

A human click is still a fallible control

A person can approve the wrong action, skim an unclear preview, or become accustomed to accepting prompts. For the high-risk systems within its scope, Article 14 of the EU AI Act illustrates how much more “human oversight” can mean: overseers should be able to understand capabilities and limitations, notice unexpected performance, remain aware of automation bias, interpret output, disregard or reverse it, and stop the system. Those legal requirements do not apply to every small AI application, and the Act's narrow two-person verification rule is not a general rule for ordinary tool calls. The useful lesson is more modest: an approval is meaningful only when the reviewer has adequate context, competence, authority, and a real way to refuse.

Human approval also does not neutralize malicious content. A poisoned document may still shape a proposed action; a compromised reviewer account may still approve it; an overpowered executor may still cause excessive damage. Sandboxing, input and output handling, rate limits, monitoring, narrow tools, and recovery plans remain relevant. The UK's secure AI development guidelines place AI-specific risks inside a full lifecycle of secure design, development, deployment, operation, and maintenance rather than treating model behavior as the whole security problem.

Three examples of a real boundary

Email

The model may search messages and create a private draft using read and draft-only capabilities. Sending pauses. The review shows every recipient, the exact subject and body, attachments, and the sending account. Approval references that frozen draft; changing a recipient invalidates it. The mail service still checks that the user may send from the account.

Publishing

Research, outlining, and saving an unpublished revision may proceed within a restricted workspace. Publication pauses at the transition to a public URL. The proposal includes the final content version, title, destination, visibility, and scheduled time. Approval of one content digest cannot publish a later rewrite. A separate deployment policy can still reject a missing translation, thumbnail, or quality gate.

Files

Reading from an allowed project tree may be automatic. Deletion presents canonical paths, count, total scope, and whether recovery is possible. The executor rejects paths outside the allowed root and rechecks them after resolving links. For especially valuable data, the better design may be no permanent-delete tool at all: move to quarantine and let a separate retention process remove it later.

These examples share a pattern, not a universal threshold. The valuable pause sits immediately before the consequential boundary, and the person reviews the effect that will cross it.

When not to build an agent around the task

Approval architecture can make an automation more controlled, but it can also be a warning that the automation is unnecessarily complicated. Anthropic's engineering article on agents recommends starting with the simplest solution and distinguishes predefined workflows from systems in which a model dynamically directs tool use. OpenAI similarly notes that a deterministic solution may be enough when a task does not require model-led handling of ambiguity.

If the steps and policy are already known, ordinary application code can collect inputs, validate them, display a preview, and execute after confirmation. The model may help draft content without controlling the transaction. If a consequence is intolerable, difficult to review, or impossible to recover from, the right answer may be to leave that capability outside the automation entirely.

Conclusion

A useful approval step is not a decorative question asked by the model. It is an application-enforced boundary that answers four things clearly: who may approve, which exact action they reviewed, how long that approval remains valid, and what code is allowed to execute afterward.

That boundary works best beside narrower tools, least-privileged credentials, current authorization checks, careful previews, audit records, and recovery paths. It still depends on fallible people and cannot turn an unacceptable operation into a safe one. The honest design question is therefore not simply “Is there a human in the loop?” It is “What can this person actually see, refuse, stop, and hold to the action that follows?”

References

1. National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, July 2024. Accessed October 26, 2026.
2. OWASP GenAI Security Project, LLM06:2025 Excessive Agency, 2025 edition. Accessed October 26, 2026.
3. Model Context Protocol, Tools - Specification 2025-06-18, June 18, 2025. Accessed October 26, 2026.
4. UK National Cyber Security Centre, Guidelines for Secure AI System Development, version 1.0, November 27, 2023. Accessed October 26, 2026.
5. OpenAI, A Practical Guide to Building AI Agents, January 15, 2026. Accessed October 26, 2026.
6. Erik S. and Barry Zhang, Anthropic, Building Effective Agents, December 19, 2024. Accessed October 26, 2026.
7. European Parliament and Council of the European Union, Regulation (EU) 2024/1689, Article 14: Human Oversight, Official Journal, July 12, 2024. Accessed October 26, 2026.