Human-in-the-loop AI agents: what approval has to mean

An agent that asks before it acts is only safe if the asking is enforced by the platform, not by the prompt.

Converge Team, 2026-11-12

A human-in-the-loop AI agent is an agent whose changes to real records wait for a person to confirm them. The agent may read, search and propose on its own. It may not write on its own.

The phrase is used loosely, so it helps to say where the loop is enforced. There are two places it can live, and only one of them holds.

In the prompt, or in the platform

An instruction such as "always ask the user before updating an order" lives in the prompt. The model usually follows it. Nothing stops it when it does not, and nothing records that it did not.

A platform rule is different. The agent has no way to write except through a tool, and the tool itself stops and waits. In an agent built in Converge, every tool runs in one of two modes, auto or confirm, and action and command tools are always in confirm mode. No setting on the agent turns that off.

Tool typeKindRuns without a person?
object_queryReadYes
functionReadYes
user_connectorsRead or write, per connector actionOnly the actions the connector marks read-only
actionWriteNo
commandWriteNo

What happens when an agent wants to write

A conversation with an agent is a session, kept as an append-only log of events. When the agent calls a write tool, the session moves to the state awaiting_review and stops. A person sees the exact call (which action, which parameters) and confirms or rejects it. The decision is itself an event in the log, next to the call it answers.

The write then goes through an action: a named change with declared parameters, validation, and a check of who is asking. The agent runs with the credentials of the person it works for, so it can never do more than that person could do by hand.

Where the claim stops

The rule above is about agent sessions. Two other things are worth knowing, because a precise claim is more useful than a broad one.

  • A Logic function can apply actions as one of its steps, and a schedule can run it unattended. There is no per-call review in that case. The action's own criteria and approval still apply, which is why approval on the action matters (see below).
  • A connector decides which of its own actions are read-only. Converge runs those automatically and holds the rest for confirmation.

Two different approvals

Reviewing an agent's tool call is not the same thing as approving a business change, and a careful setup uses both.

  • Review answers "did the agent understand me?". The person who asked for the work checks the call before it runs.
  • Approval answers "should this change happen at all?". An action type can require a second person, and applying it creates a pending request instead of changing data. Someone with approval permission accepts or rejects it, and nobody approves their own change.

An agent that proposes a price correction, through an action that requires approval, therefore passes two people before a source system sees it: the user who reviews the call, and the approver who signs off the change.

Questions to ask of any agent platform

  1. Can an agent write without a person, under any configuration?
  2. Is the pause enforced by the server, or requested in the prompt?
  3. Does the agent act with its user's permissions, or with a service account that sees more?
  4. Is the review decision recorded with the call it approved?
  5. Can a second person be required for the change itself?

If the answer to the first question is "yes, if you configure it that way", the loop is a default, not a guarantee.

Read more

Converge