Human-in-the-loop for AI agents: approval patterns that actually scale

The first version of most agent projects has one of two oversight models. Either a human approves every action, which is safe but so tedious that people start clicking "approve" without reading, or nobody approves anything, which is fast right up until the agent emails the wrong customer or deletes the wrong record.

Neither scales. The teams running agents successfully have moved to something in between: routine work runs on its own, edge cases and risky actions go to a person, and a sample of everything gets reviewed. This is human-in-the-loop (HITL) done properly.

This guide covers the patterns, how to decide which actions need a human, how to design the review experience, and how to safely give an agent more autonomy over time.

Why "approve everything" fails

Approval on every step feels safe, but it breaks down in practice:

• Alert fatigue. After the fiftieth correct suggestion, reviewers stop reading. Approval becomes a rubber stamp, and the one bad action sails through.
• It destroys the value. If a person must check every step, the agent saves little time.
• Reviewers lack context. A bare "Approve: send_email(to=…, body=…)" gives a reviewer little to judge.

Good oversight is selective: it puts human attention where it changes outcomes.

Route actions by risk

The core idea is simple: before an action executes, classify how much damage it could do if wrong, then route it accordingly.

Human attention goes where it changes outcomes.

Risk depends on two things:

1. Impact: what happens if this action is wrong? Can it be undone? Who is affected?
2. Confidence: how sure are we this action is right? That signal can come from the model, from validation checks, or from how unusual the request is.

Two simple rules cover most cases: irreversible actions need approval, and anything leaving your organisation needs approval until it has earned trust.

Five HITL patterns

1. Approval gate

The agent pauses before an action and waits for a person to approve, edit or reject it. Use this for irreversible, external or financial actions.

Make it fast to do well:

• Show the proposed action in plain language: "Refund $42.00 to Jane Doe (order #1182)".
• Show the reason the agent gives and the evidence it used.
• Offer Approve, Edit then approve, and Reject with a note. The note becomes feedback for the agent's next attempt.

2. Confidence threshold

The agent proceeds automatically when confidence is high and escalates when it is low. For example, a ticket classifier auto-routes when its top category is clear and asks a human when two categories are close.

Don't rely on the model's self-reported confidence alone. Combine it with objective checks: Did validation pass? Is this customer on a watchlist? Is the amount unusual? Did the retrieved documents actually contain the answer?

3. Sampled review

The agent acts on its own, and a percentage of completed actions (say 5–10%, plus anything flagged) is reviewed after the fact. This catches quality drift without slowing every task.

Sampling is how you measure whether the agent deserves more autonomy, so record reviewer verdicts in a structured way (correct / minor issue / wrong).

4. Draft-then-send

The agent prepares the work (an email, a pull request, a report) and a human sends, merges or publishes it. This works especially well for writing tasks: the agent does 90% of the effort and the human applies judgment at the last step.

5. Escalation path

When the agent is stuck, unsure or detects something sensitive (a complaint, a legal threat, a security issue), it should hand off to a human with a summary rather than guess. An agent that can say "I'm not sure. Escalating to a person" is far safer than one that always produces an answer.

Designing the review experience

Most HITL systems fail on interface design, not on logic. A good review screen:

• Fits on one screen: action, reason, evidence, and a clear diff of what will change.
• Highlights what's unusual: amounts above normal, new recipients, first-time actions.
• Makes rejection informative: a quick reason picker plus an optional note.
• Is fast: keyboard shortcuts, batch approval for low-risk items in a queue.
• Has deadlines and fallbacks: if nobody approves within a set time, the action expires safely rather than executing.

Audit everything

Every decision, automatic or human, should be logged:

{
  "run_id": "run_7c1e",
  "action": "send_email",
  "args_summary": "to=jane@example.com, subject='Your refund'",
  "risk": "high",
  "route": "approval",
  "decision": "approved_with_edit",
  "reviewer": "ops@company.com",
  "decided_at": "2026-10-15T10:42:00Z"
}

Audit logs answer the questions you will eventually be asked: Who approved this? What did the agent see? How often do we overrule it? They are also the dataset you use to tune thresholds.

Earning autonomy: how to loosen oversight safely

Oversight should change as evidence accumulates. A practical progression:

1. Shadow mode. The agent proposes actions but nothing executes. Humans do the work and you compare.
2. Approve everything. Every action needs approval. Measure the approval rate per action type.
3. Auto-approve the proven. Action types with a very high approval rate over enough samples move to auto with sampling.
4. Raise limits gradually. Refund ceilings, recipient lists and allowed operations grow in small steps.
5. Keep a kill switch. Any action type can be moved back to approval instantly if quality drops.

The data from sampled reviews and approval decisions is what justifies each step. "It seems fine" is not a reason to remove a human from the loop; "99% of 500 sampled refunds under $50 were correct" is.

Common mistakes

• Approving the action without the context. Reviewers need the evidence, not just the command.
• Letting approval queues grow unbounded. Stale approvals are dangerous; expire them.
• Treating the model's confidence as truth. Combine it with hard checks.
• No path for "I don't know". Agents should be able to escalate instead of guessing.
• Never revisiting thresholds. Oversight that never loosens wastes people's time; oversight that never tightens hides drift.

Human oversight and regulation

Human oversight isn't only good engineering; in some contexts it's a legal requirement. The EU AI Act, for example, requires high-risk AI systems to be designed so that people can effectively oversee them while they are in use (Article 14). The obligations include being able to understand the system's limits, interpret its outputs, decide not to use or to override them, and stop the system safely. The Act also calls out automation bias: the tendency to over-rely on a system's output.

Most everyday agents (drafting emails, triaging tickets) won't be classified as high-risk, but the same principles make any agent safer. If your use case touches areas like hiring, credit, education, essential services or law enforcement, get legal advice on which obligations apply and when.

HITL in practice: three examples

Customer support refunds. The agent reads the ticket, checks the order and proposes a refund. Refunds under a small limit for customers in good standing are auto-approved and sampled; anything above the limit, or for a flagged account, goes to a support lead with the order history attached.

Code changes. A coding agent opens pull requests but never merges. Automated checks run first; a developer reviews the diff. Over time, low-risk categories (dependency bumps with passing tests, documentation fixes) might move to auto-merge with sampling.

Content and communications. An agent drafts newsletter copy and social posts. A person edits and publishes. Because writing tasks are subjective, draft-then-send usually stays permanent, and that's fine: the time saving is in the drafting.

Measuring whether your oversight works

Track these per action type:

• Approval rate: how often reviewers approve without edits.
• Edit rate: how often they approve with changes, and what they change.
• Rejection reasons: the categories behind "no".
• Time to decision: how long actions wait in the queue.
• Escaped errors: problems found later in auto-approved actions.

A high approval rate with low escaped errors is the evidence that an action type can move to more autonomy. A rising edit rate is an early warning that quality is drifting.

Frequently asked questions

What is human-in-the-loop in AI?

Human-in-the-loop (HITL) means people review, approve or correct an AI system's decisions at chosen points instead of letting it act fully on its own. For AI agents, it usually means routing risky or uncertain actions to a person while routine actions run automatically.

When should an AI agent ask for human approval?

Before actions that are irreversible, affect people outside your organisation, move money, change permissions or delete data, and whenever the agent's confidence is low or the request is unusual. Low-risk, reversible actions can usually run automatically with sampled review.

What is the difference between human-in-the-loop and human-on-the-loop?

Human-in-the-loop means a person must act before certain decisions take effect, such as approving an action. Human-on-the-loop means the system acts on its own while a person monitors and can intervene or stop it. Many production agents mix both depending on risk.

How do you avoid approval fatigue?

Only send high-impact or uncertain actions for approval, show reviewers the reason and evidence on one screen, batch low-risk items, and move action types to automatic execution with sampled review once their approval rate proves they are reliable.

Can AI agents be fully autonomous?

For narrow, low-risk, reversible tasks with strong validation, yes. For actions with real-world consequences, most teams keep some human oversight, loosening it gradually as audit data shows the agent is reliable, and keeping the ability to restore approvals instantly.

How do you audit AI agent decisions?

Log every proposed action with its arguments, risk level, routing, decision, reviewer and timestamp, linked to a run ID that points to the full trace of what the agent saw. Review samples regularly and use the data to adjust thresholds.

Does the EU AI Act require human oversight?

Yes, for high-risk AI systems. Article 14 requires them to be designed so that people can effectively oversee them during use, including understanding their limits, overriding outputs and stopping them safely. Requirements depend on how a system is classified, so check which obligations apply to your use case.

What is automation bias?

Automation bias is the tendency of people to over-trust and over-rely on an automated system's output, approving or accepting it without enough scrutiny. Good human-in-the-loop design counters it by showing evidence, highlighting unusual actions and keeping approval queues small enough to review carefully.