How to Keep Humans in Control of AI Agents (Without Slowing Down)

Legal wants a person to approve every action the AI agent takes. The business case assumed the agent would save hundreds of hours. Both can't be true.

Australia's AI Reference Architecture (AIRA) settles the argument. It says human oversight should not mean a human approval step in every loop. This third post in our AIRA series covers what it asks for instead, and how to build it.

What does AIRA mean by human agency, oversight and control?

Human agency, oversight and control (HAOC) is the part of AIRA that keeps people meaningfully in charge of AI systems. It covers how people understand what the AI is doing, how they step in when needed, and how every AI action stays traceable to a human decision or authorisation. It sits across the whole architecture, not in one layer.

AIRA splits it into two kinds of control:

• Design-time controls. These are built into the system: limits, approval gates, automatic pauses and rollbacks.
• Run-time requirements. These stay with people: the roles, skills and attention needed to operate the system safely day to day.

It then groups the mechanisms into two families. Decision support helps people understand and act. Intervention lets people change or stop what the AI is doing. Both are covered below.

For AI that co-produces work with a person, such as drafting or summarising, AIRA adds one more expectation. The person must be able to verify the output, and authorship must stay genuinely theirs. The DTA puts it this way: people "should still verify the output and retain responsibility for the final work."

Why is "a human approves everything" the wrong default?

Requiring human approval on every AI action feels safe but often makes systems less safe. People who approve hundreds of routine items stop reading them closely, so the approval becomes a rubber stamp. The AI also stops saving time, so teams look for ways around the controls.

AIRA is direct about this. Human oversight does not mean designing systems where human approval becomes an unnecessary bottleneck in every loop. Instead, automation should run within limits that people can observe, understand, define, approve, pause, revise and revoke in time.

Three problems come with approve-everything designs:

• Rubber-stamping. Approval rates drift towards 100% and reviewers stop catching errors. Researchers call this automation bias.
• Lost value. If every step waits for a person, the agent is just a slower form.
• Misplaced attention. Reviewers spend the same effort on a routine status update as on an irreversible payment.

The better question is not "does a human approve this?" but "where does human judgement change the outcome?"

Humans set the limits. Automation runs inside them.

Where should a human sit in an AI agent's workflow?

Put human approval where an action is irreversible, high-impact or sensitive. Let pre-agreed limits and automatic safeguards handle routine, reversible actions. Use monitoring and sampling for everything in between.

How deep each control goes also depends on the use case's overall risk tier, which our post on applying AIRA by risk covers. Within any one system, the table below shows where the human sits for each kind of action.

Our guide to AI agent human approval goes deeper on which actions need a person and how to make the approval step hold.

What are AIRA's four intervention controls?

AIRA names four ways for people to change or stop what an AI system is doing: human override, circuit breakers, irreversibility guardrails and transfer of control. Together they make sure people can step in at the right moment, without watching every step.

1. Human override. Authorised people can change, reject or override what the AI recommends or does. Example: an officer rewrites an AI-drafted letter before it is sent, and the record shows both versions.

2. Circuit breakers. The system pauses or restricts itself automatically when agreed safety, risk or performance limits are crossed. Example: an agent processing claims stops taking new cases when its error rate in sampled reviews passes an agreed level, and alerts the supervisor.

3. Irreversibility guardrails. Actions that can't be undone, or that have high impact, need heightened review or approval before they happen. Example: an agent can prepare a payment but can't release it until an authorised person approves.

4. Transfer of control. Work hands over safely between the AI and a person when needed, without losing context. Example: a customer conversation moves from the AI to a human agent, with the full history and the reason for handover.

The important nuance is that circuit breakers, automatic remediation and anomaly-driven rollback can act without approval on each action, provided the conditions that trigger them were authorised in advance. The DTA describes these as "pre-approved safeguards" that can pause operations or reverse changes when risks emerge. That is how you get safety at machine speed.

Approve the trigger in advance. Then the safeguard can act without waiting for anyone.

What are AIRA's decision-support controls?

Decision-support controls help people notice what matters and act on it, without taking away their judgement or accountability. AIRA names three: alerting, anomaly highlighting and escalation routing. They make sure scarce human attention lands on the cases that need it.

• Alerting. Tell the right person about significant events, risks, threshold breaches or required actions. Example: the supervisor gets a notification when pending approvals are waiting longer than their agreed response time.
• Anomaly highlighting. Surface unusual patterns, unexpected behaviour or low-confidence outputs for review. Example: drafts where the AI was unsure, or where sources conflicted, are flagged at the top of the review queue instead of mixed in with routine ones.
• Escalation routing. Send complex, high-risk or unresolved situations to the right decision-maker. Example: a complaint that mentions hardship or legal action skips the standard queue and goes to a senior officer.

Good decision support is what makes it safe to remove approval from routine steps. If people trust that unusual cases will reach them, they don't need to see every case.

What is an agent supervisor, and who should do it?

An agent supervisor is the person responsible for judging whether AI agents are doing what they are meant to do, and stepping in when they aren't. AIRA uses this as its example of a run-time role an agentic system may need. It is a job, not a feature.

Think of it like a team leader for a group of digital workers. The supervisor:

• Watches the live work: what agents are doing, their queues, and their error and escalation rates
• Judges whether agents are staying within their objectives, especially near irreversible actions
• Intervenes by pausing an agent, taking over a case, or changing an agent's limits
• Improves the system by turning recurring problems into new rules, tests or training material

Who should do it: someone who knows the business process deeply, usually an experienced team leader from the area the agents work in, not an IT administrator. They need training in how the agents work, the authority to pause them, and enough time in their day to do the job.

A useful test: if your agents went wrong at 10am on a Tuesday, who would notice first, and could they stop it by 10:15?

If your agents went wrong at 10am, who would notice, and could they stop it by 10:15?

How do you design oversight for how people actually behave?

AIRA draws on behavioural science and human factors research. It names three risks that quietly break human oversight: automation bias, vigilance decrement and skill erosion. In the DTA's words, systems should "reflect how people actually interact with automation." Each risk needs a design response, not just a policy.

One measure is worth tracking from day one: how often reviewers change what the AI produced. If it drops close to zero, either the AI is excellent or people have stopped looking. You need to know which.

A 99% approval rate isn't oversight until you know reviewers are still reading.

How do you prove human oversight works to an auditor?

You prove oversight with records and tests, not with a policy document. An auditor will want to see that the controls exist, that they fired when they should have, and that the people involved were authorised and actually exercised judgement.

Keep evidence of five things:

1. The rules. Which actions need approval, the thresholds and limits, and who authorised them and when.
2. Attribution. For every action, whether the AI did it, a person did it, or a person approved the AI's proposal, and who that person was.
3. Interventions. Every override, pause, escalation and handover, with the reason.
4. Control tests. Evidence that circuit breakers and guardrails were tested, for example a deliberate threshold breach in a test environment.
5. Reviewer quality. Override rates, time spent per review, and results of planted test items.

This evidence connects directly to AIRA's traceability domain. If you can reconstruct who decided what, and why, you can show oversight was real. Our AI agent audit log checklist covers what each record should hold.

How human oversight works in Avesta AgentOS

In Avesta AgentOS, human oversight runs through Relay, the control plane where AI co-workers and people work side by side. Take a refund agent:

1. The agent handles routine refunds within its agreed limit on its own, and every action is logged.
2. A refund above the limit becomes a review request in a human queue. The rule is set on the tool call itself, with conditions such as the amount.
3. The reviewer can approve it, reject it, or edit the amount before it runs.
4. If nobody answers in time, the request is rejected by default, or escalated to the next reviewer if the policy says so.
5. The activity record shows what the agent proposed, who decided, and when.

The team sees where every task stands in one place, whether an AI co-worker or a person is handling it.

What to ask your AI vendor

• Which actions are blocked until a person approves, and who can change that list?
• Can approval be triggered by a threshold, such as an amount, a volume or a confidence level, rather than for every action?
• Can a supervisor pause one agent, or every agent, immediately? Can the system pause itself when limits are crossed?
• When work hands over to a person, does the full context go with it?
• Do the logs show whether each action was taken by the AI, a person, or a person approving the AI?
• Can you show us reviewer metrics such as override rates and time per review?

If you're designing oversight for an AI agent, or need to show how your agents keep people in control, book a discovery call and we'll work through it with you.

Sources

• Digital Transformation Agency, AI Reference Architecture, Version 1.0, 2026 (CC BY 4.0), section "Human agency, oversight and control", available at digital.gov.au
• Digital Transformation Agency, AI Reference Architecture: A new blueprint to help government agencies design AI systems, 30 October 2026
• Digital Transformation Agency, Agentic AI addendum to the AI technical standard, June 2026, available at digital.gov.au
• Zhu, L., Lu, Q., Ding, M., Lee, S. and Wang, C. (2026). Designing meaningful human oversight in AI. AI and Ethics. Cited by AIRA.