From AI Principles to Practice: Implementing Australia’s Responsible AI Guidance with Avesta AgentOS

A practical approach to accountability, risk management, transparency, testing, monitoring, and human control across the AI lifecycle.

Australian organisations are moving beyond AI experiments and using AI in customer service, finance, operations, document processing and other important business functions.

As AI becomes more involved in everyday work, organisations need to know that it is operating safely, accurately and under appropriate human control.

Australia’s National Artificial Intelligence Centre(NAIC) has published the Guidance for AI Adoption: Implementation Guidance. It outlines six practices that help organisations develop and use AI responsibly.

Avesta AgentOS helps organisations turn many of these practices into practical technical controls, documented processes and measurable results.

1. Decide who is accountable

Every AI system should have a clearly identified owner. People need to know who is responsible for approving the system, monitoring its performance and responding when something goes wrong.

Avesta AgentOS supports accountability through:

• Role-based access for users and agents
• Defined permissions for agent tools
• Agent version records
• Activity logs and system traces
• Human approval and escalation workflows
• Implementation and operating documentation

These controls help organisations understand who can access an AI agent, what the agent is allowed to do and who has authority to approve changes.

The organisation remains accountable for how it uses AI. Avesta provides the technology and implementation support needed to put that accountability into practice.

2. Understand the possible impacts

Before introducing an AI agent, organisations should understand who may be affected and what could happen if the system produces an incorrect, unfair or inappropriate result.

Responsible AI starts with discovery

Avesta begins this process through our AI Kickoff service. It starts with a three-day AI Value Discovery Workshop, followed by a four-week pilot and a production plan.

During discovery, we work with the customer to define the use case, expected value, affected stakeholders, data requirements, possible impacts, human approval points and pilot acceptance criteria. We consider clear questions:

• What is the agent expected to do?
• What should it never do?
• Who could be affected by its outputs?
• Which decisions require human judgement?
• How can users report a concern?

This gives the organisation a clearer foundation for responsible implementation before an agent enters a live business process.

3. Measure and manage risk

Not every AI use case has the same level of risk. An agent that summarises internal documents presents different risks from an agent that influences lending, employment or financial decisions.

Following discovery, Safe AgentOps helps turn the pilot into a controlled production system. Safe AgentOps and Avesta AgentOS provide controls that can be configured according to the use case, including:

• Guardrails for agent behaviour
• Role-based access to data and tools
• Policy controls for agent actions
• Secure credential management
• Human approval and escalation
• Controlled access to enterprise knowledge
• Evaluation, monitoring and audit trails

Avesta helps customers assess risks, select appropriate controls and define what should happen if an agent behaves unexpectedly. These controls support audit and governance readiness, but they do not replace the organisation’s risk management and legal responsibilities.

Controls should be reviewed whenever the model, data, tools or business purpose changes.

4. Share essential information

People should know when they are interacting with AI. They should also understand what the system can do, what its limitations are and how they can ask for help.

Organisations also need an internal record of the AI systems they operate.

Avesta can help customers document:

• The purpose and owner of each agent
• The models, tools and data sources being used
• Agent capabilities and known limitations
• Testing and approval results
• Identified risks and controls
• Changes made to the system
• Human oversight and escalation processes

Agent traces, source visibility and audit records provide evidence for internal reviews and can support an organisation’s AI register.

Transparency should be appropriate for the audience. A customer needs a simple explanation, while a risk, technical or compliance team may require more detailed records.

5. Test and monitor

AI systems can produce different results as models, data and operating conditions change. Testing once before launch is not enough.

Safe AgentOps and Avesta AgentOS support testing and monitoring through:

• Evaluations based on realistic business scenarios
• Accuracy and relevance scoring
• Checks for suspected ungrounded responses
• Performance baselines and regression testing
• Error, response time, token and cost monitoring
• Agent tracing and behaviour analytics
• Version control for deployed agents

MetricSense: Agent Behaviour Analytics

MetricSense provides detailed analytics on how agents behave in production. It monitors configured indicators for accuracy, grounding and brand alignment, then surfaces insights that help teams improve agent responses and workflows.

Teams can use these insights to identify unexpected behaviour, investigate suspected ungrounded outputs, review the sources an agent used and track performance over time. This creates a continuous feedback loop between production monitoring, human review and agent improvement.

Before deployment, each agent should have clear acceptance criteria. The accountable person should review the results and approve the system for its intended use. Automated evaluation supports this review but does not replace human judgement.

6. Maintain human control

AI should support human decision making, not remove appropriate human authority.

Safe AgentOps and Avesta AgentOS allow organisations to define:

• Which tasks an agent can complete independently
• Which actions need human approval
• When an issue must be escalated
• Who can intervene or override an agent
• Which data and tools the agent can access
• What happens if the system becomes unavailable

For higher-risk activities, an agent may collect information, prepare a recommendation or identify an issue while a qualified person makes the final decision.

Human oversight should be meaningful. The person reviewing the work must have enough information, training and authority to question or reject the agent’s output.

Responsible AI requires shared responsibility

Technology alone cannot make an organisation compliant or responsible. Responsible AI also requires leadership, policies, stakeholder engagement, risk assessments, staff training and ongoing governance.

AI Kickoff helps identify the opportunity, affected stakeholders and possible impacts. Safe AgentOps converts those findings into policies, permissions, evaluations, approvals and monitoring. Avesta AgentOS provides the platform for operating and improving agents throughout their lifecycle.

The goal is not simply to deploy AI faster. It is to deploy AI in a way that earns trust and continues to deliver value.

Start with AI Kickoff

Our AI Kickoff helps your organisation select a suitable use case, understand potential impacts and define the controls required for a responsible pilot.

From there, Safe AgentOps and Avesta AgentOS provide a practical path from AI opportunity to controlled, trusted operation.

Reference

Australian Government, National Artificial Intelligence Centre, Guidance for AI Adoption: Implementation Guidance, October 2025.

This article provides general information and should not be considered legal or regulatory advice.