Can You Reconstruct an AI Decision? AI Traceability Explained

An auditor asks how your AI reached an outcome three months ago. You have the chat log. You don't have the documents it retrieved, the prompt version it ran, or the name of the person who approved it.

That gap is what Australia's AI Reference Architecture (AIRA) asks you to close. If you can't reconstruct an AI decision, you can't defend it. This fourth post in our AIRA series covers what to record, how to version and roll back, and how long to keep it all.

What is the difference between AI observability and traceability?

Observability is the ability to understand what an AI system is doing right now from its logs, metrics and traces. Traceability is the ability to verify, later, where its data, models and outputs came from and what happened to them. You need observability to catch problems early, and traceability to explain and defend outcomes afterwards.

AIRA treats them as two separate trustworthiness domains:

• Observability covers collecting and analysing telemetry across the whole system, so teams can spot drift, unintended consequences and new failure modes, and step in before issues escalate.
• Traceability covers the provenance, ownership, processing history and outputs of models, data and pipelines across their full lifecycle. That includes who or what made each decision, whether a person or the system.

A simple way to remember it: observability answers "Is something going wrong?" Traceability answers "What exactly happened in case 4,512, and who was responsible?"

Observability: is something going wrong? Traceability: what happened in this case, and who was responsible?

What does a complete AI decision record contain?

A complete AI decision record lets someone who wasn't there reconstruct exactly what the AI saw, what it did, and who was accountable. At minimum it links the request, the context and sources used, the versions of everything that ran, the output, any actions taken, and the human decision.

Our AI agent audit log checklist shows a filled-in example of one record and which events need one.

How do you make AI decisions contestable?

An AI-supported decision is contestable when the affected person knows AI was involved, can understand the main reasons for the outcome, and has a real way to challenge it, reviewed by a person who can change it. Each of those needs a design choice, not just a policy statement.

AIRA links this to the experience layer, the traceability domain and human oversight. In practice:

• Tell people. Say when AI contributed to an outcome, in plain language, at the point they receive it.
• Give reasons they can act on. Show the main factors and sources behind the outcome, not the model's internals.
• Offer a clear path to challenge. Make it obvious how to ask for review, and who will do it.
• Give the reviewer everything. The person reviewing a challenge needs the full decision record, not just the final letter. Our guide to keeping humans in control of AI agents covers who that reviewer should be and what authority they need.
• Close the loop. Feed upheld challenges back into evaluations, so the same error doesn't repeat.

For low-risk internal tools this can be very light. For decisions about entitlements, compliance or enforcement, it is central to legal defensibility and procedural fairness. Our post on applying AIRA by risk shows how the depth changes by tier.

Why do you need version control for prompts and agents, not just code?

In an AI system, the prompt, the agent's instructions, its tools, its knowledge sources and the model all change behaviour as much as code does, and they often change more often. If they aren't versioned, you can't tell which version made a past decision, and you can't safely undo a bad change.

AIRA's operational guidance expects repeatable pipelines for deploying and changing models, prompts, configurations and code, not code alone. Things to version:

• Prompts and agent instructions. A one-line wording change can shift outcomes across thousands of cases.
• Model and provider. Including the exact model version, because providers update models behind the same name.
• Tools and permissions. What the agent was allowed to do at the time.
• Knowledge sources. Which version of a policy document the agent retrieved.
• Business rules and thresholds. For example, the approval limit in force that day.

The test is simple. Pick any decision from last quarter. Can you say exactly which version of each of these produced it?

Pick any AI decision from last quarter. If you can't name the prompt version that made it, you can't defend it.

What is rollback, and why does AIRA call it out?

Rollback is the ability to return an AI system to a previous known good state when something goes wrong. AIRA's traceability domain specifically includes the ability to reliably roll back models, data, configurations and decision logic when errors, drift or harm are detected.

Rollback matters more for AI than for ordinary software, for two reasons:

• Problems surface slowly. A prompt change might look fine in testing and only show its effect weeks later, across thousands of cases.
• Several things change at once. A model update, a new knowledge document and a changed threshold can all affect the same outcome.

Good rollback design means:

• Every component that affects behaviour is versioned, as above
• Previous versions stay deployable, not just archived
• You can roll back one agent or one use case without touching the others
• You can identify which past cases were affected by a bad version, so they can be reviewed

AIRA also notes that anomaly-driven rollback can run automatically, without per-action approval, as long as its triggers were authorised in advance. Our post on AI agent performance degradation covers when to roll back first and investigate second.

How long should you keep AI records?

Keep AI records for as long as the decision they support must be kept, and no longer than you need the personal information in them. For Australian Government agencies, AI outputs that form evidence of decisions, actions or communications are Commonwealth records under the Archives Act 1983. The agency's records authorities set the retention period, not the AI vendor.

This creates a tension that needs a deliberate design:

• Recordkeeping pushes you to keep more. The decision, its basis and who approved it must survive for the retention period.
• Privacy pushes you to keep less. AIRA's privacy domain warns about excessive retention, and logs and prompts often contain personal information.

Practical ways through it:

• Separate the decision record, kept for the retention period, from raw debugging logs, kept for a short operational window.
• Mask or minimise personal information in operational logs where the decision record doesn't need it. Our guide to redacting PII before the LLM shows how.
• Store logs, embeddings and traces where your data residency rules require. They count as data too.
• Agree retention and disposal with your records manager and privacy officer before go-live, not after.

Private-sector organisations face the same tension under the Privacy Act and their industry's record rules.

Keep the decision record for as long as the law needs it. Keep the raw logs only as long as engineers do.

What should an AI observability dashboard show?

An AI observability dashboard should show whether the AI is doing its job well, safely and affordably, not just whether servers are up. That means quality, behaviour, human involvement and cost, alongside the usual technical health.

Page views and response times are not enough for AI. The signals that matter are:

• Quality: accuracy against sampled reviews, hallucination or error rates, and how often users rephrase or retry
• Behaviour: unusual tool use, actions near limits, and changes in the topics or cases the AI is handling
• Human involvement: escalation rates, override rates, time to review and ageing approval queues
• Drift: whether any of the above moves after a model, prompt or data change
• Cost: usage and spend per use case, so cost surprises are caught early
• Alerts: clear thresholds that notify a named person, not just a chart someone might look at

Each metric should drill down to the individual traced case. A dashboard that shows a spike but can't open the cases behind it tells you something is wrong, but not what. Our agent analytics guide goes deeper on what to monitor in production.

How we trace and version agents in Avesta AgentOS

In Avesta AgentOS, the platform we build and run AI co-workers on, we keep two separate records: a trace for engineers, and an audit log for reviewers. The trace is the recording. The audit log is the minutes.

The trace: every step, saved as it happens

• A trace is opened before the model is first called, noting which version of the agent is running. When the request finishes, it is updated with totals: tokens, cost, the timing of each tool call and the number of steps.
• Each step is saved as soon as it finishes. It holds the tool calls and their results, the full input and output, timings and tokens. The input includes the exact instructions the model saw, with any saved memories added, so you can tell whether either changed.
• Failed steps are saved too, with the input that was sent, any partial output and the error message. If a run fails halfway, its trace still shows every step before the failure.
• Tracing is a side task. If saving a step fails, the agent carries on. Tracing should never be the reason an agent stops.
• Because traces hold the full text, each one gets the same protection as the conversation it came from. Tools return only the fields a task needs, which keeps unneeded personal details out of the model and out of the trace.

The audit log: short records a reviewer can read

• Agent change records store each changed setting, its old value, its new value, who changed it and when.
• Review requests open when an approval rule matches an agent's action. Each request records the agent, the tool, who asked, the rule and what the agent was trying to do.
• Decision records hold who decided, the outcome (approved, rejected, modified or cancelled), the reason and the time. Each decision is its own record and is locked as soon as it is created. If nobody answers, an expired or escalated record is written, so a reviewer can see that nobody said yes.

The same principle runs through the products we build for customers. In Life Insurance Direct's call quality review, Metricsense flags calls for review and a person makes every decision.

Versions and rollback: one switch, per channel

• Everything that shapes behaviour is versioned together. Each agent version holds its prompts, tools and configuration as one package, so there is never a question of which prompt ran with which tool.
• Different channels can run different versions. For example, the latest version can run on WhatsApp while email and the website widget stay on the previous one. New behaviour is proven on one channel before it reaches the others.
• Rollback means moving back to the previous version. There's no rebuild and no emergency fix: the channel returns to the last known good version.
• Every trace and audit record shows the version and the channel. For any decision, you can see exactly which version produced it and where it came from. That also lets you find every case a bad version touched.

What to ask your AI vendor

• Can you export the complete record for a single decision, including the sources retrieved and every version that ran?
• Are prompts, agent instructions, tools and knowledge sources versioned, and is the version recorded on each run?
• How quickly can you roll back a bad change, and can you list the cases it affected?
• Where are logs, traces and embeddings stored, and for how long?
• Can we set different retention for decision records and operational logs?
• What AI quality signals does your dashboard show, beyond uptime and latency?

If you need to show how an AI system's decisions can be reconstructed, or want to check whether yours can be, book a discovery call and we'll work through it with you.

Sources

• Digital Transformation Agency, AI Reference Architecture, Version 1.0, 2026 (CC BY 4.0), sections "Observability", "Traceability", "Explainability and interpretability" and "Privacy", available at digital.gov.au
• ETSI TR 104 032 (2024), Securing Artificial Intelligence: Traceability of AI Models. Cited by AIRA.
• NIST (2023), AI Risk Management Framework. Cited by AIRA.
• Chen, Y. et al. (Anthropic), Reasoning Models Don't Always Say What They Think, 2025
• National Archives of Australia, guidance on AI and Commonwealth records, available at naa.gov.au