How Boards Can Review AI Decisions With the Right Evidence

Logs alone prove nothing; boards need evidence of who approved what.

Publication desk · · 10 min read · Updated
Cover illustration for “Evidence Reconstruction Protocols for Board-Level AI Decision Reviews”
Audit-Ready AI Workflows · October 15, 2026 · 10 min read · 2,209 words

When a regulator looks into an AI decision, they ask one simple question. What evidence shows that someone approved this action? And what did that evidence say at the time? Boards that oversee AI systems often cannot answer that question. This is true even when every logging system worked perfectly.

The reason is what researchers call the container fallacy. This means assuming that because a log exists, oversight happened. A log is just a container. What matters is whether that container holds specific answers to specific questions about specific decisions. Three real incidents show why this matters. In the Replit DROP DATABASE event of July 2025, the Cursor destructive-shell-command incident of June 2025, and the Claude Code DataTalks.Club destructive-mutation event of February 2026, every destructive action was recorded somewhere. But none of the records answered the question an auditor would actually ask. Who approved this action, and under what rules? That is not a technology problem waiting for better tools. It is a gap between what logs produce and what governance questions need.

The pressure on boards to fix this is already real. Among Fortune 100 companies, nearly half now disclose AI as part of board-level risk oversight. That is almost triple the share in 2024. SEC Regulation S-K Item 407(h) requires companies to disclose how the board oversees risk, including AI risk. Shareholder legal claims under the Caremark line are already testing whether the oversight boards described on paper actually happened. That is pushing boards toward systems that can rebuild decision history on demand.

Why the evidence gap is worse for some AI systems than others

Not all AI systems have the same evidence gap. It gets structurally worse as systems move from simple rules toward agentic AI systems. Agentic AI is AI that takes a series of actions on its own. It uses tools, talks to other AI systems, and completes multi-step tasks without a human approving each step. The N4 governance evidence chain (a research framework for linking AI decisions to auditable evidence) maps this across four types of decision systems.

Even a simple hybrid system that combines rules with a machine-learning model is partly opaque. The rule part is traceable. But the model's decision boundaries were learned, not written down. After-the-fact explanations of what the model did are approximations, not records.

Agentic AI systems are a different problem entirely. The synthesis paper names three structural breaks. First, decision diffusion. One significant outcome comes from a chain of smaller decisions spread across multiple AI agents. No single record captures the moment a consequential choice was made. Second, evidence fragmentation. Each agent may log its own actions, but nothing puts those logs together into a clear answer to a governance question. Third, responsibility ambiguity. When multiple agents with overlapping roles produce a bad outcome, no named person is clearly responsible for it. Governance failures at one layer flow through to the next. A monitoring system with bad inputs cannot detect when things are going wrong.

Five things any evidence record must be able to prove about an AI decision

Sufficient evidence is not about volume or file format. It is about whether five specific things can be shown for any governed decision, on demand. These five properties stay fixed regardless of which AI system made the decision. The Decision Evidence Maturity Model, or DEMM (a research framework for measuring how well organisations can rebuild AI decisions), defines them through a formal structure called the Decision Event Schema.

Actor: which AI agent, model, or person started or approved each step. Missing actor evidence means you have a record of a harmful action with no identity attached to whoever took it. Authority: under what rule or permission the action was taken. Action: exactly what was done, including tool calls, data reads, record changes, and external communications. Outcome: the measured result, including failed, inconclusive, and reversed results, not just successful ones. Rationale: why that action was chosen, based on the evidence available at the time of the decision.

The DEMM feasibility exercise across 140 synthetic scenarios and three named-incident reconstructions showed that systems relying only on container presence overclaimed how complete their evidence actually was. Container presence means checking whether logs exist, nothing more. That confirms a structural problem, not just a configuration gap. Logs look sufficient until someone asks a real question about a real decision.

Why the approval question is the hardest one to answer in AI workflows

Diagram: The Four Tiers of Agentic Action and What Each Demands as Evidence. Visualizes: Visualise a ranked four-tier authority model showing escalating evidence requirements for AI agent actions.

Of all five properties, the authority question fails most often in agentic systems. The problem is not that organisations forget to log what the AI did. It is that the permission system for agentic workflows rarely records who approved each specific action at the moment it happened. Good oversight requires that decisions and actions stay reconstructible and verifiable throughout and after execution. But that needs the right architecture from the start, not a configuration change after the fact.

A practical way to structure this is a tiered approval model. The evidence requirement goes up with the seriousness of the action. Low-risk actions such as search, read, summarise, and draft are allowed and logged. Medium-risk actions such as creating or editing records that can be undone are allowed, logged, and rate-limited. High-risk actions such as sending external communications, deleting data, or exporting data require a human to approve them out of band. Out of band means separately from the AI workflow. The evidence record must capture that approval directly, not just the completed action. The highest-risk actions — such as payments, permission changes, bulk deletions, and production deployments — require approval plus a step-up identity check. The evidence record must capture both the decision to approve and the identity of the person who approved it. This must sit in a tamper-evident record that cannot be altered after the fact.

Long-running AI agents create an extra problem. Research on authority-preserving edge revocation for federated AI-agent workflows finds that when a business purpose, owner, or platform changes, the agent's permissions must be revoked. The agent must be disabled, its connections removed, and the audit record preserved. The difference between approving before an action versus monitoring after it matters for how you design evidence. Some workflows need a human to approve before the AI acts, producing a pre-action record. Others allow the AI to act and flag for human review afterward. A board must say which pattern applies to each workflow before deployment. Permissions must be checked on every tool call, not once at the start of a session. Data must be filtered against each user's permissions before it enters the AI's context. Both requirements produce evidence that is either present or absent in the record, making them directly testable at review.

Why checking logs after the fact is no longer enough for high-risk AI

For high-risk AI systems, reviewing logs after something has happened is not enough. Policy enforcement and audit evidence must be produced at the same moment, as the AI acts. The N4 governance evidence chain describes four components — diagnostic theory, trace schema, sufficiency measurement, and label-free monitoring — as a layered architecture. An AI agent control plane is the enforcement layer that sits between autonomous agents and the systems they can affect. It issues and revokes identity for every agent. It checks permissions on every tool call. It keeps an audit trail that can answer what an agent did and who approved it. The governance setup must include demonstrated pause-and-revoke controls, not just a policy that says they exist.

Continuous audit is becoming a separate regulatory expectation. The 2025 PCAOB inspection priorities (the body that oversees public company auditing in the US) flagged audit areas with increased use of technology including AI as a focus area. Regulators, accreditors, and auditors increasingly expect boards to show ongoing oversight, not just pass an annual review. That means the evidence infrastructure must be permanently active, not switched on for review season. The monitoring requirement is especially demanding for agentic systems. The monitoring layer must detect governance problems without waiting for outcome data. The results of agentic decisions may not be visible for days, or may be impossible to reverse before they can be observed.

Five levels boards can use to honestly rate their current evidence setup

Boards do not need to build a new governance framework from scratch. You need an honest assessment of what your current evidence setup can and cannot rebuild. Measure it against a fixed sufficiency standard, not against what peers claim or what internal teams self-report. The DEMM five-level capability rubric is built on the established CMM/CMMI, TMMi, and DCAM maturity-model lineage (well-established frameworks for rating organisational capability). It combines per-property sufficiency verdicts into an overall capability level. This gives a board a structured way to compare what its governance is claimed to produce against what a protocol-level reconstruction exercise actually recovers.

The protocol puts each evidence property into one of four categories. Sufficient means the property can be rebuilt with confidence from saved evidence, without guessing. Partial means some evidence exists but gaps remain. Absent means no usable evidence exists. Conflicting means evidence exists but contradicts itself. When evidence conflicts, reconstruction is unreliable no matter how much of it there is. See the full category definitions.

A board doing an honest assessment should be able to answer, per workflow and per decision type, which properties are sufficient, which are partial, and which are absent. The feasibility exercise showed that agentic systems averaged well below deterministic rule-based systems on reconstruction completeness. A board that hands AI oversight entirely to technical teams cannot claim the level of engagement the literature requires.

What makes a decision record useful for board oversight rather than just a log of what happened

A decision record that supports meaningful board oversight is not a system log with a governance label on it. It is a structured document. It connects evidence, human reasoning, approvals, actions, and measured outcomes. An external party must be able to ask governance questions about it, including questions about failed and inconclusive results. Without that structure, a log is just a container whose usefulness is assumed rather than checked.

A board-ready decision record must capture several things. It must capture the evidence the system acted on at the moment of decision. That is not a reconstruction of what was available. It is a timestamped record of what entered the AI's context. It must capture the human reasoning, corrections, and exceptions that were applied. This includes why an exception was made and who made it, not just that the outcome differed from the AI's recommendation. It must capture the approval that came before any high-risk or critical action. That must be an attributable record, not an inferred permission. Tamper-evidence and the ability to replay the decision are not optional. A governance setup that leaves this data in a vendor's control rather than your own environment creates a structural dependency. That dependency may not survive a vendor change or a regulatory demand for production.

Six questions a board must be able to answer to claim real AI oversight

A board that cannot answer six specific questions about any high-consequence AI workflow has not yet established meaningful oversight. This is true regardless of what its governance disclosures say. These questions come from the property schema and the governance evidence chain. They are not reserved for an annual review. They are the standing test of whether your oversight claim holds up on any given day.

First, who is the named person accountable for this workflow's permission boundaries, monitoring, and the decision to retrain or retire the system? Is that documented and accessible to auditors? Second, for any high-risk or critical action this system has taken, can you produce a tamper-evident record showing which named individual approved it before it was run? Third, which evidence properties of your current setup are sufficient, which are partial, and which are absent, per workflow and per decision type? Who owns closing the gaps? Fourth, is your governance layer producing policy enforcement and audit evidence at the same time as the AI acts? Or are you reviewing logs after the fact? If the latter, does that satisfy the requirements that apply to your high-risk AI systems? Fifth, can you demonstrate — not just describe — that your pause-and-revoke controls work, including cascading revocation for every agent in the workflow? Sixth, are your decision records kept at the decision level for the periods each applicable rule requires? Are they in an environment your organisation controls, with tamper-evidence and the ability to replay decisions?

The systematic review names the paradox of diligence as a construct in the board-AI governance literature. The answer is engaged involvement, not delegation. Where an enterprise AI system is being assessed or deployed, boards should require that the system's architecture produces decision records at the decision level. It should keep read and write permissions separate. It should maintain configurable approval gates and audit logs. It should keep the organisation's context and outcome history in an environment the organisation controls. These are structural properties of a governance-capable system, not features to be added later. Claiming oversight and being able to show it are not the same act. The distance between them is a protocol, and it can be run.

Sources

  1. Boards in the Age of AI: Antecedents, Decisions, and Outcomes of Board-Level Artificial Intelligence Governance—A Systematic Review
  2. Decision Evidence Maturity Model for Agentic AI: A Property-Level Method Specification
  3. Governed Auditable Decisioning Under Uncertainty: Synthesis and Agentic Extension
  4. Decision Trace Schema for Governance Evidence in Real-Time Risk Systems

More in Audit-Ready AI Workflows