What IT Should Be Able to See When AI Agents Are Running
Four kinds of telemetry let IT audit what agents actually did.

Agentic workflow automation for enterprise needs more than proof that a request succeeded. IT must be able to trace an agent’s decisions, tool calls, data access, cross-system actions, and human approvals, then pause the agent or revoke access when something goes wrong. The strongest governance model joins that evidence into a record that can be audited to show both whether the agent finished and whether it did the right thing.
Why traditional monitoring cannot tell IT whether an AI agent did the right thing
A 200 response can hide a wrong tool call or a hallucinated answer. The request succeeded, but the work was wrong. Traditional application performance monitoring records request rates, latency, and span counts, which help IT tell whether a service is running. Those signals do not show whether an agent chose the right tool, gave it valid arguments, or stuck to its original plan.
That gap starts with how agents choose their next step. The same prompt can lead to different tool calls on different runs because execution branches on model output. Silent failures are the distinctive risk here. A wrong retrieval, a retry loop, or a fabricated policy passed into a downstream workflow need not produce an HTTP error or an exception, leaving IT with a technically successful request and no clear view of the business failure.
How do AI agents work across enterprise systems, and what must monitoring capture?
A production agent is an orchestrated sequence of reasoning steps, tool calls, data retrievals, state changes, and handoffs. Monitoring must capture that execution graph, including the final response. Each step can introduce a different failure. A tool call can use the wrong arguments, a retrieval can return the wrong entity, and a handoff can lose the context that made an earlier decision valid.
Tool records need the invocation’s name, arguments, raw output, duration, retry count, and error state. Without those details, IT cannot distinguish a useful call from one built on invented arguments or caught in repeated retries. State transitions explain a different part of the run. Working memory before and after a step, context edits, and handoff payloads expose where information was lost or changed as a longer task progressed.
Long-term memory needs its own record of reads and writes, including the query, returned entries, relevance scores, and freshness. Those details expose stale reads, wrong-entity retrieval, and memory leakage between users. Multi-agent workflows add another boundary to the record. IT must know which agent or sub-agent handled each step, where coordination failed, and how that affected downstream decisions. Kore.ai describes how small changes to a prompt can lead to different decision trees and how tool orchestration failures can cause errors in finances or operations without a logged exception.
The four telemetry types that together give IT an auditable record of agent behavior
IT needs four telemetry types working together, namely end-to-end traces, metrics, logs with governed context, and decision traces with approval records. Each answers a different operational question. End-to-end traces connect user intent to planner decisions, routing logic, tool calls and results, data assets accessed, and the final output. Nested spans keep parent-child relationships across multi-agent handoffs, so a downstream action stays linked to the steps that led to it.
Metrics show whether behavior is degrading across runs. They include latency per run and sub-task, token consumption, compute cost, success rate, and hallucination rate by domain and query type. For a run to count as a success, it must finish without errors, hallucinations, or policy violations. Tracking these measures continuously gives IT a way to detect deterioration before it becomes an incident, rather than waiting for a user to report a bad outcome.
Logs with governed context explain what an event meant. A raw record that an agent accessed data is incomplete without the asset’s identity, its owner, the policy governing its use, and whether the dataset was current. Atlan contrasts “what happened” with “what it meant. That distinction is practical, not cosmetic. The same access event may look normal on its own, but a link to a deprecated dataset or a restricted column can show a governance problem.
Decision traces and approval records connect the evidence the agent used, its recorded reasoning path, human review, the action taken, and the result, including failed attempts. They provide the artifact needed for audit and forensics. Isn't this just more logging overhead? Instrumentation does have a cost, but IT needs all four types to reliably tell a correct run from one that finished but caused harm, and wrong changes to records or leaked data are harder to undo.
Why cross-system actions raise the stakes for every gap in that record
An agent working across HR, finance, sales, and support can turn a gap in one system’s record into a consequence in another. No single team necessarily sees both sides. An update may succeed in the destination system even though it relied on stale information from the source. Without a connected trace, neither system’s owners can diagnose that error independently, because each sees only part of the chain.
Atlan identifies the lack of an auditable record across systems, APIs, databases, and tools unless observability is explicitly instrumented across those boundaries. A trace also needs a connection to data lineage and policy. Otherwise, it can show that a query ran without revealing that the dataset was deprecated or the column was outside the agent’s permissions. The missing context changes the diagnosis entirely. What looks like a normal data read may be the point where the workflow crossed a policy boundary.
The Dynatrace 2026 report says that as AI gains more authority to make decisions, the need for validation, integration, and oversight grows. Leaders need to see what agents do and what happens as a result before deciding where to give them more autonomy. Shadow agents make that decision especially difficult. Deployed by individual teams without central registration or monitoring, they can access production data, make consequential decisions, and accumulate costs without a record IT or finance can inspect.
The governance controls IT needs on top of observability data
To close that gap, IT needs a central agent inventory before it can govern what is running. The register must show which systems each agent can access, what it is allowed to do, and who owns its approval. Without it, monitoring has no reliable scope. The inventory also shows which agent operational controls should apply to, linking suspicious events to the agent under review and the permissions and authority it was given.
Reading and changing must be separate permissions. An agent that reads across systems to surface a finding should not automatically gain write access to act on it. The approval between observation and action is itself evidence and must be logged. Validation also needs more than one check. Dynatrace’s research found that organizations use nearly four validation methods on average and that accurate source-data validation is the most widely used.
Human reviewers cannot check every trace across an enterprise. Automated anomaly detection must surface failing runs, guardrail violations, timeout spikes, and quality drift for review. Someone must also have authority to intervene. Pause controls need to stop an agent mid-run, while revoke controls need to rescind its access to a system. An agent that cannot be stopped is not under effective operational control, however detailed its telemetry may be.
Human intervention must leave evidence as well. Every approval, rejection, and override belongs in the decision trace, so the human layer is as auditable as the agent layer. Kore.ai describes traces enriched with model versioning, policy tagging, and access control metadata as part of a governed audit trail. Those details connect a decision to the model and rules in effect, rather than leaving reviewers to interpret it without its operating context.
What OpenTelemetry standardization means for teams building this capability now
The practical instrumentation choice is to select the telemetry standard before selecting the platform. OpenTelemetry has a vendor-neutral format for recording what agents do. Its GenAI conventions cover agent creation and invocation, planning, workflow execution, and tool execution, with guardrail outcomes and policy decision attributes covering governance events. A common format lets the enterprise send that telemetry to the observability platform it chooses without tying the instrumentation to that platform alone.
For teams invested in Datadog or Dynatrace, the test is whether LLM and agent observability connects to the same telemetry pipeline or remains separate. Separate traces leave a familiar blind spot. If an agent trace cannot be correlated with infrastructure signals, IT still lacks the connected view needed to investigate failures that cross systems.
The standard therefore comes before the platform decision. Choosing in the reverse order risks building the same lock-in that an open standard is meant to avoid. The Dynatrace report identifies open standards as a way to maintain telemetry continuity across environments and lifecycle stages. Agents, models, and platforms will change faster than governance requirements, so the instrumentation needs to outlast any one tool.
How cross-system AI, human approval, and audit trails work together in practice
An agent supporting accounts payable, accounts receivable, or month-end close may identify an invoice above contract, flag a duplicate payment, or surface a renewal priced above market. Finding the issue is only the read stage. Before a correction is made, the business owner needs to see the evidence assembled, the proposed action, the systems it would affect, and who is accountable for the result.
A platform connecting those systems fits this model when it separates read access from write access and gates the transition on logged human approval. It can surface a finding without treating that finding as permission to change records. The evidence reviewed, the approval given, and the resulting action need to form one connected record, rather than separate histories that IT must reconcile after the fact.
The Decision Trace is the artifact that joins the chain. It records the evidence assembled, the reasoning applied, the approval received, the action taken, and the result, including failed attempts. A CFO, compliance officer, or external auditor can then check whether the action stayed within policy. A record of success alone is insufficient. Failed attempts belong in the same history because they are part of what the agent actually did.
Agents that run in an enterprise’s own cloud environment with separate read and write permissions, role-based approvals, an agent list, and the ability to pause them or revoke access are governed differently from agents with broad permissions on a shared platform. Observability cannot substitute for those boundaries. The final standard is not whether an agent completed the task, but whether the enterprise can prove it used the right data, acted with the right approval, and left a record that survives audit.
This article was commissioned by and produced with AI assistance on behalf of its publisher.


