There’s no way to know what an AI agent is doing. Not in the sense of knowing exactly the steps the agent took to complete a task, or to reach a failing state. There’s no backtrace, no information about why something didn’t work. The only thing you get is a generic error message. And that doesn’t help if you’re trying to debug and improve your system. Fortunately, there’s a solution. It involves using OpenTelemetry (OTel) to record traces, metrics, and logs of your agent.
This article is brought to you with the help of our supporter, n8n.
n8n is the fastest way to plug AI into your own data. Build autonomous, multi-step agents, using any model, including self-hosted. Use over 400 integrations to build powerful AI-native workflows.
Traditional software doesn’t change while it’s running. Software written once and then deployed does what it was designed to do until you replace it. By contrast, you can’t guarantee that an AI agent will do the same thing in the same way twice. AI agents can make decisions based on live data, affecting the tools they choose to use. So, if a tool request returns something unexpected, it will negatively affect all subsequent choices. One, or many, of those following requests will certainly fail. But it’s not that failure that matters the most. What you want to know is which choice led to that failed request, and what data informed the decision. Traditional monitoring won’t give you that information. You can have all the health checks green and still see the AI agents tripping while using your API or MCP. Standard observability solutions won’t be of much help here, either. They’re focused on errors and aren’t good at detecting patterns, which is what you need to look for. Not knowing what’s making AI agents fail has a real cost, too. API owners will see traffic spikes whenever AI agents enter endless retry loops. And end users will also see token usage rise as a result of all the increased chatter. On top of that, not knowing why things are failing keeps your SRE team busy and unavailable to work on other difficulties.
The solution, until now, has been to go through chat transcripts and tool request logs, trying to find a cause for the bad choice that led to a failure. Some teams have even been trying to repurpose traditional APM tools. As the frustration grew, vendor-specific observability products like LangSmith, Arize AI, and Helicone entered the market. They can help solve the problem, but they’re not the long-term solution we’re looking for. Each one of them uses its own proprietary data format. That’s why. Instead, what we need is to follow the standard observability pattern of traces, metrics, and logs. This is what frameworks like OTel follow, so it’s a proven way of working. How does it work, then? With traces, you can map an end-to-end chain of tool executions in chronological order. You get to see every request and its response in sequence, rather than a list of isolated events. With metrics, you can aggregate what happens in individual executions and obtain long-term trends. You can track token consumption, latency between workflow steps, cost per task, and the rate of failure across thousands of executions over time. Finally, with logs, you can inspect the exact payload at a specific point in time. These three things, together, make a powerful solution to understand the reason behind the failures.
Why Standard Observability Matters
Relying on custom SDKs ties your telemetry directly to a single platform and its dashboard format. The moment you want to switch tools or analyze data elsewhere, you end up rewriting your instrumentation from scratch. Standard observability avoids that lock-in by separating how you collect telemetry from where you store and visualize it.
Making the solution work isn’t necessarily complicated. But it needs a working hierarchical span tree set up at execution start. Wait, a span tree? What does that even mean? A span, in OTel, is the single unit of work. It represents a tool call, for example, with a start time, duration, any attached attributes, and a status. A span lets you know when the call happened, how long it took, what it was about, and if there was an error. When an AI agent starts a task, OTel opens a root span that covers the entire execution lifecycle. At that point, OTel generates a trace ID that will be used across all subsequent actions. This is how those actions, also known as child spans, are attached to the root. And that’s why this span representation is called a tree. With this approach, you can identify a failure, understand all steps that led to it, and what the initial action was. Here’s an example of what a span tree looks like:
trace_id: 8f2c1e... (root: agent.execution)
├── span: llm.call (model: gpt-4o, status: ok, 310ms)
├── span: tool.call (name: validate_email_address, status: ok, 95ms)
├── span: tool.call (name: add_subscriber_to_list, status: ok, 240ms)
├── span: llm.call (model: gpt-4o, status: ok, 620ms)
└── span: tool.call (name: send_confirmation_email, status: error [502 Bad Gateway], 50ms)It’s important that the information you’re capturing follows open standards and isn’t attached to any proprietary format or convention. So, following OTel’s Generative AI Semantic Conventions is mandatory. They “establish standardized guidelines for how telemetry data is structured and collected across platforms, defining inputs, outputs, and operational details.”1 GenAI spans, in particular, “represent logical operations as observed by the caller.”2 The advantages of following these conventions are that you don’t have to reinvent everything yourself and, at the same time, you can use any available open-source compatible tool. In August 2026, instrumentation tooling was available for Agno, Anthropic, LangChain, OpenAI, Qwen, smolagents, and Google. Instrumentation for Claude Agent, CrewAI, and others was still being developed. If your code is using one of these agents’ SDKs, all you have to do is configure the appropriate instrumentation. The alternative, in case you don’t want to get your hands dirty, is to use a commercial solution. If you’re using a modern workflow service that talks to AI agents, check its feature list. As the technology matures, more platforms are offering OTel-compatible traces out of the box.
Keep in mind that OTel GenAI Semantic Conventions aren’t fully mature yet, so I’d expect substantial changes. While commercial solutions can protect you against those changes, they can’t guarantee everything will always work between SDKs, platforms, and AI agents. It all depends on how quickly each solution adapts to changes. One SDK might comply with the latest version of the conventions, and the tracing solution you’re using might fall behind, creating potential incompatibilities. On top of that, if you use an open-source solution, you also have to think about the overhead of maintaining the telemetry backend yourself. But even if everything is working, capturing everything that happens with an AI agent is expensive. All those system prompts, LLM responses, and frequent tool responses consume a lot of storage and bandwidth. So, think twice before you blindly follow the advice of one of the many available tutorials. Most of them are pure marketing material aimed at selling you a specific solution. Always be skeptical of what you read (this article included).
Being skeptical means you should evaluate all options before deciding. You can continue to follow ad-hoc debugging techniques that you know still work, or you can invest now in building proper observability for your AI agent. With observability, you’ll be able to understand what happened during an agent’s execution of a task. This is just the beginning. The next step would be to evaluate the quality of the execution and how well tasks are completed. But to get there, you need to start with observability so you can analyze execution traces. Yes, putting all this together requires time and effort. But using the same manual tools also has its cost when serious production failures happen. Whether you need full OpenTelemetry tracing today depends on how critical your AI agent is to your operations. The decision is all yours to make.
Thanks for reading The API Changelog!
Please consider subscribing to receive new posts and support our work.
Not convinced yet? Get to know why we’re considered “the most important API newsletter out there.”
OpenTelemetry for Generative AI, Robbins et al, December 2024
Semantic conventions for generative client AI spans, GitHub repository, August 2026
