Observability for Agents: Tracing Tool Calls and Reasoning

The single biggest lie you can tell yourself about an agent is that it is a single API call. It is a loop: model thinks → picks a tool → tool executes → result feeds back → model thinks again. When that loop returns a wrong answer, the proximate blame lands on “the model,” but the actual defect usually lives in the action layer — a tool that timed out, an argument the model generated badly, or a business error disguised as a successful RPC. — “Run AI like production infrastructure, not like an experiment. Trace workflows, log every call, and keep the system observable at all times.”

Logs of ping-pong can’t answer “why did the agent pick that tool, with those arguments?” You need traces, and the Model Context Protocol gives the tool boundary a stable, standard shape — spans that cross process and language boundaries via a shared trace_id.

Prerequisites

• Python 3.10+, pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp.
• An OTLP endpoint (Jaeger/Grafana Tempo, or local http://localhost:4318/v1/traces).
• MCP standardizes tools/call; OpenTelemetry semantic conventions for agents define span kinds (AGENT, LLM, TOOL) and recommend capturing tool name + arguments + result with trace context propagated across the client/server boundary.

Step 1: A tracer and an AGENT root span

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)
tracer = trace.get_tracer("agent-runtime")

Wrap the whole run in a root span so every tool and model call nests underneath (one trace = one agent run).

Step 2 — Wrap each LLM reasoning round

def reason(agent, content, response):
    with tracer.start_as_current_span("llm.round") as span:
        span.set_attribute("gen_ai.operation.name", "generate")
        span.set_attribute("gen_ai.prompt", content)
        span.set_attribute("gen_ai.response", response)
        span.set_attribute("gen_ai.usage.total_tokens", response.usage_metadata.total_token_count)
        return response

Notice we attach token usage to the span — this is where cost and observability meet.

Step 3 — Wrap tool calls as TOOL spans

The tool span answers the questions logs cannot. tools/call <name> {args} spans make each execution a first-class, searchable event.

import json

def call_mcp_tool(name, args, call_id):
    with tracer.start_as_current_span(f"tools/call {name}") as span:
        span.set_attribute("mcp.method.name", "tools/call")
        span.set_attribute("gen_ai.operation.name", "execute_tool")
        span.set_attribute("tool_name", name)
        span.set_attribute("tool_call_id", call_id)
        span.set_attribute("gen_ai.tool.call.arguments", json.dumps(args))
        try:
            result = execute_mcp(name, args)   # your MCP client round trip
            span.set_attribute("gen_ai.tool.call.result", truncate(json.dumps(result)))
            return result
        except Exception as e:
            span.record_exception(e)
            span.set_status(trace.Status(trace.StatusCode.ERROR))
            raise

Capture arguments and result intentionally — payloads are sensitive, so only opt-in when the debugging value outweighs retention risk.

Putting It All Together — the agent loop

def run_agent(prompt):
    state = prompt
    with tracer.start_as_current_span("agent.run") as root:
        for round_no in range(MAX_ROUNDS):
            resp = client.models.generate_content(model=MODEL, contents=state)
            state = f"{state}\n{resp.text}"
            reason(agent, state, resp)
            action = choose_tool(resp)          # parse structured tool call
            if not action:
                return resp.text                # no tool -> answer ready
            result = call_mcp_tool(action.name, action.args, action.call_id)
            state += f"\nTool={action.name} -> {result}"
    raise RuntimeError("no answer within rounds")

Why this matters for debugging

• Attribution — distinguish bad reasoning (LLM span high latency/rounds) from a slow dependency (TOOL span) from a tool that returned an error the model misread (TOOL Status ERROR).
• Cross-service — MCP propagates trace context, so the MCP server’s own spans join the same trace_id.
• Cost + latency on every LLM span answer “how many model rounds, how slow, how expensive.”

Conclusion & Next Steps

Observability for agents is distributed tracing plus LLM and tool semantic conventions. Next: sample spans at scale to bound cardinality, forward traces to Langfuse/LangSmith/Grafana, and alert on “TOOL span error rate by tool name” — the highest-signal agent health metric.

References / Sources

• Model Context Protocol. https://modelcontextprotocol.io
• OTel GenAI semantic conventions. https://opentelemetry.io/docs/specs/semconv/gen-ai/
• Gemini API (usage_metadata). https://ai.google.dev/gemini-api/docs