Explainable Actions: Surfacing Agent Reasoning for Trust

“The agent refunded the customer” is not an acceptable sentence in a production incident review. Someone will ask which tool was called, with what arguments, what came back, and which policy allowed it — and if the only answer is “the model decided,” the feature ships once and gets turned off twice.

Explainability for agents is usually discussed as a model problem: attribution, logprobs, chain-of-thought. That matters for researchers, but it is the wrong layer for a product team. Users and auditors do not need raw reasoning tokens. They need a structured action record: what the agent decided, which tool it called, with what arguments, what the tool returned, how long it took, and what changed in state. This post is about building that record as first-class data.

Why “the agent did X” fails in production

Three audiences need different slices of the same underlying record:

• Audits. A regulator or a customer asks why a decision was made three weeks ago. You need a durable, ordered history — not a log line that rotated out of your aggregator.
• Debugging. An engineer reproduces a bad run. The failure is almost never in the final answer; it is in step 4, where the agent called lookup_order with an order ID it invented.
• User trust. A user watching an agent act on their behalf wants to see the action, not the deliberation. “Checked your last 3 orders → found order #4471 → issued refund” earns confidence. A wall of reasoning text does the opposite: it reads as noise, and it invites users to over-trust fluent text that was never verified.

Chain-of-thought is the wrong artifact to expose

Exposing chain-of-thought is tempting and mostly a dead end. It is unverifiable — nothing in the transcript links the prose to the action actually taken. It is unstable across model versions. It leaks system instructions. And in most frameworks the intermediate reasoning never survives the run anyway.

The action record does survive, because it is already how agent frameworks move data around. In Google’s Agent Development Kit (ADK), every significant occurrence in a run is an Event — user messages, agent replies, function calls, tool results, state changes, and control signals. The Events documentation describes an Event as carrying author, invocation_id, timestamp, actions, and content parts, and states plainly that the sequence recorded in session.events “provides a complete, chronological history of an interaction, invaluable for debugging, auditing, and understanding agent behavior step-by-step.”

That is your explainability layer. You are not reconstructing what the agent did after the fact; you are rendering the stream the runtime already produces.

Design: sessions and events as first-class data

1. Persist the run, not just the answer

ADK’s Session model gives you the container: a Session holds id, app_name, user_id, an events history, session state, and last_update_time. The SessionService appends each Event and applies the deltas it carries, so state changes are recorded against the event that caused them.

Pick a persistent service for anything you intend to explain later. InMemorySessionService loses everything on restart, which is fine for a demo and useless for an audit. DatabaseSessionService (PostgreSQL, MySQL, SQLite) or VertexAiSessionService give you history that outlives the process.

2. Classify events instead of dumping them

Every event already answers the three questions a reviewer asks:

def action_card(event) -> dict | None:
    """Render one ADK event as a human-readable action record."""
    if calls := event.get_function_calls():
        c = calls[0]
        return {"kind": "tool_call", "tool": c.name, "args": c.args,
                "invocation": event.invocation_id}

    if responses := event.get_function_responses():
        r = responses[0]
        return {"kind": "tool_result", "tool": r.name,
                "summary": summarize(r.response)}

    if event.actions and event.actions.state_delta:
        return {"kind": "state_change", "delta": event.actions.state_delta}

    if event.actions and event.actions.transfer_to_agent:
        return {"kind": "handoff", "to": event.actions.transfer_to_agent}

    if event.content and event.content.parts and not event.partial:
        return {"kind": "message", "author": event.author,
                "text": event.content.parts[0].text}
    return None

The framework gives you the semantics for free: event.author says who acted, event.invocation_id correlates everything within one user request, get_function_calls() yields name and args, get_function_responses() yields the result dictionary, and event.actions carries state_delta, artifact_delta, plus control signals like transfer_to_agent and escalate. Use event.is_final_response() to decide what is a finished, displayable answer versus an intermediate step.

3. Capture the parts the event stream does not know

Events tell you what happened, not how long it took or whether it was blocked. For that, ADK’s callbacks are the right hook: before_agent_callback / after_agent_callback, before_model_callback / after_model_callback, and before_tool_callback / after_tool_callback wrap each stage. Returning None lets the step proceed; returning an object overrides it — which is also how you implement guardrails that produce an explainable “blocked” record instead of a silent failure.

import time

def before_tool(context, tool, args):
    context.state["_tool_started"] = time.perf_counter()
    audit.emit("tool.start", agent=context.agent_name,
               tool=tool.name, args=redact(args))
    return None  # allow execution

def after_tool(context, tool, args, result):
    elapsed = time.perf_counter() - context.state.pop("_tool_started", time.perf_counter())
    audit.emit("tool.end", tool=tool.name, duration_ms=round(elapsed * 1000),
               ok=not is_error(result))
    return None

4. Stream state deltas to the UI

Do not wait for the final answer to reveal what is happening. ADK signals state changes through event.actions.state_delta, which the UI can apply incrementally — the same delta the SessionService merges into persistent state. Partial text (event.partial == True) plus state deltas gives you live progress without inventing a second channel.

5. Render tool-call cards, not transcripts

In the product surface, each action becomes a card: tool name, arguments, result summary, latency, status. Collapsed by default, expandable for support staff, linkable by invocation_id so an engineer and a customer can point at the same step.

What not to show — and what to keep anyway

There is a critical distinction between what users see and what you retain.

• Do not dump raw reasoning tokens. They are unverifiable and they leak your prompt.
• Do not render prompt-injected content verbatim. If a tool returned attacker-controlled text, echoing it into your UI is a second injection vector. Summarize, sandbox, and mark provenance.
• Do not enable content capture by default. ADK elides prompt content in logs for security. Turning on full capture is explicit: the OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT environment variable accepts NO_CONTENT, EVENT_ONLY, SPAN_ONLY, and SPAN_AND_EVENT, and can be scoped to a single run via RunConfig(telemetry=TelemetryConfig(capture_message_content=ContentCapturingMode.SPAN_AND_EVENT)). The logging docs warn directly that this records full prompts and responses and may capture PII.
• Retention and redaction are part of the design. Keep action metadata (tool, args schema, duration, outcome) for your retention window; treat full content as short-lived and access-controlled. ADK’s DebugLoggingPlugin already redacts credentials and temp:-scoped state keys in Python, but its output file still holds raw prompts and tool arguments — the docs tell you to treat it as sensitive.

Production log levels follow the same rule: ADK’s INFO level records tool executions with names and arguments, while DEBUG (as in adk web --log_level DEBUG) logs full LLM prompts. The guidance is INFO or WARNING in production, DEBUG only while actively troubleshooting.

A checklist for explainable agent actions

1. Every run has a durable Session with an ordered events history that outlives the process.
2. Every event classifies cleanly: tool call, tool result, state change, handoff, message, error.
3. Every action record carries author, invocation_id, timestamp, and outcome.
4. Before/after model and tool callbacks record duration, errors, and overrides.
5. State deltas stream to the UI as they happen, not in a final batch.
6. The UI shows tool cards (name, args, result summary, latency) — never raw reasoning.
7. Content capture is opt-in, scoped, and paired with a retention schedule.
8. Injected and third-party content is summarized and attributed, never echoed.
9. A support engineer can open any past invocation and reconstruct the sequence.
10. A non-engineer can read the same sequence and understand what the agent did.

Closing

Explainability in an agent system is not a model feature; it is a data-modeling decision you make once, at the event layer, and then render for each audience. Start from the runtime’s own history rather than trying to interpret prose after the fact.

Primary sources: the ADK Python repository, ADK’s Events, Sessions, Callbacks, and activity logging documentation, and the OpenTelemetry GenAI semantic conventions for structured content-capture controls.