Skip to content
Blog

Retry Policies and Backoff for Unreliable Tool Calls

Design a retry policy for tool calls inside agent loops: error classification, exponential backoff with jitter, budget-aware attempts, idempotency, circuit breakers, and surfacing failures to the model.

Published on • November 16, 2026

AI Assistant

Your agent called charge_customer, the network hiccupped, and the SDK retried — now the customer is charged twice. Or the opposite: a single 429 killed the whole run because nobody thought to back off. Tool calls inside agent loops sit at the intersection of flaky infrastructure and irreversible side effects, and a naive for _ in range(3): try/except is wrong in both directions.

In this post, you will learn how to design a retry policy for agent tool calls — classifying errors as retryable or fatal, exponential backoff with jitter, honoring Retry-After on 429s, budget-aware attempt limits, idempotency keys for non-idempotent operations, circuit breakers, and surfacing failures back to the model as tool errors so the agent can adapt — implemented as a runnable, hand-rolled Python policy (no dependencies) and tied to the OpenAI Agents SDK’s tool and runner model.

Prerequisites

  • Python 3.11+; pip install openai-agents httpx for the later sections (the retry core itself is stdlib-only).
  • An agent that calls at least one HTTP-backed tool (the examples use httpx-style callables).
  • Basic familiarity with the OpenAI Agents SDK: Agent, @tool, Runner (the tools docs cover what you need).

Classify before you retry: the error taxonomy

The first rule of retry design: never retry an exception you don’t understand. Split every failure into three buckets:

import enum

class ErrorClass(enum.Enum):
    RETRYABLE = "retryable"   # 429, 5xx, timeouts, connection resets
    FATAL = "fatal"           # 400, 401, 403, 404, schema violations
    AMBIGUOUS = "ambiguous"   # timeout AFTER the request may have landed

def classify(exc: Exception) -> ErrorClass:
    status = getattr(exc, "status_code", None)
    if status in (429, 500, 502, 503, 504):
        return ErrorClass.RETRYABLE
    if status is not None and 400 <= status < 500:
        return ErrorClass.FATAL
    if isinstance(exc, (TimeoutError, ConnectionError)):
        # A timeout is ambiguous: the server may have processed it.
        return ErrorClass.AMBIGUOUS
    return ErrorClass.FATAL   # unknown errors fail closed

RETRYABLE is safe to replay. FATAL will fail identically 100 times — retrying burns budget and delays the error the model could have handled. AMBIGUOUS is the interesting bucket: a read-only lookup can retry, but charge_customer cannot — replaying it risks a double charge. That decision belongs to the operation, not the transport, which is why the next section separates the two.

Backoff done right: exponential, jittered, bounded

Once you know an error is retryable, wait — but not a fixed amount. Fixed delays synchronize your clients into a thundering herd. Exponential backoff with full jitter decorrelates them:

import asyncio
import random

async def backoff(attempt: int, *, base: float = 0.5, cap: float = 30.0,
                   retry_after: float | None = None) -> float:
    """Full-jitter delay for the given 0-based attempt."""
    if retry_after is not None:
        return retry_after                      # server told us exactly how long
    ceiling = min(cap, base * (2 ** attempt))
    delay = random.uniform(0, ceiling)          # full jitter
    await asyncio.sleep(delay)
    return delay

Three details carry the weight:

  • retry_after wins. When a 429 or 503 carries Retry-After, the server is making a promise; ignore it and you are just hammering.
  • Full jitter (random.uniform(0, ceiling)) beats “equal jitter” for distributed agents — it maximizes decorrelation, which is what prevents a fleet of workers from retrying in lockstep.
  • The cap (min(cap, base * 2 ** attempt)) keeps a long outage from sleeping an agent for hours.

Budget-aware attempts

Max attempts should not be a constant — they should come from the remaining budget of the run. A triage agent with 2 seconds of latency budget can afford one retry; an overnight batch can afford five:

from dataclasses import dataclass

@dataclass
class RetryPolicy:
    max_attempts: int = 4
    budget_seconds: float = 10.0
    idempotent: bool = False        # replay-safe? set per tool (or True when
                                    # the tool sends an idempotency key)

    def can_retry(self, attempt: int, elapsed: float, cls: ErrorClass) -> bool:
        if cls is ErrorClass.FATAL:
            return False
        if attempt >= self.max_attempts:
            return False
        if elapsed >= self.budget_seconds:
            return False
        if cls is ErrorClass.AMBIGUOUS and not self.idempotent:
            return False            # never blind-replay a charge
        return True

Note the idempotent flag: it is a property of the tool (lookup_ticket is idempotent, send_refund is not — unless it presents an idempotency key, which is exactly what the next section adds), so it is declared next to the tool, not guessed at runtime.

Idempotency keys for non-idempotent operations

For operations that create or move money, retries are only safe if the server deduplicates. Pass an idempotency key derived from the call — not from the attempt — so every retry presents the same identity:

import hashlib
import time

def idempotency_key(ticket_id: str, action: str, args: dict) -> str:
    """Stable across retries: same call → same key."""
    payload = f"{ticket_id}:{action}:{sorted(args.items())}"
    return hashlib.sha256(payload.encode()).hexdigest()[:32]

async def send_refund(ticket_id: str, amount: int, policy: RetryPolicy) -> dict:
    key = idempotency_key(ticket_id, "refund", {"amount": amount})
    start = time.monotonic()
    last_exc: Exception | None = None
    for attempt in range(policy.max_attempts):
        try:
            return await refunds_api.create(ticket_id, amount, idempotency_key=key)
        except Exception as exc:
            last_exc = exc
            cls = classify(exc)
            elapsed = time.monotonic() - start
            if not policy.can_retry(attempt, elapsed, cls):
                raise
            await backoff(attempt, retry_after=getattr(exc, "retry_after", None))
    raise last_exc  # unreachable, but keeps mypy happy

The key is computed once, outside the loop — that is the whole point. If you derived it per attempt, every retry would be a new operation and the server could never dedupe it.

Circuit breakers: stop asking the question you already know the answer to

If a downstream service is down, retrying every call just burns your agent’s latency budget. A circuit breaker trips after repeated failures and fails fast until a probe succeeds:

import time

class CircuitBreaker:
    def __init__(self, failure_threshold: int = 5, reset_seconds: float = 30.0):
        self.failures = 0
        self.threshold = failure_threshold
        self.reset_seconds = reset_seconds
        self.open_until = 0.0

    def check(self) -> None:
        if time.monotonic() < self.open_until:
            raise ConnectionError("circuit open: downstream known-unhealthy")

    def record_success(self) -> None:
        self.failures = 0

    def record_failure(self) -> None:
        self.failures += 1
        if self.failures >= self.threshold:
            self.open_until = time.monotonic() + self.reset_seconds

Wire it around the tool, before the retry loop: breaker.check() raises a retryable-class error immediately, and the model sees a fast, honest failure instead of a 30-second cascade.

Surfacing failures back to the model

This is the part most people miss. When retries are exhausted, do not swallow the error — return it to the model as tool output so the agent can adapt: try a different tool, explain the outage to the user, or ask for clarification. The OpenAI Agents SDK does this natively. Per the tools docs, @tool(failure_error_function=...) controls what the LLM sees when a tool crashes; the default tells the model an error occurred, a custom function gives it a user-friendly message, and passing None re-raises to your code. Tool timeouts follow the same philosophy: timeout_behavior="error_as_result" (the default) returns a model-visible timeout message so the agent recovers, while "raise_exception" fails the run.

from agents import Runner
from agents.decorators import tool

def tool_error_to_model(context, exc: Exception) -> str:
    """Tell the model exactly what happened and what it may do next."""
    if classify(exc) is ErrorClass.RETRYABLE:
        return "PAYMENTS_TIMEOUT: the payments service is slow right now. " \
               "Do not retry the charge. Tell the user we could not complete " \
               "payment and offer to open a status ticket."
    return f"TOOL_FATAL: {exc}. Do not call this tool again for this request."

@tool(failure_error_function=tool_error_to_model)
async def charge_customer(ticket_id: str, amount: int) -> str:
    """Charge the customer for an approved refund adjustment."""
    return await send_refund(ticket_id, amount, POLICY)

The instructions-level guidance matters too: tell the model “if a tool returns TOOL_FATAL, never retry it; if it returns PAYMENTS_TIMEOUT, offer the user a status update.” Error strings are only half the channel — the other half is teaching the model what the strings mean.

The SDK’s own model-call retries are a separate layer and are deliberately opt-in: retries happen only if you set ModelSettings(retry=...) with a policy such as retry_policies.network_error(), which matches transient transport and timeout failures, or retry_policies.provider_suggested(), which follows provider retry advice (including retry_after). The same principle applies one level up: retries are explicit, classified, and bounded — never a blanket except Exception.

Putting it all together

Compose the pieces into a single guarded tool call:

async def run_tool_with_policy(fn, *args, policy: RetryPolicy,
                               breaker: CircuitBreaker) -> object:
    start = time.monotonic()
    attempt = 0
    while True:
        breaker.check()                       # fail fast if downstream is known-bad
        try:
            result = await fn(*args)
            breaker.record_success()
            return result
        except Exception as exc:
            breaker.record_failure()
            cls = classify(exc)
            elapsed = time.monotonic() - start
            if not policy.can_retry(attempt, elapsed, cls):
                raise                         # fatal, out of budget, or unsafe
            await backoff(attempt, retry_after=getattr(exc, "retry_after", None))
            attempt += 1

A typical run against a flaky payments mock — with POLICY = RetryPolicy(idempotent=True) because send_refund presents an idempotency key — looks like:

attempt 1 -> TimeoutError            (ambiguous, but key-safe: replay presents the same key)
attempt 2 -> backoff 0.31s (full jitter)
attempt 3 -> 429 Retry-After: 2.0    -> backoff 2.0s (server-directed)
attempt 4 -> OK {'status': 'refunded', 'idempotency_reused': True}

The run succeeds in ~2.4 seconds, the refund happens exactly once, and if all four attempts had failed, tool_error_to_model would have handed the model a PAYMENTS_TIMEOUT string it knows how to act on.

Conclusion and Next Steps

You built a retry policy that is classified (retryable vs. fatal vs. ambiguous), decorrelated (exponential backoff with full jitter), server-aware (honors Retry-After), budget-bound (attempts scale with the run’s budget), safe for side effects (idempotency keys computed outside the loop), fail-fast (circuit breaker), and model-visible (failures return as tool errors the agent can adapt to). That combination is what separates an agent that survives a bad afternoon from one that double-charges a customer.

Your homework:

  1. Wrap your most dangerous tool with send_refund-style idempotency and prove dedup with a mock server that counts key collisions.
  2. Add metrics: retry count, breaker state, and backoff duration as OpenTelemetry instruments — you cannot tune what you cannot see.
  3. Implement adaptive budgets: lower max_attempts when the run’s token/latency budget shrinks mid-loop.
  4. Extend classify() with your own failure modes (gRPC status codes, database deadlock errors) and re-run your fault-injection suite to confirm nothing lands in FATAL that deserves a retry.