Retry Policies and Backoff for Unreliable Tool Calls
Design a retry policy for tool calls inside agent loops: error classification, exponential backoff with jitter, budget-aware attempts, idempotency, circuit breakers, and surfacing failures to the model.
Published on • November 16, 2026
AI Assistant

Your agent called charge_customer, the network hiccupped, and the SDK retried — now the customer is charged twice. Or the opposite: a single 429 killed the whole run because nobody thought to back off. Tool calls inside agent loops sit at the intersection of flaky infrastructure and irreversible side effects, and a naive for _ in range(3): try/except is wrong in both directions.
In this post, you will learn how to design a retry policy for agent tool calls — classifying errors as retryable or fatal, exponential backoff with jitter, honoring Retry-After on 429s, budget-aware attempt limits, idempotency keys for non-idempotent operations, circuit breakers, and surfacing failures back to the model as tool errors so the agent can adapt — implemented as a runnable, hand-rolled Python policy (no dependencies) and tied to the OpenAI Agents SDK’s tool and runner model.
Prerequisites
- Python 3.11+;
pip install openai-agents httpxfor the later sections (the retry core itself is stdlib-only). - An agent that calls at least one HTTP-backed tool (the examples use
httpx-style callables). - Basic familiarity with the OpenAI Agents SDK:
Agent,@tool,Runner(the tools docs cover what you need).
Classify before you retry: the error taxonomy
The first rule of retry design: never retry an exception you don’t understand. Split every failure into three buckets:
import enum
class ErrorClass(enum.Enum):
RETRYABLE = "retryable" # 429, 5xx, timeouts, connection resets
FATAL = "fatal" # 400, 401, 403, 404, schema violations
AMBIGUOUS = "ambiguous" # timeout AFTER the request may have landed
def classify(exc: Exception) -> ErrorClass:
status = getattr(exc, "status_code", None)
if status in (429, 500, 502, 503, 504):
return ErrorClass.RETRYABLE
if status is not None and 400 <= status < 500:
return ErrorClass.FATAL
if isinstance(exc, (TimeoutError, ConnectionError)):
# A timeout is ambiguous: the server may have processed it.
return ErrorClass.AMBIGUOUS
return ErrorClass.FATAL # unknown errors fail closed
RETRYABLE is safe to replay. FATAL will fail identically 100 times — retrying burns budget and delays the error the model could have handled. AMBIGUOUS is the interesting bucket: a read-only lookup can retry, but charge_customer cannot — replaying it risks a double charge. That decision belongs to the operation, not the transport, which is why the next section separates the two.
Backoff done right: exponential, jittered, bounded
Once you know an error is retryable, wait — but not a fixed amount. Fixed delays synchronize your clients into a thundering herd. Exponential backoff with full jitter decorrelates them:
import asyncio
import random
async def backoff(attempt: int, *, base: float = 0.5, cap: float = 30.0,
retry_after: float | None = None) -> float:
"""Full-jitter delay for the given 0-based attempt."""
if retry_after is not None:
return retry_after # server told us exactly how long
ceiling = min(cap, base * (2 ** attempt))
delay = random.uniform(0, ceiling) # full jitter
await asyncio.sleep(delay)
return delay
Three details carry the weight:
retry_afterwins. When a 429 or 503 carriesRetry-After, the server is making a promise; ignore it and you are just hammering.- Full jitter (
random.uniform(0, ceiling)) beats “equal jitter” for distributed agents — it maximizes decorrelation, which is what prevents a fleet of workers from retrying in lockstep. - The cap (
min(cap, base * 2 ** attempt)) keeps a long outage from sleeping an agent for hours.
Budget-aware attempts
Max attempts should not be a constant — they should come from the remaining budget of the run. A triage agent with 2 seconds of latency budget can afford one retry; an overnight batch can afford five:
from dataclasses import dataclass
@dataclass
class RetryPolicy:
max_attempts: int = 4
budget_seconds: float = 10.0
idempotent: bool = False # replay-safe? set per tool (or True when
# the tool sends an idempotency key)
def can_retry(self, attempt: int, elapsed: float, cls: ErrorClass) -> bool:
if cls is ErrorClass.FATAL:
return False
if attempt >= self.max_attempts:
return False
if elapsed >= self.budget_seconds:
return False
if cls is ErrorClass.AMBIGUOUS and not self.idempotent:
return False # never blind-replay a charge
return True
Note the idempotent flag: it is a property of the tool (lookup_ticket is idempotent, send_refund is not — unless it presents an idempotency key, which is exactly what the next section adds), so it is declared next to the tool, not guessed at runtime.
Idempotency keys for non-idempotent operations
For operations that create or move money, retries are only safe if the server deduplicates. Pass an idempotency key derived from the call — not from the attempt — so every retry presents the same identity:
import hashlib
import time
def idempotency_key(ticket_id: str, action: str, args: dict) -> str:
"""Stable across retries: same call → same key."""
payload = f"{ticket_id}:{action}:{sorted(args.items())}"
return hashlib.sha256(payload.encode()).hexdigest()[:32]
async def send_refund(ticket_id: str, amount: int, policy: RetryPolicy) -> dict:
key = idempotency_key(ticket_id, "refund", {"amount": amount})
start = time.monotonic()
last_exc: Exception | None = None
for attempt in range(policy.max_attempts):
try:
return await refunds_api.create(ticket_id, amount, idempotency_key=key)
except Exception as exc:
last_exc = exc
cls = classify(exc)
elapsed = time.monotonic() - start
if not policy.can_retry(attempt, elapsed, cls):
raise
await backoff(attempt, retry_after=getattr(exc, "retry_after", None))
raise last_exc # unreachable, but keeps mypy happy
The key is computed once, outside the loop — that is the whole point. If you derived it per attempt, every retry would be a new operation and the server could never dedupe it.
Circuit breakers: stop asking the question you already know the answer to
If a downstream service is down, retrying every call just burns your agent’s latency budget. A circuit breaker trips after repeated failures and fails fast until a probe succeeds:
import time
class CircuitBreaker:
def __init__(self, failure_threshold: int = 5, reset_seconds: float = 30.0):
self.failures = 0
self.threshold = failure_threshold
self.reset_seconds = reset_seconds
self.open_until = 0.0
def check(self) -> None:
if time.monotonic() < self.open_until:
raise ConnectionError("circuit open: downstream known-unhealthy")
def record_success(self) -> None:
self.failures = 0
def record_failure(self) -> None:
self.failures += 1
if self.failures >= self.threshold:
self.open_until = time.monotonic() + self.reset_seconds
Wire it around the tool, before the retry loop: breaker.check() raises a retryable-class error immediately, and the model sees a fast, honest failure instead of a 30-second cascade.
Surfacing failures back to the model
This is the part most people miss. When retries are exhausted, do not swallow the error — return it to the model as tool output so the agent can adapt: try a different tool, explain the outage to the user, or ask for clarification. The OpenAI Agents SDK does this natively. Per the tools docs, @tool(failure_error_function=...) controls what the LLM sees when a tool crashes; the default tells the model an error occurred, a custom function gives it a user-friendly message, and passing None re-raises to your code. Tool timeouts follow the same philosophy: timeout_behavior="error_as_result" (the default) returns a model-visible timeout message so the agent recovers, while "raise_exception" fails the run.
from agents import Runner
from agents.decorators import tool
def tool_error_to_model(context, exc: Exception) -> str:
"""Tell the model exactly what happened and what it may do next."""
if classify(exc) is ErrorClass.RETRYABLE:
return "PAYMENTS_TIMEOUT: the payments service is slow right now. " \
"Do not retry the charge. Tell the user we could not complete " \
"payment and offer to open a status ticket."
return f"TOOL_FATAL: {exc}. Do not call this tool again for this request."
@tool(failure_error_function=tool_error_to_model)
async def charge_customer(ticket_id: str, amount: int) -> str:
"""Charge the customer for an approved refund adjustment."""
return await send_refund(ticket_id, amount, POLICY)
The instructions-level guidance matters too: tell the model “if a tool returns TOOL_FATAL, never retry it; if it returns PAYMENTS_TIMEOUT, offer the user a status update.” Error strings are only half the channel — the other half is teaching the model what the strings mean.
The SDK’s own model-call retries are a separate layer and are deliberately opt-in: retries happen only if you set ModelSettings(retry=...) with a policy such as retry_policies.network_error(), which matches transient transport and timeout failures, or retry_policies.provider_suggested(), which follows provider retry advice (including retry_after). The same principle applies one level up: retries are explicit, classified, and bounded — never a blanket except Exception.
Putting it all together
Compose the pieces into a single guarded tool call:
async def run_tool_with_policy(fn, *args, policy: RetryPolicy,
breaker: CircuitBreaker) -> object:
start = time.monotonic()
attempt = 0
while True:
breaker.check() # fail fast if downstream is known-bad
try:
result = await fn(*args)
breaker.record_success()
return result
except Exception as exc:
breaker.record_failure()
cls = classify(exc)
elapsed = time.monotonic() - start
if not policy.can_retry(attempt, elapsed, cls):
raise # fatal, out of budget, or unsafe
await backoff(attempt, retry_after=getattr(exc, "retry_after", None))
attempt += 1
A typical run against a flaky payments mock — with POLICY = RetryPolicy(idempotent=True) because send_refund presents an idempotency key — looks like:
attempt 1 -> TimeoutError (ambiguous, but key-safe: replay presents the same key)
attempt 2 -> backoff 0.31s (full jitter)
attempt 3 -> 429 Retry-After: 2.0 -> backoff 2.0s (server-directed)
attempt 4 -> OK {'status': 'refunded', 'idempotency_reused': True}
The run succeeds in ~2.4 seconds, the refund happens exactly once, and if all four attempts had failed, tool_error_to_model would have handed the model a PAYMENTS_TIMEOUT string it knows how to act on.
Conclusion and Next Steps
You built a retry policy that is classified (retryable vs. fatal vs. ambiguous), decorrelated (exponential backoff with full jitter), server-aware (honors Retry-After), budget-bound (attempts scale with the run’s budget), safe for side effects (idempotency keys computed outside the loop), fail-fast (circuit breaker), and model-visible (failures return as tool errors the agent can adapt to). That combination is what separates an agent that survives a bad afternoon from one that double-charges a customer.
Your homework:
- Wrap your most dangerous tool with
send_refund-style idempotency and prove dedup with a mock server that counts key collisions. - Add metrics: retry count, breaker state, and backoff duration as OpenTelemetry instruments — you cannot tune what you cannot see.
- Implement adaptive budgets: lower
max_attemptswhen the run’s token/latency budget shrinks mid-loop. - Extend
classify()with your own failure modes (gRPC status codes, database deadlock errors) and re-run your fault-injection suite to confirm nothing lands inFATALthat deserves a retry.