“llm-ops”
Rate-Limiting Agent Tools: 429s, Throttling, and Graceful Degradation
A 429 is the one error an agent cannot think its way out of. Where limits come from, how to honor them at the gateway with LiteLLM, and what your agent should do when the queue is full.
Circuit Breakers for LLM and Tool Dependencies
Apply the circuit-breaker pattern specifically to LLM providers and agent tool endpoints — failure classification, per-dependency state machines, LiteLLM gateway wiring, and fault-injection tests that prove the breaker trips when it should.
Root Cause Analysis Playbooks for Agent Failures: From Symptom to Fix
A systematic method for debugging agent failures: classify the symptom, walk a trace from the failing span upward, and apply a playbook for each failure class — routing, tool, context, hallucination, and infrastructure — with durable execution and state inspection as first-class techniques.
Correlation IDs Across Agent Handoffs
Keep a single identifier alive as control moves between agents: why log IDs break at handoffs, how OpenTelemetry context propagation and baggage carry correlation across boundaries, the security rules for propagating IDs, and a concrete schema for multi-agent trace correlation.
Trace Spans for Tool Calls: Instrumenting Function Execution
Instrument agent tool calls with OpenTelemetry spans: span structure, trace and span IDs, parent relationships, attributes versus events, status codes, wrapping sync and async tools, and the schema that makes tool spans queryable instead of just present.
Token Caps and Output Truncation: Containing Runaway Agents
Enforce strict token budgets, context limits, and output truncation using LiteLLM to prevent runaway cost spikes in multi-turn autonomous agent loops.