Context Engineering: Managing What the Agent Sees

The quality of an agent’s output is directly determined by the quality of its input. Context engineering is the discipline of carefully controlling what information reaches the model, in what order, and at what cost. In agentic systems, this becomes critical: every tool result, conversation history, and system instruction competes for limited context window space.

This guide covers the principles and techniques for managing context in production agent systems.

Why Context Engineering Matters

LLMs have fixed context windows. Every token counts. Poor context management leads to:

• Dropped information: Relevant context pushed out by noise
• Increased costs: Paying for irrelevant tokens on every API call
• Degraded performance: Models confused by contradictory or excessive input
• Latency: Longer prompts take more time to process

Good context engineering means your agent sees exactly what it needs, when it needs it.

The Context Budget

Every context window has a finite budget. Allocate it deliberately:

System instructions:    ~500 tokens (5%)
Conversation history:   ~4,000 tokens (40%)
Tool results:           ~3,000 tokens (30%)
Retrieved context:      ~2,000 tokens (20%)
Response buffer:        ~500 tokens (5%)

The exact allocation depends on your use case. A coding assistant needs more tool result space. A research agent needs more retrieved context.

Context Layering

Structure context in layers, from most to least important:

Layer 1: System Instructions

The system prompt is always present. Keep it concise and actionable:

agent = Agent(
    name="Assistant",
    instructions=(
        "You are a code review assistant. "
        "Focus on security, performance, and correctness. "
        "Always cite specific line numbers when reporting issues."
    ),
)

Layer 2: Working Memory

The current task state, including recent tool results and decisions:

# Tool results from this turn
# Recent conversation turns
# Active task context

Layer 3: Historical Context

Previous conversation turns and past tool results. Summarize aggressively:

# Instead of full history:
# "User asked about X. I found Y. User then asked about Z."

# Keep the last 2-3 turns in full, summarize older turns

Layer 4: Retrieved Knowledge

RAG results, search results, or database queries. Filter for relevance:

# Only include results with relevance score > threshold
# Limit to top-k results
# Truncate long document excerpts

Token Budgeting Strategies

Static Budgeting

Pre-allocate fixed token limits per context section:

MAX_HISTORY = 4000
MAX_TOOL_RESULTS = 3000
MAX_RETRIEVED = 2000

def build_context(history, tool_results, retrieved):
    history_text = truncate_tokens(history, MAX_HISTORY)
    tool_text = truncate_tokens(tool_results, MAX_TOOL_RESULTS)
    retrieved_text = truncate_tokens(retrieved, MAX_RETRIEVED)

    return f"{history_text}\n{tool_text}\n{retrieved_text}"

Dynamic Budgeting

Adjust allocations based on the current task:

def dynamic_budget(task_type, total_budget):
    if task_type == "coding":
        return {"history": 0.3, "tools": 0.5, "retrieved": 0.2}
    elif task_type == "research":
        return {"history": 0.2, "tools": 0.3, "retrieved": 0.5}
    else:
        return {"history": 0.4, "tools": 0.3, "retrieved": 0.3}

Overflow Handling

When context exceeds the budget, prioritize:

1. System instructions (always keep)
2. Most recent tool results (critical for continuity)
3. User’s last message (never drop)
4. Retrieved context (summarize or truncate)
5. Older history (summarize aggressively)

Context Compression

Reduce token usage without losing information:

Summarization

Replace long tool results with summaries:

# Instead of full tool output:
# "File contains 500 lines of Python code implementing..."

# Summarize:
# "Python file with 500 lines: authentication module with JWT handling"

Deduplication

Remove redundant information across context layers:

# Don't repeat the same code snippet in tool results AND retrieved context
# Keep it in one place, reference it in others

Selective Inclusion

Only include context that’s relevant to the current turn:

def select_relevant(history, current_query):
    # Use embedding similarity to filter history
    # Only keep turns relevant to the current question
    pass

Context for Multi-Agent Systems

In multi-agent architectures, context management becomes more complex:

Shared Context

When agents share context, define clear ownership:

# Shared context: conversation history, user preferences
# Agent-specific context: tool results, working memory

shared = {"history": [...], "user": {...}}
agent_a_context = {"tools": [...], "working": {...}}
agent_b_context = {"tools": [...], "working": {...}}

Context Passing

When handing off between agents, pass only what’s needed:

# Don't pass entire context history
# Pass a summary and the relevant facts for the next agent

handoff_context = {
    "summary": "User wants to deploy service X to production",
    "relevant_facts": ["service_x is on v2.1", "deployment requires approval"],
    "current_state": "awaiting deployment approval",
}

Measuring Context Quality

Track these metrics to evaluate your context engineering:

• Token efficiency: Useful tokens / total tokens
• Information completeness: Required info present in context
• Cost per task: API cost relative to task complexity
• Latency: Time to process the full context
• Accuracy: Task success rate with current context strategy

Best Practices

1. Start minimal: Add context only when the agent needs it
2. Measure token usage: Track costs per workflow
3. Compress aggressively: Summarize, don’t truncate
4. Layer strategically: Most important context first
5. Test edge cases: Ensure context handles worst-case scenarios
6. Monitor drift: Context quality can degrade as data grows

Context Engineering with Gemini

Gemini models offer large context windows (up to 10M+ tokens), but context engineering is still important:

• Context caching: Cache frequently used context to reduce costs
• Prompt caching: Store system instructions and tool definitions
• Selective retrieval: Use Gemini’s built-in retrieval capabilities

# Gemini's context caching
cache = model.create_cache(
    system_instruction="You are a helpful assistant.",
    tools=[...],
    ttl=3600,  # Cache for 1 hour
)

Next Steps

• Read about Optimizing Context Windows in the Agents SDK
• Explore Prompt Caching for Agentic Loops to reduce costs
• Learn about Structured Outputs to control what agents return

Context engineering is not a one-time task. It’s an ongoing practice of monitoring, measuring, and optimizing what your agents see. The better your context, the better your agent.