Evals for Enterprise Agents: From Unit Tests to Regression Suites

Enterprise agents make decisions that affect your business. Without comprehensive evaluation, you’re shipping changes blind. Evals are the systematic testing of agent behavior—from individual tool calls to end-to-end workflows—ensuring your agents perform correctly at scale.

This guide covers building evaluation suites for enterprise agents, from simple unit tests to complex regression suites that catch regressions before they reach production.

The Evaluation Pyramid

Structure your evaluation strategy like a testing pyramid:

block-beta
    columns 2

    block:topLayer
        E2E["<b>E2E Tests</b><br/>(Full runs)"]
    end
    Note1["← Few, expensive, high confidence"]

    block:midLayer
        Integration["<b>Integration Tests</b>"]
    end
    Note2["← Moderate, test agent interactions"]

    block:botLayer
        Unit["<b>Unit Tests</b><br/>(Tools, Prompts)"]
    end
    Note3["← Many, fast, test individual components"]

Unit Tests

Test individual components in isolation:

import pytest
from agents import Agent, Runner

def test_tool_execution():
    """Test that a tool produces correct output."""
    @function_tool
    def calculate(expression: str) -> str:
        return str(eval(expression))

    result = calculate(expression="2 + 2")
    assert result == "4"

def test_prompt_validation():
    """Test that agent instructions are well-formed."""
    agent = Agent(
        name="Test agent",
        instructions="You are a test assistant.",
    )
    assert agent.name == "Test agent"
    assert "test" in agent.instructions.lower()

Integration Tests

Test agent workflows with real (or mocked) LLM calls:

@pytest.mark.asyncio
async def test_agent_selects_correct_tool():
    """Test that the agent chooses the right tool for a query."""
    agent = Agent(
        name="Research agent",
        tools=[search_tool, calculator_tool, summarizer_tool],
    )

    result = await Runner.run(
        agent,
        "What's the population of France divided by 2?",
    )

    # Verify the agent used calculator, not search
    assert "tool_calls" in result
    assert any("calculator" in tc.name for tc in result.tool_calls)

End-to-End Tests

Test complete workflows against production-like environments:

@pytest.mark.e2e
async def test_full_research_workflow():
    """Test the complete research agent workflow."""
    agent = build_research_agent()

    result = await Runner.run(
        agent,
        "Research the latest trends in quantum computing and summarize in 3 bullet points.",
    )

    # Validate output structure
    assert result.final_output is not None
    lines = result.final_output.strip().split('\n')
    bullet_points = [l for l in lines if l.startswith('-') or l.startswith('*')]
    assert len(bullet_points) == 3

Golden Trajectories

Golden trajectories are human-approved agent runs that serve as reference implementations:

# tests/golden_trajectories.py

GOLDEN_TRAJECTORIES = [
    {
        "input": "Summarize the latest earnings report",
        "expected_tool_sequence": ["search", "read_document", "summarize"],
        "expected_output_contains": ["revenue", "profit", "growth"],
        "max_tokens": 500,
    },
    {
        "input": "Calculate the ROI of our marketing campaign",
        "expected_tool_sequence": ["database_query", "calculator"],
        "expected_output_type": "dict",
        "required_fields": ["roi_percentage", "total_spend", "total_revenue"],
    },
]

@pytest.mark.parametrize("trajectory", GOLDEN_TRAJECTORIES)
async def test_golden_trajectory(trajectory):
    """Test against human-approved reference runs."""
    agent = build_production_agent()

    result = await Runner.run(agent, trajectory["input"])

    # Check tool sequence
    actual_tools = [tc.name for tc in result.tool_calls]
    assert actual_tools == trajectory["expected_tool_sequence"]

    # Check output constraints
    if "expected_output_contains" in trajectory:
        for term in trajectory["expected_output_contains"]:
            assert term.lower() in result.final_output.lower()

    if "max_tokens" in trajectory:
        assert count_tokens(result.final_output) <= trajectory["max_tokens"]

LLM-as-a-Judge

Use an LLM to evaluate agent outputs when exact matching is insufficient:

JUDGE_PROMPT = """Evaluate this agent response on a scale of 1-5:

Query: {query}
Response: {response}
Reference: {reference}

Score based on:
1. Accuracy (1-5): Is the information correct?
2. Completeness (1-5): Does it address all aspects?
3. Clarity (1-5): Is it well-organized and clear?
4. Conciseness (1-5): Is it appropriately brief?

Return JSON: {{"accuracy": N, "completeness": N, "clarity": N, "conciseness": N}}
"""

async def judge_response(query, response, reference):
    """Use LLM to evaluate agent output quality."""
    judge = Agent(name="Judge", instructions=JUDGE_PROMPT)
    result = await Runner.run(
        judge,
        JUDGE_PROMPT.format(query=query, response=response, reference=reference),
    )
    return json.loads(result.final_output)

Tool Selection Evals

Verify agents pick the right tools:

TOOL_SELECTION_CASES = [
    {
        "input": "What's the weather in Paris?",
        "expected_tool": "get_weather",
        "category": "simple_lookup",
    },
    {
        "input": "Compare sales this quarter vs last quarter",
        "expected_tool": "database_query",
        "category": "analysis",
    },
    {
        "input": "Write a Python function to sort a list",
        "expected_tool": "code_generator",
        "category": "code_generation",
    },
]

@pytest.mark.parametrize("case", TOOL_SELECTION_CASES)
async def test_tool_selection(case):
    """Verify agent selects the correct tool."""
    agent = build_agent_with_multiple_tools()

    # Run with tool tracking
    result = await Runner.run(agent, case["input"])

    # Check the first tool called
    assert len(result.tool_calls) > 0
    assert result.tool_calls[0].name == case["expected_tool"]

Multi-Turn Regression Tests

Test agent behavior across conversation turns:

async def test_multi_turn_conversation():
    """Test agent maintains context across turns."""
    agent = build_conversational_agent()

    # Turn 1: Establish context
    result1 = await Runner.run(agent, "My name is Alice and I work at Acme Corp.")
    assert "Alice" in result1.final_output or "Acme" in result1.final_output

    # Turn 2: Verify context retention
    result2 = await Runner.run(
        agent,
        "What's my name and where do I work?",
        previous_context=result1.context,
    )
    assert "Alice" in result2.final_output
    assert "Acme" in result2.final_output

    # Turn 3: Test context-dependent reasoning
    result3 = await Runner.run(
        agent,
        "What industry is my company in?",
        previous_context=result2.context,
    )
    assert any(industry in result3.final_output.lower()
               for industry in ["technology", "software", "tech"])

Regression Suite Structure

Organize your evaluation suite:

tests/
├── unit/
│   ├── test_tools.py
│   ├── test_prompts.py
│   └── test_validators.py
├── integration/
│   ├── test_agent_workflows.py
│   ├── test_tool_selection.py
│   └── test_multi_turn.py
├── e2e/
│   ├── test_production_flows.py
│   └── test_golden_trajectories.py
├── evals/
│   ├── judge_prompts.py
│   ├── benchmark_data.py
│   └── scoring.py
└── conftest.py

CI Integration

Run evals on every commit:

# .github/workflows/agent-evals.yml
name: Agent Evals

on: [push, pull_request]

jobs:
  evals:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run unit tests
        run: pytest tests/unit/ -v
      - name: Run integration tests
        run: pytest tests/integration/ -v --timeout=60
      - name: Run golden trajectory tests
        run: pytest tests/evals/golden_trajectories.py -v
      - name: Run regression suite
        run: python tests/evals/regression_suite.py --threshold=0.85

Metrics to Track

Monitor these evaluation metrics over time:

Metric	Description	Target
Tool selection accuracy	% of times correct tool is chosen	>95%
Output validity	% of outputs passing schema validation	>99%
Golden trajectory match	% matching reference runs	>90%
LLM judge average score	Mean quality score (1-5)	>4.0
Regression pass rate	% of regression tests passing	>95%

Best Practices

1. Start with unit tests: They’re fast and catch obvious issues
2. Build golden trajectories early: Document expected behavior before it drifts
3. Use LLM-as-a-judge for nuance: Exact matching misses quality differences
4. Run evals in CI: Catch regressions before merge
5. Track metrics over time: Trend lines reveal gradual degradation
6. Test edge cases: Empty inputs, malformed data, adversarial prompts
7. Version your eval data: Track test cases alongside code changes

Next Steps

• Explore Agent Tracing to understand agent behavior during evals
• Read about Building Guardrails for output validation
• Learn about Evals with OpenTelemetry for production monitoring

Enterprise agents demand enterprise-grade testing. By building comprehensive evaluation suites—from unit tests to regression suites—you can ship agent changes with confidence, knowing your systems will perform correctly at scale.