Fan-Out/Fan-In: Parallel Execution in Multi-Agent Systems

In complex multi-agent architectures, executing tasks sequentially is often inefficient. If an analysis task requires evaluating market data across five different regions, running five agents sequentially takes 5x longer than running them concurrently.

The Fan-Out / Fan-In Pattern (also known as Map-Reduce for agents) solves this by splitting a task into independent sub-tasks, fanning them out to worker agents in parallel, and aggregating the results in a final synthesis node.

Anatomy of the Fan-Out / Fan-In Pattern

The workflow consists of three distinct graph execution phases:

1. Orchestrator / Dispatcher (Fan-Out): Analyzes the initial input and generates $N$ parallel sub-tasks or sub-queries.
2. Parallel Worker Nodes: $N$ independent agent instances execute concurrently, each working on its assigned sub-task without waiting for others.
3. Aggregator / Synthesizer Node (Fan-In): Collects all $N$ worker outputs, resolves discrepancies, and compiles a unified final response.

                  --> [Worker Agent A (North America)] --
                 /                                       \
[Dispatch Node] ------> [Worker Agent B (Europe)] --------> [Aggregator Node]
                 \                                       /
                  --> [Worker Agent C (Asia-Pacific)] ---

Implementing Fan-Out / Fan-In in LangGraph

LangGraph natively supports parallel branch execution using conditional edges or map-reduce graph structures.

import operator
from typing import TypedDict, Annotated, List
from langgraph.graph import StateGraph, END

# Define shared state with an annotated reducer for collecting parallel outputs
class FanOutState(TypedDict):
    topic: str
    subtasks: List[str]
    # 'operator.add' appends parallel worker outputs to a single list safely
    worker_results: Annotated[List[str], operator.add]
    final_summary: str

def dispatch_node(state: FanOutState):
    """Splits high-level topic into parallel region queries"""
    topic = state["topic"]
    subtasks = [
        f"Analyze market trends for {topic} in North America",
        f"Analyze market trends for {topic} in Europe",
        f"Analyze market trends for {topic} in Asia"
    ]
    return {"subtasks": subtasks}

def worker_agent_node(state: FanOutState):
    """Worker node executed concurrently for each subtask"""
    # In practice, each worker processes its assigned slice
    results = []
    for subtask in state["subtasks"]:
        results.append(f"Result for '{subtask}': Growth rate +14%")
    return {"worker_results": results}

def aggregator_node(state: FanOutState):
    """Fan-In node that compiles parallel outputs into final report"""
    combined = "\n".join(state["worker_results"])
    summary = f"Executive Summary for {state['topic']}:\n{combined}"
    return {"final_summary": summary}

Graph Construction

workflow = StateGraph(FanOutState)

workflow.add_node("dispatch", dispatch_node)
workflow.add_node("workers", worker_agent_node)
workflow.add_node("aggregate", aggregator_node)

workflow.set_entry_point("dispatch")
workflow.add_edge("dispatch", "workers")
workflow.add_edge("workers", "aggregate")
workflow.add_edge("aggregate", END)

app = workflow.compile()

Performance & Scalability Benefits

• Dramatic Latency Reduction: Reduces wall-clock latency from $O(N)$ to $O(1)$ (bounded by the single slowest worker execution time).
• Fault Isolation: If one parallel worker fails or times out, the aggregator can still compile a partial result from the remaining completed workers.
• Context Window Efficiency: Worker agents process small, focused contexts rather than forcing one massive model call to process all sub-domains simultaneously.

For comprehensive guides on state graph reducers, parallel branching, and map-reduce patterns, explore the official LangGraph Documentation.