Legal Contract Review Agents: Extraction, Clause Risk, and Citation

Contract review is one of the most attractive agent workloads there is: high volume, repetitive structure, and a clear business cost when slow. It is also one where a fluent wrong answer is worse than no answer. An agent that flags “Section 8.2 contains an unlimited indemnity” when Section 8.2 is actually a notices clause does not save lawyer time — it burns trust and, in the worst case, a deal.

The discipline that makes contract agents useful is the same one that makes any document agent trustworthy: extraction grounded in source spans, risk scoring against known policies, and citations that survive scrutiny. Here is the shape of a system that does all three, using the component patterns from LlamaIndex as the reference architecture.

Stage 1: Parse and chunk with the document’s structure in mind

Contracts are not flat text. Numbered sections, nested subclauses, definitions blocks, and cross-references (“as defined in Section 2.1”) are the load-bearing structure. A generic text splitter that cuts every 512 tokens will slice a clause in half and make cross-reference resolution impossible.

Two adjustments fix most of it:

• Parse PDFs properly. A dedicated parse layer (LlamaParse, or an equivalent layout-aware parser) preserves headings and tables — critical because indemnity caps and payment terms love to live in tables.
• Split on section boundaries. LlamaIndex’s node parsers accept custom splitters; configure separators that respect numbering schemes (1., 1.1., (a)) and keep each clause with its heading as metadata.

from llama_index.core.node_parser import SentenceSplitter
from llama_index.core.ingestion import IngestionPipeline

pipeline = IngestionPipeline(
    transformations=[
        SectionAwareSplitter(),          # split on clause boundaries
        TitleExtractor(),                # infer section titles
        OpenAIEmbedding(),
    ],
)
nodes = pipeline.run(documents=docs)

The ingestion pipeline’s per-node transformation cache means a re-run against a revised contract only re-processes changed nodes — and its document management layer tracks doc_id → hash, so an unchanged agreement is skipped entirely while an amended one is re-ingested and upserted. For a review workflow that sees the same master agreement six times a year, that caching is not a luxury; it is what keeps the pipeline affordable.

Each node carries metadata you will lean on later: document id, page number, section number, clause type.

Stage 2: Structured extraction, not free-form summary

The extraction stage’s job is to fill a schema, nothing more:

class ClauseExtraction(BaseModel):
    clause_type: Literal["indemnity", "limitation_of_liability",
                         "termination", "governing_law", "auto_renewal", "other"]
    section_ref: str
    text_span: str          # verbatim text from the document
    page: int
    extracted_terms: dict   # e.g. {"cap": "12 months fees", "survival": "indefinite"}

Two rules keep this honest:

1. Require a verbatim text_span. If the extraction cannot quote the clause, it did not find it. Validate that the span is a substring of the source node before accepting the output — a cheap check that catches hallucinated clauses immediately.
2. Use tools for multi-step review. LlamaIndex-style agents with tool access can query the extracted index (“find every clause mentioning assignment”), then reason over the results — rather than one giant prompt pretending to hold the whole contract. The agent’s tool calls also give you an audit trail of how it reached a conclusion.

Stage 3: Risk scoring against your playbook

Extraction without interpretation is a database. The value-add is comparing each extracted clause to your firm’s or client’s playbook:

PLAYBOOK = {
    "limitation_of_liability": {
        "preferred": "12 months of fees, mutual",
        "acceptable": "24 months of fees",
        "red_flags": ["uncapped", "consequential damages waiver absent", "one-way"],
    },
    "auto_renewal": {"red_flags": ["notice window < 60 days"]},
}

Score each clause green/amber/red against the playbook, and require the extracted terms as input to the judgment — the scorer reasons over {"cap": "uncapped"}, not over the model’s memory of the document. This is where grounding compounds: a wrong extraction now produces a visibly wrong score that a human catches at the span level.

Stage 4: Citations that survive a lawyer’s spot-check

Every assertion in the output must carry its provenance: document name, section number, page, and the quoted span. Present findings as:

Red — Limitation of Liability (MSA §9.3, p. 14): “…liable for any and all damages without limitation…” — playbook requires a mutual cap.

The citation format matters less than the invariant: the quoted text is copied from the node the agent actually read, and you verified the substring before rendering it. A reviewer who clicks through three citations and finds all three verbatim stops spot-checking — which is exactly the trust state you want before volume scales.

Evaluation: the part everyone skips

Contract review is high-stakes enough that “it looked good” is not a test strategy. LlamaIndex’s evaluation module gives you the pattern: build a labeled set of 30–50 real contracts with known clause positions and known playbook verdicts, then measure:

• Extraction recall — did the agent find every red-flag clause? (Missing one is the expensive failure.)
• Span accuracy — is the quoted text verbatim and correctly attributed?
• Verdict accuracy — green/amber/red agreement with lawyer labels.

Run the suite on every prompt or model change, and add observability so individual review runs are traceable when something looks off.

Where humans stay in the loop

The right operating model is triage, not replacement: the agent processes 100% of contracts, surfaces red-flag findings with citations, and a lawyer approves the output. Standard NDAs with all-green clauses fast-track; anything amber or above gets eyes. That keeps a human as the liability boundary while the agent eliminates the reading time on the boring eighty percent.

Further reading

• LlamaIndex component guides — the full module map: loading, indexing, agents, evaluation, observability
• Ingestion pipeline — transformations, caching, and document management for revision-heavy corpora
• LlamaIndex evaluation module — building regression evals for document workflows

Contract review agents do not need to be lawyers. They need to be citable: find the clause, quote it exactly, score it against a written policy, and show its work. Everything else is prompt engineering on top of that foundation.