Skip to content
Blog

Tax Season Automation: Agents That Prep and Review Returns

Build a Gemini API pipeline that ingests W-2 and 1099 PDFs, extracts structured fields, drafts a return summary, and runs an independent review pass that flags inconsistencies.

Published on • November 16, 2026

AI Assistant

Every February the same scene plays out: a folder of W-2s, three 1099-NECs, a brokerage PDF that is really a scan of a scan, and a person retyping box 1 into a spreadsheet. Transcription is not the hard part — the hard part is that nobody notices the mismatch between the W-2 wage total and the sum of the 1099s until the return is already filed.

Retyping is automation. Checking is automation too, and it is the part that actually saves you.

In this post, you will learn how to build a two-stage tax document pipeline on the Gemini API with the google-genai Python SDK: a preparation stage that ingests W-2/1099 PDFs and extracts typed fields using structured outputs, and an independent review stage that re-reads the data, calls your verification functions through function calling, and emits flagged inconsistencies. You will also use document understanding for native PDF parsing and safety settings.

Disclaimer: This is an engineering tutorial, not tax advice. Nothing here substitutes for a licensed CPA, EA, or attorney. Tax rules change yearly and vary by jurisdiction — treat every number this pipeline produces as a draft a qualified human must verify before anything is filed.

Prerequisites

  • Python 3.9+ and pip install google-genai pydantic.
  • A Gemini API key from Google AI Studio, exported as GEMINI_API_KEY — genai.Client() picks it up automatically.
  • Sample W-2 or 1099 PDFs you may legally process: redacted or synthetic forms only while developing.
  • Basic comfort with Pydantic; no prior Gemini experience required.

A note on API surface first: as the Gemini API docs state, the Interactions API (client.interactions.create(...)) is the default interface as of June 2026, while generateContent remains supported but is legacy. All code below uses the current interface.

Step 1: Define the Schema Before You Write the Prompt

Structured outputs let you constrain the model to a strict JSON Schema so downstream code never parses prose. The SDK integrates natively with Pydantic, so your schema is a class, not a string you maintain in two places.

# tax_pipeline/schemas.py
from pydantic import BaseModel, Field
from typing import Optional


class W2(BaseModel):
    """Fields pulled from a Form W-2."""
    tax_year: int = Field(description="Tax year shown on the form")
    employer_name: str
    employer_ein: str = Field(description="EIN, digits only")
    employee_name: str
    employee_ssn_last4: str = Field(description="SSN last 4, digits only")
    wages_tips_other_compensation: int = Field(description="Box 1, whole dollars")
    federal_income_tax_withheld: int = Field(description="Box 2, whole dollars")
    social_security_wages: int = Field(description="Box 3")
    medicare_wages_and_tips: int = Field(description="Box 5")
    box_12_codes: dict[str, str] = Field(
        default_factory=dict,
        description="Box 12 code → value, e.g. {'D': '23000'}",
    )
    state_wages: Optional[int] = Field(None, description="Box 16 if present")


class Form1099NEC(BaseModel):
    tax_year: int
    payer_name: str
    payer_tin: str
    recipient_name: str
    nonemployee_compensation: int = Field(description="Box 1, whole dollars")
    federal_income_tax_withheld: int = Field(description="Box 4")

Why this matters: a field described as "digits only" and typed as int removes a class of downstream bugs, and model_validate_json(...) either succeeds or raises — no “sometimes it returns a string” branch. The Optional on state wages is deliberate: a schema demanding a field the document lacks invites invention.

Step 2: Ingest and Extract from PDFs

Gemini does vision-based PDF parsing rather than plain OCR — up to 1,000 pages per request, with tables, checkboxes, and scattered layouts intact. Send the file as a base64 document part alongside the extraction instruction:

# tax_pipeline/extract.py
import base64
from google import genai
from tax_pipeline.schemas import W2

client = genai.Client()
MODEL = "gemini-4-hydrogen"   # 1M-token context, text/image/audio/video/PDF


def extract_w2(pdf_path: str) -> W2:
    """Read one W-2 PDF and return a validated, typed record."""
    with open(pdf_path, "rb") as f:
        pdf_b64 = base64.b64encode(f.read()).decode("utf-8")

    interaction = client.interactions.create(
        model=MODEL,
        input=[
            {"type": "document", "data": pdf_b64, "mime_type": "application/pdf"},
            {
                "type": "text",
                "text": (
                    "Extract the Form W-2 fields exactly as printed. "
                    "Never infer or calculate a value that is not printed on the form. "
                    "If a box is blank or illegible, use null."
                ),
            },
        ],
        response_format={
            "type": "text",
            "mime_type": "application/json",
            "schema": W2.model_json_schema(),
        },
    )
    return W2.model_validate_json(interaction.output_text)

Three choices worth calling out:

  • The instruction forbids inference. “Never infer or calculate” is the most important sentence in the prompt. Extraction is transcription, not arithmetic — arithmetic belongs to the review stage, where a disagreement is a finding instead of a silently corrected number.
  • response_format carries the schema (type text, MIME application/json, your JSON Schema), so the model cannot drift out of shape — and model_validate_json raises a loud ValidationError at ingestion instead of corrupting a record three stages later.

For very large returns — a 300-page statement or a folder of forms — the docs recommend client.files.upload(file=...) plus a {"type": "file", "file_uri": uploaded.uri} part instead of inlining base64.

Step 3: Draft the Return Summary

With typed records in hand, drafting is a plain structured-output call over the assembled data. Keep it a summary of what was ingested, not a computed liability.

# tax_pipeline/draft.py
from pydantic import BaseModel, Field
from google import genai
from tax_pipeline.schemas import W2, Form1099NEC

client = genai.Client()


class ReturnSummary(BaseModel):
    taxpayer: str
    tax_year: int
    income_sources: list[str] = Field(description="One line per source")
    total_box1_wages: int = Field(description="Sum of all W-2 Box 1 values")
    total_nonemployee_compensation: int
    total_federal_withholding: int = Field(description="Sum of all withholding boxes")
    documents_received: list[str]
    documents_missing: list[str] = Field(description="Forms not yet provided")
    notes_for_preparer: list[str] = Field(description="Questions for the preparer")


def draft_summary(w2s: list[W2], necs: list[Form1099NEC]) -> ReturnSummary:
    interaction = client.interactions.create(
        model=MODEL,
        input=(
            "You are preparing a draft summary for a tax preparer. "
            f"W-2s: {[w.model_dump() for w in w2s]}\n"
            f"1099-NECs: {[n.model_dump() for n in necs]}\n"
            "Summarize what was received, list obvious missing documents, and "
            "raise questions for the preparer. Do not compute tax owed."
        ),
        response_format={
            "type": "text",
            "mime_type": "application/json",
            "schema": ReturnSummary.model_json_schema(),
        },
    )
    return ReturnSummary.model_validate_json(interaction.output_text)

documents_missing and notes_for_preparer are what make this useful: an agent that only echoes what it received is a formatter, one that notices the 1099-B never arrived is doing the job a preparer wants delegated.

Step 4: The Review Pass With Function Calling

Here is the part that earns the “agent” label: the reviewer must not trust the drafter. It re-derives arithmetic itself and consults your source-of-truth functions. Function calling lets you declare functions with JSON Schema and let Gemini extract arguments and issue calls — the docs describe parallel, compositional multi-step calls with strict schema adherence.

# tax_pipeline/review.py
from pydantic import BaseModel, Field
from google import genai
from tax_pipeline.schemas import W2, Form1099NEC

client = genai.Client()

BOX1_BOX3_MAX_DELTA = 2000   # pre-tax benefits legitimately differ


def verify_arithmetic(kind: str, values: list[int]) -> dict:
    """Recompute a total from a list of extracted amounts."""
    return {"sum": sum(values), "count": len(values)}


TOOLS = [
    {
        "type": "function",
        "name": "verify_arithmetic",
        "description": "Recompute a total from a list of extracted amounts.",
        "parameters": {
            "type": "object",
            "properties": {
                "kind": {"type": "string", "description": "e.g. 'box1_total'"},
                "values": {"type": "array", "items": {"type": "integer"}},
            },
            "required": ["kind", "values"],
        },
    },
]


class Finding(BaseModel):
    severity: str = Field(description="one of: blocking, warning, info")
    field: str
    summary: str
    evidence: str = Field(description="The exact values that disagree")


class ReviewReport(BaseModel):
    passed: bool
    findings: list[Finding]
    totals_verified: dict[str, int] = Field(description="Totals recomputed by the reviewer")
    reviewer_notes: str


def run_review(w2s: list[W2], necs: list[Form1099NEC]) -> ReviewReport:
    interaction = client.interactions.create(
        model=MODEL,
        input=(
            "You are an independent reviewer. You did NOT extract this data. "
            "Re-verify every total yourself with verify_arithmetic, and confirm "
            f"no W-2 Box 1 and Box 3 differ by more than {BOX1_BOX3_MAX_DELTA}. "
            "Flag any mismatch as a finding. Do not silently correct anything.\n\n"
            f"W-2s: {[w.model_dump() for w in w2s]}\n"
            f"1099-NECs: {[n.model_dump() for n in necs]}"
        ),
        tools=TOOLS,
        response_format={
            "type": "text",
            "mime_type": "application/json",
            "schema": ReviewReport.model_json_schema(),
        },
    )

    # Execute any tool calls the model requested, then continue the interaction.
    for step in interaction.steps:
        if step.type == "function_call":
            result = verify_arithmetic(**step.arguments)
            interaction = client.interactions.continue_interaction(
                interaction, function_results={step.name: result}
            )

    return ReviewReport.model_validate_json(interaction.output_text)

The prompt matters twice over: “you did NOT extract this data” and “re-verify every total yourself”. Independence is the point — two passes sharing a conclusion prove nothing, while a drafter and a skeptic that disagree produce a finding worth a human’s time. Narrate arithmetic instead of calling tools, and you will ship a hallucinated sum with high confidence.

The exact continuation call for tool results varies by SDK version — check client.interactions in your installed google-genai. The pattern is fixed: create the interaction, inspect interaction.steps for function_call, execute locally, feed results back, parse the final output_text.

Step 5: Safety Settings and Blocked Requests

Tax documents contain SSNs, employer disputes, and sometimes sensitive personal narrative. Safety settings pair a harm category with a block threshold:

SAFETY = [
    {"type": "hate_speech", "threshold": "block_low_and_above"},
    {"type": "harassment", "threshold": "block_medium_and_above"},
]

interaction = client.interactions.create(
    model=MODEL,
    input=prompt,
    safety_settings=SAFETY,
    response_format={"type": "text", "mime_type": "application/json",
                     "schema": ReviewReport.model_json_schema()},
)

Know the failure mode: a filter that blocks input or response fails the request with HTTP 400 and error code of safety, jailbreak, prohibited_content, or blocklist — blocked content is never returned. Catch it explicitly instead of surfacing a stack trace:

from google.genai import errors

try:
    report = run_review(w2s, necs)
except errors.APIError as exc:
    if getattr(exc, "code", None) == 400 and "safety" in str(exc):
        report = ReviewReport(passed=False, findings=[], totals_verified={},
                              reviewer_notes="Blocked by safety filter; route to human review.")
    else:
        raise

Docs caveat: filters are Off by default on Gemini 2.5 and later, core harms (child safety) are always blocked and cannot be adjusted, and less restrictive settings may face Terms-of-Service review. Set only what you need — over-tightening blocks legitimate form text.

Putting It All Together

# tax_pipeline/main.py
from pathlib import Path
from tax_pipeline.extract import extract_w2
from tax_pipeline.draft import draft_summary
from tax_pipeline.review import run_review

forms = sorted(Path("incoming").glob("*.pdf"))

w2s = [extract_w2(pdf) for pdf in forms]      # stage 1: typed, schema-validated
summary = draft_summary(w2s, [])              # stage 2: human-readable draft
report = run_review(w2s, [])                  # stage 3: independent re-verification

print(summary.model_dump_json(indent=2))
print(report.model_dump_json(indent=2))

if not report.passed or any(f.severity == "blocking" for f in report.findings):
    raise SystemExit("Route to a human preparer — do not auto-file.")

A representative run prints the draft summary — taxpayer, income lines, documents_missing: ["1099-INT for savings interest"] — followed by the reviewer’s report:

{
  "passed": false,
  "findings": [
    {"severity": "blocking", "field": "wages_tips_other_compensation",
     "summary": "Recomputed Box 1 total disagrees with draft by 300",
     "evidence": "draft=84500 recomputed=84200"}
  ],
  "totals_verified": {"box1_total": 84200, "nec_total": 12000},
  "reviewer_notes": "Investigate before filing."
}

The blocking finding is the payoff: a discrepancy that used to be discovered after filing now surfaces before a human signs anything.

Conclusion and Next Steps

You built a three-stage pipeline: typed extraction from W-2/1099 PDFs with schema-constrained structured outputs, a preparer-facing draft, and an independent review that recomputes totals through its own function calls and refuses to pass a return with blocking findings. The not-tax-advice boundary is enforced in code by the “route to a human” exit.

Homework:

  • Add a third model role — an arbiter that adjudicates disagreements between drafter and reviewer, so only unresolved conflicts reach the preparer.
  • Build a golden set of synthetic forms with known answers and assert exact matches in CI; extraction regressions are otherwise silent.
  • Extend TOOLS with fetch_prior_year so the reviewer compares this year’s wages against the last filed return.
  • Persist every interaction to an audit log: for tax work, what the model saw and returned matters as much as the result.

Automation does not file your return. It makes sure the human who does signs off on correct numbers.