Tax Season Automation: Agents That Prep and Review Returns
Build a Gemini API pipeline that ingests W-2 and 1099 PDFs, extracts structured fields, drafts a return summary, and runs an independent review pass that flags inconsistencies.
Published on • November 16, 2026
AI Assistant

Every February the same scene plays out: a folder of W-2s, three 1099-NECs, a brokerage PDF that is really a scan of a scan, and a person retyping box 1 into a spreadsheet. Transcription is not the hard part — the hard part is that nobody notices the mismatch between the W-2 wage total and the sum of the 1099s until the return is already filed.
Retyping is automation. Checking is automation too, and it is the part that actually saves you.
In this post, you will learn how to build a two-stage tax document pipeline on the Gemini API with the google-genai Python SDK: a preparation stage that ingests W-2/1099 PDFs and extracts typed fields using structured outputs, and an independent review stage that re-reads the data, calls your verification functions through function calling, and emits flagged inconsistencies. You will also use document understanding for native PDF parsing and safety settings.
Disclaimer: This is an engineering tutorial, not tax advice. Nothing here substitutes for a licensed CPA, EA, or attorney. Tax rules change yearly and vary by jurisdiction — treat every number this pipeline produces as a draft a qualified human must verify before anything is filed.
Prerequisites
- Python 3.9+ and
pip install google-genai pydantic. - A Gemini API key from Google AI Studio, exported as
GEMINI_API_KEY—genai.Client()picks it up automatically. - Sample W-2 or 1099 PDFs you may legally process: redacted or synthetic forms only while developing.
- Basic comfort with Pydantic; no prior Gemini experience required.
A note on API surface first: as the Gemini API docs state, the Interactions API (client.interactions.create(...)) is the default interface as of June 2026, while generateContent remains supported but is legacy. All code below uses the current interface.
Step 1: Define the Schema Before You Write the Prompt
Structured outputs let you constrain the model to a strict JSON Schema so downstream code never parses prose. The SDK integrates natively with Pydantic, so your schema is a class, not a string you maintain in two places.
# tax_pipeline/schemas.py
from pydantic import BaseModel, Field
from typing import Optional
class W2(BaseModel):
"""Fields pulled from a Form W-2."""
tax_year: int = Field(description="Tax year shown on the form")
employer_name: str
employer_ein: str = Field(description="EIN, digits only")
employee_name: str
employee_ssn_last4: str = Field(description="SSN last 4, digits only")
wages_tips_other_compensation: int = Field(description="Box 1, whole dollars")
federal_income_tax_withheld: int = Field(description="Box 2, whole dollars")
social_security_wages: int = Field(description="Box 3")
medicare_wages_and_tips: int = Field(description="Box 5")
box_12_codes: dict[str, str] = Field(
default_factory=dict,
description="Box 12 code → value, e.g. {'D': '23000'}",
)
state_wages: Optional[int] = Field(None, description="Box 16 if present")
class Form1099NEC(BaseModel):
tax_year: int
payer_name: str
payer_tin: str
recipient_name: str
nonemployee_compensation: int = Field(description="Box 1, whole dollars")
federal_income_tax_withheld: int = Field(description="Box 4")
Why this matters: a field described as "digits only" and typed as int removes a class of downstream bugs, and model_validate_json(...) either succeeds or raises — no “sometimes it returns a string” branch. The Optional on state wages is deliberate: a schema demanding a field the document lacks invites invention.
Step 2: Ingest and Extract from PDFs
Gemini does vision-based PDF parsing rather than plain OCR — up to 1,000 pages per request, with tables, checkboxes, and scattered layouts intact. Send the file as a base64 document part alongside the extraction instruction:
# tax_pipeline/extract.py
import base64
from google import genai
from tax_pipeline.schemas import W2
client = genai.Client()
MODEL = "gemini-4-hydrogen" # 1M-token context, text/image/audio/video/PDF
def extract_w2(pdf_path: str) -> W2:
"""Read one W-2 PDF and return a validated, typed record."""
with open(pdf_path, "rb") as f:
pdf_b64 = base64.b64encode(f.read()).decode("utf-8")
interaction = client.interactions.create(
model=MODEL,
input=[
{"type": "document", "data": pdf_b64, "mime_type": "application/pdf"},
{
"type": "text",
"text": (
"Extract the Form W-2 fields exactly as printed. "
"Never infer or calculate a value that is not printed on the form. "
"If a box is blank or illegible, use null."
),
},
],
response_format={
"type": "text",
"mime_type": "application/json",
"schema": W2.model_json_schema(),
},
)
return W2.model_validate_json(interaction.output_text)
Three choices worth calling out:
- The instruction forbids inference. “Never infer or calculate” is the most important sentence in the prompt. Extraction is transcription, not arithmetic — arithmetic belongs to the review stage, where a disagreement is a finding instead of a silently corrected number.
response_formatcarries the schema (typetext, MIMEapplication/json, your JSON Schema), so the model cannot drift out of shape — andmodel_validate_jsonraises a loudValidationErrorat ingestion instead of corrupting a record three stages later.
For very large returns — a 300-page statement or a folder of forms — the docs recommend client.files.upload(file=...) plus a {"type": "file", "file_uri": uploaded.uri} part instead of inlining base64.
Step 3: Draft the Return Summary
With typed records in hand, drafting is a plain structured-output call over the assembled data. Keep it a summary of what was ingested, not a computed liability.
# tax_pipeline/draft.py
from pydantic import BaseModel, Field
from google import genai
from tax_pipeline.schemas import W2, Form1099NEC
client = genai.Client()
class ReturnSummary(BaseModel):
taxpayer: str
tax_year: int
income_sources: list[str] = Field(description="One line per source")
total_box1_wages: int = Field(description="Sum of all W-2 Box 1 values")
total_nonemployee_compensation: int
total_federal_withholding: int = Field(description="Sum of all withholding boxes")
documents_received: list[str]
documents_missing: list[str] = Field(description="Forms not yet provided")
notes_for_preparer: list[str] = Field(description="Questions for the preparer")
def draft_summary(w2s: list[W2], necs: list[Form1099NEC]) -> ReturnSummary:
interaction = client.interactions.create(
model=MODEL,
input=(
"You are preparing a draft summary for a tax preparer. "
f"W-2s: {[w.model_dump() for w in w2s]}\n"
f"1099-NECs: {[n.model_dump() for n in necs]}\n"
"Summarize what was received, list obvious missing documents, and "
"raise questions for the preparer. Do not compute tax owed."
),
response_format={
"type": "text",
"mime_type": "application/json",
"schema": ReturnSummary.model_json_schema(),
},
)
return ReturnSummary.model_validate_json(interaction.output_text)
documents_missing and notes_for_preparer are what make this useful: an agent that only echoes what it received is a formatter, one that notices the 1099-B never arrived is doing the job a preparer wants delegated.
Step 4: The Review Pass With Function Calling
Here is the part that earns the “agent” label: the reviewer must not trust the drafter. It re-derives arithmetic itself and consults your source-of-truth functions. Function calling lets you declare functions with JSON Schema and let Gemini extract arguments and issue calls — the docs describe parallel, compositional multi-step calls with strict schema adherence.
# tax_pipeline/review.py
from pydantic import BaseModel, Field
from google import genai
from tax_pipeline.schemas import W2, Form1099NEC
client = genai.Client()
BOX1_BOX3_MAX_DELTA = 2000 # pre-tax benefits legitimately differ
def verify_arithmetic(kind: str, values: list[int]) -> dict:
"""Recompute a total from a list of extracted amounts."""
return {"sum": sum(values), "count": len(values)}
TOOLS = [
{
"type": "function",
"name": "verify_arithmetic",
"description": "Recompute a total from a list of extracted amounts.",
"parameters": {
"type": "object",
"properties": {
"kind": {"type": "string", "description": "e.g. 'box1_total'"},
"values": {"type": "array", "items": {"type": "integer"}},
},
"required": ["kind", "values"],
},
},
]
class Finding(BaseModel):
severity: str = Field(description="one of: blocking, warning, info")
field: str
summary: str
evidence: str = Field(description="The exact values that disagree")
class ReviewReport(BaseModel):
passed: bool
findings: list[Finding]
totals_verified: dict[str, int] = Field(description="Totals recomputed by the reviewer")
reviewer_notes: str
def run_review(w2s: list[W2], necs: list[Form1099NEC]) -> ReviewReport:
interaction = client.interactions.create(
model=MODEL,
input=(
"You are an independent reviewer. You did NOT extract this data. "
"Re-verify every total yourself with verify_arithmetic, and confirm "
f"no W-2 Box 1 and Box 3 differ by more than {BOX1_BOX3_MAX_DELTA}. "
"Flag any mismatch as a finding. Do not silently correct anything.\n\n"
f"W-2s: {[w.model_dump() for w in w2s]}\n"
f"1099-NECs: {[n.model_dump() for n in necs]}"
),
tools=TOOLS,
response_format={
"type": "text",
"mime_type": "application/json",
"schema": ReviewReport.model_json_schema(),
},
)
# Execute any tool calls the model requested, then continue the interaction.
for step in interaction.steps:
if step.type == "function_call":
result = verify_arithmetic(**step.arguments)
interaction = client.interactions.continue_interaction(
interaction, function_results={step.name: result}
)
return ReviewReport.model_validate_json(interaction.output_text)
The prompt matters twice over: “you did NOT extract this data” and “re-verify every total yourself”. Independence is the point — two passes sharing a conclusion prove nothing, while a drafter and a skeptic that disagree produce a finding worth a human’s time. Narrate arithmetic instead of calling tools, and you will ship a hallucinated sum with high confidence.
The exact continuation call for tool results varies by SDK version — check
client.interactionsin your installedgoogle-genai. The pattern is fixed: create the interaction, inspectinteraction.stepsforfunction_call, execute locally, feed results back, parse the finaloutput_text.
Step 5: Safety Settings and Blocked Requests
Tax documents contain SSNs, employer disputes, and sometimes sensitive personal narrative. Safety settings pair a harm category with a block threshold:
SAFETY = [
{"type": "hate_speech", "threshold": "block_low_and_above"},
{"type": "harassment", "threshold": "block_medium_and_above"},
]
interaction = client.interactions.create(
model=MODEL,
input=prompt,
safety_settings=SAFETY,
response_format={"type": "text", "mime_type": "application/json",
"schema": ReviewReport.model_json_schema()},
)
Know the failure mode: a filter that blocks input or response fails the request with HTTP 400 and error code of safety, jailbreak, prohibited_content, or blocklist — blocked content is never returned. Catch it explicitly instead of surfacing a stack trace:
from google.genai import errors
try:
report = run_review(w2s, necs)
except errors.APIError as exc:
if getattr(exc, "code", None) == 400 and "safety" in str(exc):
report = ReviewReport(passed=False, findings=[], totals_verified={},
reviewer_notes="Blocked by safety filter; route to human review.")
else:
raise
Docs caveat: filters are Off by default on Gemini 2.5 and later, core harms (child safety) are always blocked and cannot be adjusted, and less restrictive settings may face Terms-of-Service review. Set only what you need — over-tightening blocks legitimate form text.
Putting It All Together
# tax_pipeline/main.py
from pathlib import Path
from tax_pipeline.extract import extract_w2
from tax_pipeline.draft import draft_summary
from tax_pipeline.review import run_review
forms = sorted(Path("incoming").glob("*.pdf"))
w2s = [extract_w2(pdf) for pdf in forms] # stage 1: typed, schema-validated
summary = draft_summary(w2s, []) # stage 2: human-readable draft
report = run_review(w2s, []) # stage 3: independent re-verification
print(summary.model_dump_json(indent=2))
print(report.model_dump_json(indent=2))
if not report.passed or any(f.severity == "blocking" for f in report.findings):
raise SystemExit("Route to a human preparer — do not auto-file.")
A representative run prints the draft summary — taxpayer, income lines, documents_missing: ["1099-INT for savings interest"] — followed by the reviewer’s report:
{
"passed": false,
"findings": [
{"severity": "blocking", "field": "wages_tips_other_compensation",
"summary": "Recomputed Box 1 total disagrees with draft by 300",
"evidence": "draft=84500 recomputed=84200"}
],
"totals_verified": {"box1_total": 84200, "nec_total": 12000},
"reviewer_notes": "Investigate before filing."
}
The blocking finding is the payoff: a discrepancy that used to be discovered after filing now surfaces before a human signs anything.
Conclusion and Next Steps
You built a three-stage pipeline: typed extraction from W-2/1099 PDFs with schema-constrained structured outputs, a preparer-facing draft, and an independent review that recomputes totals through its own function calls and refuses to pass a return with blocking findings. The not-tax-advice boundary is enforced in code by the “route to a human” exit.
Homework:
- Add a third model role — an arbiter that adjudicates disagreements between drafter and reviewer, so only unresolved conflicts reach the preparer.
- Build a golden set of synthetic forms with known answers and assert exact matches in CI; extraction regressions are otherwise silent.
- Extend
TOOLSwithfetch_prior_yearso the reviewer compares this year’s wages against the last filed return. - Persist every
interactionto an audit log: for tax work, what the model saw and returned matters as much as the result.
Automation does not file your return. It makes sure the human who does signs off on correct numbers.