Building Guardrails for LLM Outputs

An LLM is not a function with a contract — it is a predictor. It will occasionally return an invalid shape, a hallucinated number, or a value that breaks your downstream system. If you accept whatever the model emits, every response is unverified output that snowballs into broken automation.

The fix is a distinct, named verification layer: guardrails. You define the shape and the rules the output must satisfy before the model runs, and you decide — deterministically — what happens when it fails. This tutorial builds that layer with Guardrails AI and Gemini’s schema-constrained generation. — “Do not rely on hope. Human review, fact checking, evaluation, and testing must be designed as explicit steps in every workflow that involves AI.” A guardrail is hope’s replacement in code.

Prerequisites

• pip install "guardrails-ai" and pip install -U google-genai.
• GEMINI_API_KEY in the environment. Gemini supports strict structured outputs via response_mime_type="application/json" plus a response_schema; Guardrails wraps any Guard with validators and an on-fail behavior.

Step 1: Trust the API constraint, but verify anyway

Gemini can enforce a JSON Schema, but as the docs note, JSON mode without a schema is only a strong hint. Belt-and-suspenders: enforce schema server-side, then validate semantics client-side.

from google import genai
from google.genai import types
from pydantic import BaseModel

class Ticket(BaseModel):
    summary: str
    priority: str        # must be low|medium|high
    estimated_hours: int # clamped to 0..40

client = genai.Client()

resp = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Extract a support ticket: 'DB down, rebooting nightly, maybe 8 hours to patch'",
    config=types.GenerateContentConfig(
        response_mime_type="application/json",
        response_schema=Ticket,
    ),
)
print(resp.parsed or resp.text)

Schema enforcement returns valid JSON, but nothing checks that priority is one of the three enum values or that estimated_hours isn’t -5.

Step 2: Add semantic validators with Guardrails

Guardrails’ unit of logic is a Validator; a Guard wraps validators and applies an on-fail action (reask, fix, filter, exception, noop).

from guardrails import Guard, OnFailAction
from guardrails_ai.valid_choice import ValidChoice
from guardrails_ai.valid_range import ValidRange

guard = (
    Guard().for_pydantic(Ticket)
    .use(ValidChoice(choices=["low", "medium", "high"], on_fail=OnFailAction.FIX))
    .use(ValidRange(lo=0, hi=40, on_fail=OnFailAction.FIX))
)

A deterministic FIX (pick the allowed value, clamp the range) is preferable to a reask, because a reask costs another model call — token spend and latency you can measure.

Step 3: Validate real output and handle failures

raw = {
    "summary": "Rebrand nightly, database down",
    "priority": "urgent",        # not in allowed choices
    "estimated_hours": 200,      # out of range
}

outcome = guard.validate(raw)
if outcome.validation_passed:
    print(outcome.validated_output)
else:
    # log and route: escalate for human review, don't silently pass bad data
    print(outcome.validated_output)  # priority fixed, hours clamped

Step 4: Guardrail the input too

OWASP calls symmetric defense insecure output handling (LLM02) — but the cheapest fix upstream is input validation. Reject hostile or out-of-domain input before it reaches tokens.

import json

def input_allowed(prompt: str) -> bool:
    return len(prompt) < 4000 and not prompt.startswith("IGNORE_SYSTEM")

def route(prompt, guard, client):
    if not input_allowed(prompt):
        return {"error": "blocked"}
    text = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=prompt,
        config=types.GenerateContentConfig(response_mime_type="application/json"),
    ).text
    return guard.validate(json.loads(text))

Putting It All Together

The pattern: ask Gemini for schema-constrained JSON, then run it through a semantics Guard with bounded reasks so the rule set and the model negotiate a fix — but never unboundedly, since each reask is a billable call.

def safe_extract(prompt: str) -> Ticket:
    guard = (
        Guard().for_pydantic(Ticket)
        .use(ValidChoice(choices=["low", "medium", "high"], on_fail=OnFailAction.REASK))
        .use(ValidRange(lo=0, hi=40, on_fail=OnFailAction.FIX))
    )
    raw = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=prompt,
        config=types.GenerateContentConfig(
            response_mime_type="application/json",
            response_schema=Ticket,
        ),
    ).text
    return guard.validate(json.loads(raw)).validated_output

References / Sources

• Guardrails AI. https://github.com/guardrails-ai/guardrails
• Gemini structured outputs. https://ai.google.dev/gemini-api/docs/structured-output
• Beyond AI: An Executive Brief for C-Level Leaders: https://blog.redlinesoft.net/posts/beyond-ai-executive-brief/