Reliable JSON from Any LLM: Pydantic + Zod Patterns for 2026

LLM structured output is the mechanism that guarantees a language model's response conforms to a predefined schema, not just valid JSON, but schema-valid JSON with the exact fields, types, and constraints you specified. Every major provider now supports it natively, and it's changed how production LLM applications get built.

This page owns the implementation patterns: schemas, constrained decoding, provider APIs, retries, Pydantic, and Zod. If your question is which abstraction to adopt, compare the eight structured-output libraries in our tool ranking.

Quick Summary: Structured Outputs at a Glance

If you're short on time, here's the landscape in 2026:

Now let's break down each piece.

What Are LLM Structured Outputs?

Structured output is the difference between hoping an LLM returns valid JSON and guaranteeing it. When you enable structured output, the model physically cannot produce tokens that violate your schema. You define a JSON Schema (or Pydantic model, or Zod schema), pass it to the API, and get back a response that matches it every single time.

Why does this matter? Before structured output, developers wrote fragile regex parsers, wrapped every LLM call in try/catch JSON.parse blocks, and still dealt with "almost right" responses, valid JSON that was missing a field or had the wrong type. That entire class of bug is gone.

There are three levels of structure enforcement, and they represent a clear evolution:

1. Prompt engineering, "Please return JSON with these fields." Unreliable. The model might comply 80-90% of the time.
2. JSON Mode, Guarantees syntactically valid JSON, but doesn't enforce your schema. You could get {"foo": "bar"} when you expected {"name": string, "age": number}.
3. Strict Mode / Constrained decoding, Guarantees 100% schema compliance. The model literally cannot output invalid tokens. This is what "structured output" means in 2026.

As of early 2026, OpenAI, Anthropic, and Google Gemini all support native structured output. The ecosystem has converged.

Verdict: If you're parsing LLM responses with regex or JSON.parse in production, you're doing it the hard way. Native structured output eliminates that entire failure mode.

JSON Mode vs Strict Mode: What Actually Changed?

This distinction trips up a lot of developers because the names sound similar. They're not.

The timeline: OpenAI introduced JSON Mode in late 2023. It was a step forward, but developers quickly realized "valid JSON" wasn't enough, they needed schema-valid JSON. In August 2024, OpenAI launched Structured Outputs with Strict Mode, which uses constrained decoding to guarantee schema compliance. By 2025-2026, every major provider had adopted the same approach.

JSON Mode still has a narrow use case: when you genuinely don't know the shape of the response ahead of time and just want some valid JSON for unstructured exploration. But that's rare in production.

Verdict: Use Strict Mode for everything in production. JSON Mode is effectively deprecated for schema-bound use cases. If you have a schema (and you should), use type: "json_schema" with strict: true.

How Does Constrained Decoding Actually Work?

Here's the mechanism that makes 100% schema compliance possible, not 99.9%, but literally 100%.

When you send a JSON Schema to a provider with Strict Mode enabled, the schema gets compiled into a finite state machine (FSM). This FSM represents every valid path through your schema. At each token generation step, the inference engine checks which tokens would keep the output on a valid path and which wouldn't. Invalid tokens get their logits set to negative infinity before sampling, which means they have zero probability of being selected.

Think of it like autocomplete on steroids. If the model has just output {"rating": and your schema says rating is an integer, the only tokens allowed next are digit tokens. Quotation marks, letters, brackets, all masked out. The model can't output "five" even if it "wants" to.

This is the same core mechanism used by XGrammar (the engine behind vLLM, SGLang, and most local inference servers) and Outlines (the open-source Python library for constrained generation). The API providers have just built it into their inference infrastructure.

There's one trade-off to know about: the first request with a new schema incurs a compilation latency hit (typically 50-200ms) while the FSM is built. Subsequent requests with the same schema use a cached FSM and add near-zero overhead. There's also a subtle quality consideration, constraining the token vocabulary can occasionally reduce output quality for creative or free-form fields, so keep your schemas focused on truly structured data.

Verdict: Constrained decoding is what separates "usually works" from "always works." It's the engineering that makes structured output production-ready.

Multi-Provider Implementation: OpenAI, Anthropic, and Gemini

Here's something none of the other guides show you: the same extraction task implemented across all three major providers. We'll extract a structured product review from unstructured text.

The Pydantic schema (shared across all providers):

from pydantic import BaseModel, Field
from typing import Literal

class ProductReview(BaseModel):
    reasoning: str = Field(description="Think through the review before scoring")
    rating: int = Field(description="Rating from 1-5", ge=1, le=5)
    sentiment: Literal["positive", "negative", "neutral"]
    pros: list[str] = Field(description="Key positive points")
    cons: list[str] = Field(description="Key negative points")
    summary: str = Field(description="One-sentence summary")

OpenAI Implementation

from openai import OpenAI

client = OpenAI()

response = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=[
        {"role": "system", "content": "Extract a structured review from the text."},
        {"role": "user", "content": review_text}
    ],
    response_format=ProductReview,  # Pydantic model directly
)

review = response.choices[0].message.parsed  # Typed ProductReview object

OpenAI's implementation is the most mature. The parse() method accepts a Pydantic model directly and returns a typed object. One constraint: OpenAI's Strict Mode supports a subset of JSON Schema, no $ref, limited anyOf, and all fields must be required with additionalProperties: false.

Anthropic Implementation

from anthropic import Anthropic

client = Anthropic()

response = client.messages.create(
    model="claude-sonnet-4-5-20250514",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": f"Extract a structured review:\n\n{review_text}"}
    ],
    output_config={
        "format": {
            "type": "json_schema",
            "json_schema": ProductReview.model_json_schema()
        }
    }
)

import json
review_data = json.loads(response.content[0].text)
review = ProductReview(**review_data)

Anthropic's native structured output uses output_config.format with a JSON Schema. It reached GA in early 2026. Anthropic also supports the older pattern of defining a "fake" tool and extracting via tool_use, that still works but native structured output is cleaner for pure extraction.

Gemini Implementation

from google import genai

client = genai.Client()

response = client.models.generate_content(
    model="gemini-2.0-flash",
    contents=f"Extract a structured review:\n\n{review_text}",
    config={
        "response_mime_type": "application/json",
        "response_schema": ProductReview,  # Pydantic model directly
    }
)

import json
review = ProductReview(**json.loads(response.text))

Gemini supports Pydantic models directly in the Python SDK via response_schema. A unique feature: Gemini respects propertyOrdering in the schema, so you can control field output order (useful for the reasoning-first pattern).

Provider Comparison

Verdict: OpenAI has the most polished DX with its parse() method. Anthropic offers the most capable underlying models. Gemini's property ordering is uniquely useful. All three get the job done, pick based on your existing provider relationship.

Pydantic Patterns for Python Developers

Pydantic is the de facto standard for defining structured output schemas in Python. Here are the patterns that matter.

Basic Schema with Descriptions

from pydantic import BaseModel, Field
from typing import Literal, Optional

class ExtractedEntity(BaseModel):
    reasoning: str = Field(description="Think step by step about the entity")
    name: str = Field(description="Full name of the entity")
    entity_type: Literal["person", "company", "location"]
    confidence: float = Field(description="Confidence score 0.0-1.0", ge=0.0, le=1.0)
    context: Optional[str] = Field(description="Surrounding context, if relevant")

Those description strings aren't just for documentation, they become part of the JSON Schema sent to the model and directly influence what the model generates. Think of them as prompt engineering within the schema.

Nested Models

class Address(BaseModel):
    street: str
    city: str
    country: str
    postal_code: Optional[str] = None

class Company(BaseModel):
    reasoning: str = Field(description="Analysis of the company details")
    name: str
    industry: Literal["tech", "finance", "healthcare", "retail", "other"]
    headquarters: Address  # Nested model
    key_products: list[str] = Field(description="Top 3 products or services")

Keep nesting to 2-3 levels max. Deeply nested schemas increase error rates and slow down schema compilation.

The Reasoning-First Pattern

This is the single most impactful schema design pattern. Put a reasoning field before your answer fields:

# Reliable JSON from Any LLM: Pydantic + Zod Patterns for 2026
class ClassificationBad(BaseModel):
    category: Literal["spam", "ham"]
    confidence: float

# Good -- model reasons through the problem first
class ClassificationGood(BaseModel):
    reasoning: str = Field(description="Analyze the text before classifying")
    category: Literal["spam", "ham"]
    confidence: float = Field(ge=0.0, le=1.0)

LLMs generate tokens left-to-right. If category comes first, the model picks a category and then rationalizes it. If reasoning comes first, the model works through the problem and then commits to a category. It's chain-of-thought baked into the schema.

JSON Schema Export

# Generate the JSON Schema for any Pydantic model
schema = ProductReview.model_json_schema()
# Pass this to any provider that accepts raw JSON Schema

Verdict: Pydantic + descriptive fields + reasoning-first ordering is the Python structured output trifecta. Master these three patterns and you'll handle 90% of use cases.

Zod Patterns for TypeScript Developers

Zod is the TypeScript equivalent of Pydantic, and it's just as central to structured output workflows.

Basic Schema with Descriptions

import { z } from "zod";

const ProductReview = z.object({
  reasoning: z.string().describe("Think through the review before scoring"),
  rating: z.number().int().min(1).max(5),
  sentiment: z.enum(["positive", "negative", "neutral"]),
  pros: z.array(z.string()).describe("Key positive points"),
  cons: z.array(z.string()).describe("Key negative points"),
  summary: z.string().describe("One-sentence summary"),
});

// Infer the TypeScript type automatically
type ProductReview = z.infer<typeof ProductReview>;

Like Pydantic's Field(description=...), Zod's .describe() becomes part of the JSON Schema and guides the model's output.

Integration with OpenAI Node SDK

import OpenAI from "openai";
import { zodResponseFormat } from "openai/helpers/zod";

const client = new OpenAI();

const response = await client.beta.chat.completions.parse({
  model: "gpt-4o-2024-08-06",
  messages: [
    { role: "system", content: "Extract a structured review." },
    { role: "user", content: reviewText },
  ],
  response_format: zodResponseFormat(ProductReview, "product_review"),
});

const review = response.choices[0].message.parsed; // Typed!

Integration with Vercel AI SDK

import { generateObject } from "ai";
import { openai } from "@ai-sdk/openai";

const { object: review } = await generateObject({
  model: openai("gpt-4o"),
  schema: ProductReview,
  prompt: `Extract a structured review:\n\n${reviewText}`,
});
// review is fully typed as ProductReview

The Vercel AI SDK uses Zod natively with generateObject(), making it the cleanest TypeScript integration. It works with OpenAI, Anthropic, Gemini, and other providers through a unified API.

JSON Schema Conversion

import { zodToJsonSchema } from "zod-to-json-schema";

const jsonSchema = zodToJsonSchema(ProductReview);
// Use with any provider that accepts raw JSON Schema

Verdict: Zod + .describe() + the Vercel AI SDK is the TypeScript structured output stack. If you're in the Node/Next.js ecosystem, this is the path of least resistance.

Structured Output vs Function Calling: When Do You Use Each?

This is one of the most common sources of confusion. Both involve schemas, both return structured data, but they solve different problems.

Structured output says: "Give me data in this exact shape." It's for extraction, classification, and formatting. You're pulling structured information out of unstructured text.

Function calling (tool use) says: "Here are actions you can take, decide which one to run and provide the arguments." It's for agent workflows where the model picks from multiple tools and triggers actions.

The confusion makes sense historically. Anthropic's original "structured output" was literally function calling, you'd define a fake tool called extract_review and grab the arguments. That still works, but native structured output is simpler for pure extraction.

Structured output powers the tool-calling pipelines in AI agent systems. See our guide to AI agents for business for how these fit into production workflows.

Verdict: Use structured output when you know what shape the data should be. Use function calling when the model needs to choose an action. In practice, most applications use both, structured output for data extraction and function calling for agent orchestration.

Production Patterns: Errors, Retries, and Streaming

Getting structured output working in a demo is easy. Keeping it reliable in production requires handling three things: refusals, validation failures, and streaming.

Refusal Handling

Sometimes a model refuses to generate your requested output, typically because safety filters flagged the input. When this happens, structured output APIs don't return your schema. They return a refusal.

response = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=messages,
    response_format=ProductReview,
)

# ALWAYS check for refusal before accessing parsed content
if response.choices[0].message.refusal:
    print(f"Model refused: {response.choices[0].message.refusal}")
else:
    review = response.choices[0].message.parsed

If you skip the refusal check and try to access .parsed on a refusal, you'll get None and a confusing downstream error. Check first, always.

Retry Patterns with Validation Feedback

Schema compliance is guaranteed by constrained decoding, but semantic correctness isn't. The model might return {"rating": 1, "sentiment": "positive"}, valid schema, contradictory content. That's where validation + retries come in.

import instructor

client = instructor.from_openai(OpenAI())

# Instructor handles retries automatically
review = client.chat.completions.create(
    model="gpt-4o",
    response_model=ProductReview,
    max_retries=3,  # Retries with validation error feedback
    messages=[
        {"role": "user", "content": review_text}
    ],
)

Instructor feeds the validation error back to the model on retry, so it can self-correct. For manual retry patterns without Instructor:

from pydantic import ValidationError

for attempt in range(3):
    try:
        response = client.beta.chat.completions.parse(
            model="gpt-4o-2024-08-06",
            messages=messages,
            response_format=ProductReview,
        )
        review = response.choices[0].message.parsed
        # Run additional semantic validation here
        break
    except ValidationError as e:
        messages.append({"role": "assistant", "content": str(response)})
        messages.append({"role": "user", "content": f"Validation error: {e}. Fix it."})

Streaming Structured Output

For large structured responses, long arrays, many fields, complex nested objects, streaming lets you render partial results progressively.

import instructor

client = instructor.from_openai(OpenAI())

# Stream partial results as fields populate
review_stream = client.chat.completions.create_partial(
    model="gpt-4o",
    response_model=ProductReview,
    messages=[{"role": "user", "content": review_text}],
)

for partial_review in review_stream:
    # Fields populate one by one as tokens stream in
    if partial_review.summary:
        print(f"Summary so far: {partial_review.summary}")

One gotcha: individual streaming chunks aren't schema-valid on their own. The reasoning field might be populated while rating is still None. Plan your UI accordingly, show a loading state for unpopulated fields.

Verdict: Refusal checks are non-negotiable. Retries with validation feedback catch semantic errors. Streaming is worth it for any response that takes more than a couple seconds.

Structured Output Libraries Compared

You can use structured output through native APIs, but libraries add validation, retries, streaming, and multi-provider support. Here's the landscape.

Instructor is the most popular option at 11K+ GitHub stars and 3M+ monthly downloads. It wraps OpenAI, Anthropic, Gemini, Cohere, Ollama, and more with a unified Pydantic-based interface. Key features: automatic retries with validation feedback, streaming via create_partial(), and dead-simple setup (instructor.from_openai(client)). If you're a Python team, start here.

BAML takes a different approach: schema-first via a custom DSL. You define schemas in .baml files and auto-generate clients for Python, TypeScript, Ruby, and more. Its SAP (schema-aligned parsing) algorithm handles messy model outputs gracefully. Best for cross-language teams or when you want contracts between your LLM layer and application layer. Trade-off: extra build step and a new syntax to learn.

LangChain offers .with_structured_output(schema) for provider-agnostic structured output. Convenient if you're already in the LangChain ecosystem. Trade-off: it's a heavy dependency, and the abstraction can hide provider-specific features you might need.

Native APIs, direct calls with response_format / output_config, require zero dependencies beyond the provider SDK. You get full control and full visibility. Best for simple use cases or teams that prefer minimal abstraction.

Choosing the right structured output library is part of a broader AI stack decision. We break down the full stack in our Best AI Stack for SaaS guide.

See our Best Libraries for LLM Structured Outputs [coming soon] for an in-depth comparison of Instructor, BAML, Mirascope, and more.

Verdict: Start with Instructor for Python, native APIs for TypeScript. Move to BAML if you need cross-language schema contracts. Avoid LangChain just for structured output, it's overkill.

Schema Design Best Practices (and Common Mistakes)

Your schema design directly impacts output quality. Here are the patterns that matter and the mistakes that cost you accuracy.

Put Reasoning Before Answers

We covered this in the Pydantic section, but it bears repeating because it's the highest-impact design decision:

# Before: model guesses the answer, then rationalizes
class Bad(BaseModel):
    answer: str
    reasoning: str

# After: model thinks first, then commits
class Good(BaseModel):
    reasoning: str = Field(description="Think step by step")
    answer: str

LLMs generate left-to-right. Field order is prompt order. Reasoning first means the model has to work through the problem before committing to an answer.

The Anti-Pattern Table

Handle Nulls Explicitly

If a field might not have data in the source text, make it optional. Forcing a required field when data doesn't exist leads to hallucination:

class PersonInfo(BaseModel):
    name: str  # Always present
    email: Optional[str] = Field(None, description="Email if mentioned, null otherwise")
    phone: Optional[str] = Field(None, description="Phone if mentioned, null otherwise")

Keep Schemas Focused

One schema per task. Don't try to extract everything in a single massive schema. If you need 50+ fields, split into multiple extraction calls. OpenAI's Strict Mode has practical limits on schema complexity, and even when it works, very large schemas degrade output quality.

Verdict: Reasoning-first, descriptive fields, explicit nulls, and focused schemas. Get these four right and your structured output accuracy jumps measurably.

Structured Output with Local LLMs

You don't need an API provider for structured output. Local inference engines support it through grammar-based constrained decoding, the same fundamental mechanism, running on your own hardware.

Ollama

The easiest path for local structured output. Ollama accepts a JSON Schema via the format parameter:

import ollama
from pydantic import BaseModel

class Country(BaseModel):
    name: str
    capital: str
    languages: list[str]

response = ollama.chat(
    model="llama3.2",
    messages=[{"role": "user", "content": "Tell me about Japan."}],
    format=Country.model_json_schema(),
)

import json
country = Country(**json.loads(response.message.content))

Ollama uses XGrammar under the hood for constrained decoding. Same guarantee as the API providers: 100% schema compliance.

vLLM and SGLang

For production-grade local inference, vLLM and SGLang both support structured output through guided_json and guided_regex parameters. XGrammar is the default backend, delivering near-zero overhead on JSON generation, up to 3.5x faster than alternative grammar engines.

Outlines

Outlines is the open-source Python library that pioneered grammar-based constrained generation. It works with any Hugging Face model and supports JSON Schema, regex, and full context-free grammar (CFG/EBNF) constraints. It's also integrated into vLLM and SGLang as a grammar backend option.

The key difference from API providers: local structured output has no schema subset limitations. You control the grammar entirely. But model quality varies more, a 7B parameter local model won't match GPT-4o or Claude on complex extraction tasks. The schema will always be valid; the content quality depends on the model.

Verdict: Ollama for development, vLLM/SGLang with XGrammar for production. Local structured output is mature enough for most use cases, with the caveat that smaller models produce lower-quality content within the schema.

FAQ

What is structured output in LLMs?

Structured output is a mechanism that guarantees an LLM's response conforms to a predefined JSON Schema. Unlike plain text or even JSON Mode, structured output uses constrained decoding to ensure every field, type, and constraint in your schema is met -- 100% of the time, not "usually."

What is the difference between JSON Mode and Structured Outputs?

JSON Mode guarantees syntactically valid JSON but doesn't enforce your schema, you could get any valid JSON object. Structured Outputs (Strict Mode) guarantees full schema compliance through constrained decoding. Use Strict Mode for production; JSON Mode is only relevant when you don't have a schema upfront.

Which LLM providers support structured output natively?

OpenAI (since August 2024), Google Gemini (2024, expanded 2026), Anthropic (beta November 2025, GA early 2026), Cohere, and xAI (Grok) all support native structured output. On the local side, Ollama, vLLM, and SGLang support it through grammar-based constrained decoding.

How does constrained decoding guarantee schema compliance?

The JSON Schema is compiled into a finite state machine (FSM). At each token generation step, only tokens that keep the output on a valid path through the FSM are allowed, invalid tokens get their logits set to negative infinity. This means invalid tokens have zero probability of being generated, giving you a mathematical guarantee, not a statistical one.

Should I use structured output or function calling?

Use structured output for extraction and classification, when you want data in a specific shape. Use function calling for agent workflows, when the model needs to decide which action to take. Many production applications use both: structured output for data extraction and function calling for orchestration.

Can I stream structured output?

Yes. OpenAI supports streaming with the parse() method, and Instructor provides create_partial() for streaming Pydantic models that populate field-by-field. Keep in mind that individual streaming chunks aren't individually schema-valid, fields populate incrementally.

What is the Instructor library?

Instructor is the most popular structured output library (11K+ GitHub stars, 3M+ monthly downloads). It wraps provider SDKs with Pydantic-based validation, automatic retries with validation feedback, and streaming support. It works with OpenAI, Anthropic, Gemini, Cohere, Ollama, and 10+ other providers.

Does structured output work with local LLMs?

Yes. Ollama supports structured output via the format parameter with JSON Schema. vLLM and SGLang support it through guided_json parameters. All three use XGrammar or Outlines for constrained decoding. The schema compliance guarantee is the same as API providers; content quality depends on the model.

What are common schema design mistakes?

The top mistakes: putting the reasoning field after the answer field (model decides before thinking), deeply nested schemas (4+ levels increase errors), missing field descriptions (model guesses intent), no null handling for optional data (forces hallucination), and overly large schemas (50+ fields degrade quality).

Does structured output add latency?

There's a schema compilation overhead on the first request, typically 50-200ms while the FSM is built. Subsequent requests with the same schema use a cached FSM and add near-zero latency. For most applications, this is negligible compared to the overall model inference time.

Can I use structured output with images or multimodal inputs?

Yes. Structured output applies to the response format, not the input. You can send an image to GPT-4o or Gemini with a structured output schema and get back a schema-compliant analysis of the image. This is powerful for visual extraction workflows, extracting structured data from receipts, forms, or product images.

Sources

• OpenAI Structured Outputs Guide
• Anthropic Tool Use Documentation
• Google Gemini Structured Output
• Instructor Library Documentation
• BAML Documentation
• Pydantic Documentation
• Zod Documentation
• Outlines Library
• XGrammar GitHub
• Ollama Structured Outputs
• Vercel AI SDK