OpenAI Structured Outputs: Using response_format with JSON Schema

What Are OpenAI Structured Outputs?

OpenAI's structured outputs feature guarantees that the model's response conforms to a JSON Schema you provide. Unlike asking the model to "please return JSON" in a system prompt (which works most of the time), structured outputs use constrained decoding to make schema conformance a hard guarantee, not a best effort.

This matters in production systems where a single malformed response can crash a pipeline, corrupt data, or trigger expensive retry logic.

The response_format Parameter

There are two modes for structured JSON output:

1. json_object — the model returns valid JSON, but you have no schema control
2. json_schema — the model returns JSON that conforms to your exact schema

Always prefer json_schema mode. Here is a basic example:

from openai import OpenAI

client = OpenAI()

response = client.chat.completions.create(
    model="gpt-4o-2024-08-06",
    messages=[
        {
            "role": "system",
            "content": "Extract product information from the user's text."
        },
        {
            "role": "user",
            "content": "The new iPhone 16 Pro costs $999 and has 256GB storage."
        }
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "product_extraction",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "product_name": {"type": "string"},
                    "price_usd": {"type": "number"},
                    "storage_gb": {"type": "integer"},
                    "category": {
                        "type": "string",
                        "enum": ["smartphone", "laptop", "tablet", "accessory"]
                    }
                },
                "required": ["product_name", "price_usd", "storage_gb", "category"],
                "additionalProperties": False
            }
        }
    }
)

import json
product = json.loads(response.choices[0].message.content)
print(product)
# {"product_name": "iPhone 16 Pro", "price_usd": 999, "storage_gb": 256, "category": "smartphone"}

Strict Mode

When strict: True is set, OpenAI enforces additional constraints:

• All fields listed in properties must be in required
• additionalProperties must be False at every object level
• Optional fields must use a union type with null: {"type": ["string", "null"]}

Strict mode enables constrained decoding, meaning the token generation process itself is constrained to only produce valid schema-conforming output. Without strict mode, the model can still occasionally produce output that does not match.

Handling Optional Fields

Since strict mode requires all properties in the required array, use nullable types for optional fields:

schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "email": {"type": ["string", "null"]},
        "phone": {"type": ["string", "null"]},
    },
    "required": ["name", "email", "phone"],
    "additionalProperties": False
}

The model will return null for fields it cannot extract, rather than hallucinating values.

Nested and Complex Schemas

Structured outputs support deeply nested schemas. Define reusable components using $defs:

schema = {
    "type": "object",
    "properties": {
        "company": {"type": "string"},
        "employees": {
            "type": "array",
            "items": {"$ref": "#/$defs/Employee"}
        }
    },
    "required": ["company", "employees"],
    "additionalProperties": False,
    "$defs": {
        "Employee": {
            "type": "object",
            "properties": {
                "name": {"type": "string"},
                "role": {"type": "string"},
                "department": {
                    "type": "string",
                    "enum": ["engineering", "sales", "marketing", "operations"]
                }
            },
            "required": ["name", "role", "department"],
            "additionalProperties": False
        }
    }
}

Handling Refusals

When the model refuses a request (due to safety filters), the response includes a refusal field instead of content:

response = client.chat.completions.create(
    model="gpt-4o-2024-08-06",
    messages=[{"role": "user", "content": "some problematic request"}],
    response_format={"type": "json_schema", "json_schema": my_schema}
)

message = response.choices[0].message

if message.refusal:
    print(f"Model refused: {message.refusal}")
else:
    data = json.loads(message.content)
    process_data(data)

Always check for refusals before parsing content. Attempting to parse a None content field is a common bug in production code.

Using the Pydantic Helper

OpenAI's Python SDK includes a parse method that combines schema generation and response parsing:

from pydantic import BaseModel
from typing import List

class ProductInfo(BaseModel):
    product_name: str
    price_usd: float
    features: List[str]

response = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=[
        {"role": "user", "content": "Tell me about the MacBook Air M3."}
    ],
    response_format=ProductInfo,
)

product = response.choices[0].message.parsed
print(product.product_name)   # Typed access, no json.loads needed
print(product.price_usd)

The parse method automatically generates the JSON schema from your Pydantic model, sends it to the API, and deserializes the response back into a typed Python object.

FAQ

What models support structured outputs with json_schema?

As of early 2026, gpt-4o-2024-08-06 and later snapshots, gpt-4o-mini, and the o1 family all support json_schema response format. Older models like gpt-4-turbo only support the weaker json_object mode.

Is there a token overhead for structured outputs?

Yes, the schema is included in the request and counts toward your input tokens. Complex schemas with many nested objects and enums can add 200-500 tokens. The tradeoff is worth it because you eliminate retry costs from malformed responses.

Can I use structured outputs with function calling simultaneously?

No. The response_format parameter and tools/functions parameter are mutually exclusive in a single API call. If you need both structured extraction and tool use, split them into separate calls or use the Pydantic parsing approach within your tool functions.

---

#OpenAI #StructuredOutputs #JSONSchema #API #Python #AgenticAI #LearnAI #AIEngineering