BYD TechThe short answer
Structured outputs make an AI model return data in a format you define, usually JSON that matches a JSON Schema, instead of free text. Major providers, including Anthropic and OpenAI, can constrain the model so its output always fits the schema, which removes most parsing errors and retries. A valid format is not a correct answer, though: you still need to check the values, handle refusals and cut-off responses, and decide what happens when a field is unknown.
Key points
- Define the shape you need as a JSON Schema.
- Constrained decoding makes the output fit that shape.
- Valid JSON is not the same as correct data.
- Allow "unknown" values instead of forcing guesses.
- Handle refusals and truncated responses in code.
Why structure matters
A chat answer can be any text. A product feature usually needs specific data: an extracted invoice total, a category, a yes or no, a list of action items. Before structured outputs, teams asked the model for JSON in the prompt and wrote code to repair the broken responses. Now the provider can enforce the format.
How it works
- Describe the output as a JSON Schema: field names, types, required fields, allowed values.
- Send the schema with the request using the provider’s structured output option.
- The provider constrains generation so the response matches the schema.
- Your code parses the result and checks the values.
Anthropic’s documentation describes two related features: JSON outputs, which shape the model’s response, and strict tool use, which guarantees that tool calls have valid parameters. It notes exceptions: a refusal or a response cut off by the token limit may not match the schema.
Designing a good schema
- Keep it as flat and small as the feature allows.
- Use enums for categories, so values are from a known list.
- Allow an empty or "unknown" value where the information may be missing, so the model is not pushed to invent it.
- Add a short description to each field explaining what belongs there.
- Ask for supporting evidence, such as the source sentence, when correctness matters.
Checks you still need
| Check | Example |
|---|---|
| Business rules | Line items add up to the total |
| Known values | The customer ID exists in your database |
| Ranges | Dates are in the past, quantities are positive |
| Stop reason | The response was not refused or cut off |
| Confidence | Low-confidence or unknown fields go to a person |
Structured outputs are the backbone of document extraction. See Intelligent document processing and How to reduce AI hallucinations. They are standard in the features we build in AI agent development.
Sources
Checked when this guide was written.
