AI & Automation

Structured Outputs for AI Automation - A Schema Is a Contract, Not a Safety Check

Written by Adam Muiz 10 Sep 2026 Updated: 10 Sep 2026 9 min read

A language model can return neat JSON instead of a paragraph. That feels like a decisive step toward reliable automation: the keys are present, the types look right, and the parser no longer has to rescue a half-finished code block. But what, exactly, has become reliable?

A structured response can make an uncertain model easier for software to talk to. It does not make every value true, every requested action appropriate, or every side effect safe. The useful question is therefore not, “Did the model return valid JSON?” It is, “Which gates must this request pass before the application is allowed to act?”

This distinction matters most when output stops being text and starts becoming authority: publishing a post, sending a message, changing a file, or calling an administrative API. The conservative design is to let the model propose a narrowly structured action while deterministic application code decides whether that proposal may proceed.

Five Different Questions Hidden Behind “Valid”

The word valid is overloaded. A small automation pipeline should separate at least five questions:

Can it be parsed? Is the response valid JSON?
Does it match the contract? Are required fields present, values of the expected type, and choices within declared limits?
Does it make sense now? Does the referenced draft exist, is the locale supported, and is the requested schedule still available?
May this actor do it? Is the user or service authorized for this resource and operation, and does the action require approval?
Can it be executed safely? Will retries duplicate the effect, are resources bounded, and will failure leave a comprehensible state?

RFC 8259 defines JSON as a data-interchange format with a small grammar. Passing that grammar answers only the first question. It says nothing about whether a slug names a real draft or whether the caller may publish it.

JSON Schema validation can answer part of the second question through assertions such as type, required, enum, and length constraints. That is valuable. A contract that rejects unexpected shapes is much better than searching prose for a token such as [PUBLISH]. It is still one layer, not the whole decision.

A Schema Narrows the Conversation

Imagine an assistant that may propose what to do with an existing article draft. A deliberately small contract could look like this:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "action": {
      "type": "string",
      "enum": ["request_review", "schedule"]
    },
    "draft_id": {
      "type": "integer",
      "minimum": 1
    },
    "locale": {
      "type": "string",
      "enum": ["en", "id", "de"]
    },
    "scheduled_at": {
      "type": ["string", "null"]
    },
    "reason": {
      "type": "string",
      "minLength": 1,
      "maxLength": 500
    }
  },
  "required": ["action", "draft_id", "locale", "scheduled_at", "reason"],
  "additionalProperties": false
}

The enum prevents the model from inventing an action such as delete_everything. The integer constraint rejects a negative identifier. The locale allowlist keeps the interface within languages the application claims to support. additionalProperties: false makes accidental or invented fields visible instead of silently accepting them.

Current model APIs demonstrate two related patterns. OpenAI documents schema-constrained model responses, while Anthropic documents tool definitions with an input_schema. Their APIs and supported schema features can change, so an application must follow the documentation for the provider and model it actually uses. “Strict” should not be treated as one portable behavior shared by every implementation.

The durable idea is provider-neutral: define the smallest machine-readable proposal the application needs. Do not expose a general shell command when the task only requires choosing between review and scheduling. Narrow vocabulary reduces ambiguity for both the model and the executor.

Shape Is Not Truth

Now consider this schema-conforming proposal:

{
  "action": "schedule",
  "draft_id": 417,
  "locale": "de",
  "scheduled_at": "2026-09-10T00:00:00+07:00",
  "reason": "The translation is complete."
}

Its shape may be perfect while every operational conclusion is wrong. Draft 417 may not exist. The German translation may still be missing a paragraph. The timestamp may be in the past when processed. Another article may already occupy the daily slot. The caller may only have review permission. The sentence in reason is still generated text, not evidence.

This limit follows from what schemas do. The JSON Schema core specification says an instance can only fail an assertion that is present in the schema. A schema cannot reject an unstated rule. Even stated string formats have subtleties: the validation specification explains that format may be annotation-only depending on the vocabulary and implementation, and that syntactic checking does not establish whether an identified entity exists.

Application code therefore needs a semantic validation phase after schema validation. For this example, that phase might load the draft by ID, compare its actual locale and status, check the publication calendar, and confirm that required translations and media are present. These are ordinary queries against authoritative state. Asking the model to “double-check” by generating another answer is not equivalent.

Treat the Proposal as Untrusted Input

A model response may be influenced by user prompts, retrieved documents, tool results, or earlier model output. The application should not assume that structured origin means trusted origin. OWASP’s guidance on improper output handling recommends treating model output like untrusted input and validating it before it reaches downstream functions.

That principle leads to familiar engineering rules:

Resolve identifiers through application code; do not turn generated strings into SQL fragments.
Map an allowed action name to a specific function; do not pass model text to a shell or eval().
Encode generated text for its destination context; schema-valid HTML-shaped text is not automatically safe HTML.
Reject unknown fields and values rather than guessing what the model intended.
Return validation errors as data, not as permission to attempt a more powerful fallback.

This is not an argument against structured output. It explains why structured output is useful: a narrow object is easier to validate and route safely than free-form prose. The mistake is allowing the data contract to double as a security boundary it was never designed to be.

Authorization Must Live Outside the Model

A prompt can tell a model which actions are allowed, but a prompt is not an access-control system. The executor must authenticate the real caller and enforce policy using server-side state. If a user may edit only their own drafts, the database query or service layer must preserve that ownership boundary regardless of the ID proposed by the model.

OWASP’s excessive-agency guidance separates three useful reductions: minimize available functionality, minimize downstream permissions, and minimize autonomy. A read-only lookup should use a read-only identity. A tool that schedules one approved draft should not also expose arbitrary update or delete operations. High-impact actions can stop at a review state until a person approves the exact target and effect.

Human approval is not automatically useful. A vague button asking someone to “approve AI output” encourages reflexive clicking. A meaningful checkpoint shows the proposed action, target, important changes, source of authority, and consequence. Approval should also expire if the underlying draft changes; otherwise a person may approve one version while the system executes another.

Separate Proposal, Decision, and Effect

A clean workflow does not move directly from model response to side effect. It records distinct states:

model response
  -> parsed proposal
  -> schema accepted
  -> policy decision
  -> awaiting approval (when required)
  -> execution attempt
  -> succeeded or failed

This separation makes failures less mysterious. A schema rejection means the contract was not met. A policy rejection means the request was well formed but not allowed. An execution failure means an approved operation encountered an operational problem. Combining all three into “the AI failed” hides the part of the system that needs repair.

The decision record should refer to stable identifiers and the version of the object that was checked. For consequential writes, an idempotency key or equivalent uniqueness rule can prevent a retry from repeating the same logical effect. This matters because timeouts are ambiguous: a client may lose the response after the server has already completed the action.

Retries should also be bounded. Retrying malformed or forbidden proposals wastes resources and can obscure an attack. Retrying a transient network failure may be reasonable, but only when the operation is repeat-safe and the system can distinguish “not attempted” from “completed but response lost.”

Logs Should Explain Decisions Without Becoming a New Leak

An audit trail should answer practical questions: which authenticated actor initiated the request, which model configuration produced the proposal, which schema version was applied, which rules passed or failed, who approved it, which tool ran, and what final state resulted?

That does not require storing every prompt and response forever. Raw context may contain personal data, credentials, private documents, or hostile instructions. Prefer structured event fields, redact secrets, limit retention, and restrict access. Store enough to reconstruct a decision, not an unlimited copy of everything the model saw.

Versioning matters here. If a schema or policy changes, a later reviewer needs to know which rules evaluated the old proposal. The broader lesson matches the lifecycle framing of NIST AI 600-1: risk management belongs across design, development, use, and evaluation. It cannot be compressed into one API option at generation time.

A Small Review Checklist

Is the response parsed with a JSON parser rather than executable evaluation?
Does the application validate against the exact schema and dialect it intends to support?
Are actions, identifiers, sizes, and collections narrowly constrained?
Does deterministic code verify current business state after schema validation?
Does the downstream service enforce authentication, ownership, and least privilege?
Do consequential actions require a specific, inspectable approval?
Can a timeout or retry repeat the side effect?
Are failure states distinct and logs useful without retaining secrets unnecessarily?
Are schema, policy, and tool changes covered by tests with invalid as well as valid proposals?

A Contract, Not a Verdict

Structured output solves a real integration problem. It replaces brittle prose parsing with an explicit contract and can keep model responses within a much smaller set of shapes. That is a meaningful improvement.

The boundary is equally important: a schema can establish that a proposal looks like schedule draft 417. It cannot, by itself, establish that draft 417 exists, is complete, belongs to the caller, has not changed, fits today’s editorial policy, or should be published now.

Reliable automation comes from composing modest guarantees. Parse the data. Validate its declared shape. Check reality and policy with deterministic code. Enforce authority downstream. Ask for approval where consequences justify friction. Then execute in a bounded, repeat-safe way. The model may suggest the next move, but the application must remain responsible for deciding whether that move is allowed.