How do you force structured output (JSON) from an LLM?

medium

Answer

$Options: (1) plain prompt asking for JSON — unreliable, parses fail.$
$(2) JSON mode: model constrains output to valid JSON syntax (OpenAI, Anthropic, most APIs).$
$(3) Constrained decoding: mask logits at each step to only allow tokens consistent with a schema (Outlines, XGrammar, LM Format Enforcer) — guarantees schema compliance.$
$(4) Function calling / tool schemas: model outputs a JSON matching a declared function signature.$
$(3) and (4) are the production standard.$

Check yourself — multiple choice

Just ask nicely
Constrained decoding / JSON mode / function calling → schema-guaranteed output; prompt-only is unreliable
Only regex on output
Impossible to constrain

Structured output: JSON mode / constrained decoding / function calling — guarantees schema.

#prompting#generation#production

214 interview questions in this topic.