What are stop sequences and why do they matter in production?
easyAnswer
- Strings that, when generated, terminate the response early.
- Uses: (1) chat markers like '<|user|>' or '\n\n' to prevent the model from continuing beyond its turn; (2) end-of-JSON marker for structured output; (3) tool-call markers.
- Without stops, models often hallucinate a new user turn or keep going past the answer.
- Providers set defaults (assistant turn markers) but exposing them in the API is essential for custom formats.
Check yourself — multiple choice
- Never needed
- Strings that terminate generation early — prevent hallucinated turns, enforce format boundaries, control cost
- Only for classification
- Same as temperature
Stop sequences: essential for controlling format, preventing hallucinated turns, and cost.
#generation#production
Practise LLMs & GenAI
214 interview questions in this topic.
Related questions
- How do you force structured output (JSON) from an LLM?
- How does JSON mode actually work?
- What is logit bias and when do you use it?
- How should max_tokens be set in production?
- Why is streaming output important in production LLM apps?
- Structured Outputs (OpenAI) / Constrained decoding — how do they differ from JSON mode?