What is logit bias and when do you use it?

medium

Answer

$\mathrm{logits}[\mathrm{token}_{\mathrm{id}}]$
$Uses: (1) ban tokens (bias=-100 \to probability ~0) — e.g., forbid a competitor's name; (2) encourage a token slightly (bias=+2); (3) force a token (very high positive bias).$
$Available in OpenAI API.$
$Not a substitute for prompt engineering or fine-tuning, but useful for last-mile constraints: banning specific tokens, forcing JSON to start with '{', etc.$

Check yourself — multiple choice

Only for training
Additive bias on specific token logits: ban (-100), encourage (+2), force (large +) — last-mile decoding constraint
Same as temperature
Only for images

Logit bias: per-token additive adjustment for last-mile decoding control.

#generation#production

214 interview questions in this topic.