EasyDeepLearn

How does DPO simplify RLHF?

hard

Answer

  • DPO (Rafailov 2023) skips both the reward model and the PPO loop.
  • Given preference pairs (prompt, chosen, rejected), directly fine-tune the policy π_θ against a frozen reference πref{\pi}_{\mathrm{ref}} with loss: -log sigmoid(β    (log  πθ(chosenx)    log  πref(chosenx))    β    (log  πθ(rejectedx)    log  πref(rejectedx)){\beta}\; \cdot \;(\operatorname{log}\;{\pi}{\theta}(\mathrm{chosen} \mid x)\; - \;\operatorname{log}\;{\pi}_{\mathrm{ref}}(\mathrm{chosen} \mid x))\; - \;{\beta}\; \cdot \;(\operatorname{log}\;{\pi}{\theta}(\mathrm{rejected} \mid x)\; - \;\operatorname{log}\;{\pi}_{\mathrm{ref}}(\mathrm{rejected} \mid x))).
  • Equivalent to RLHF optimum under Bradley-Terry preferences.
  • Much simpler: one supervised loss, no rollouts, no RM.
  • Now the default alignment method for most open LLMs (Zephyr, Tulu, Mistral, Qwen).
Check yourself — multiple choice
  • DPO uses PPO internally
  • Direct preference optimization: supervised loss on log-ratio between policy and reference — no RM, no PPO
  • Only classification
  • Same as RLHF

DPO: reparametrize RLHF as a supervised log-sigmoid loss over log-ratios — simpler, cheaper.

#dpo#alignment#preference

Practise LLMs & GenAI

214 interview questions in this topic.

Related questions