How is high-quality instruction-tuning data constructed?
mediumAnswer
- Sources: (1) hand-written by experts (expensive, small); (2) distilled from a stronger model with human filtering (OpenHermes, Alpaca-style — cheap, scalable); (3) real user conversations with filtering + rewriting; (4) task-mixture datasets (FLAN, T0).
- Quality > quantity: LIMA (Meta, 2023) showed 1k carefully curated pairs beat 50k noisy ones.
- Modern SFT recipes use 10k-100k high-quality examples across diverse tasks (chat, code, math, extraction, safety refusals).
Check yourself — multiple choice
- Only quantity matters
- Hand-written + distilled + real conversations, quality > quantity (LIMA: 1k curated beats 50k noisy)
- Random web scrapes
- Never distill
Instruction data: quality > quantity; LIMA showed 1k curated pairs suffice.
#fine-tuning#data#alignment
Practise LLMs & GenAI
214 interview questions in this topic.
Related questions
- What is supervised fine-tuning (SFT) and where does it fit in the alignment pipeline?
- How are 'reasoning models' like o1 / R1 trained?
- Fine-tuning vs RAG — which do you use when?
- What is LoRA and why is it popular for fine-tuning LLMs?
- What data goes into modern LLM pretraining?
- Why is data deduplication critical for LLM pretraining?