How do Medusa and EAGLE differ from vanilla speculative decoding?
hardAnswer
- Vanilla speculative decoding uses a separate small draft model — must train / maintain two models, and inference bounces between them.
- Medusa (Cai 2024) trains multiple parallel 'draft heads' attached to the target model that predict the next K tokens directly — one model, one forward pass drafts K candidates.
- EAGLE (Li 2024) trains a small LSTM-style head to predict future embeddings from current hidden states → higher acceptance rate than Medusa.
- Both give 2-3× decode speedup without a separate draft model.
Check yourself — multiple choice
- Same as speculative decoding
- Draft heads attached to target model (Medusa) or a learned head on hidden states (EAGLE) → 2-3× speedup without separate draft model
- Impossible
- Only for training
Medusa / EAGLE: parallel draft heads on the target → no separate draft model, 2-3× speedup.
#inference#generation
Practise LLMs & GenAI
214 interview questions in this topic.
Related questions
- What is speculative decoding?
- What do temperature and top-p do at generation time?
- What is the context window and what tricks extend it?
- What is quantization and what tradeoffs come with it?
- What is the KV cache and why does it matter for LLM inference?
- What is Grouped-Query Attention (GQA) and why does it matter?