Why do modern LLMs use decoder-only architectures?
mediumAnswer
- (1) Simpler training objective (next-token) unifies pretraining and generation.
- (2) Scales more predictably — no separate encoder to balance.
- (3) Zero-shot / few-shot works out of the box via in-context learning.
- (4) Encoder-decoder architectures (T5) don't scale as gracefully for open-ended generation.
- (5) KV cache is straightforward with causal attention.
- Encoder-only (BERT) is for classification / retrieval; decoder-only (GPT, Llama, Mistral) dominates general-purpose LLMs.
Check yourself — multiple choice
- Encoder-decoder is universal
- Simpler training, cleaner scaling, native in-context learning, easy KV cache — decoder-only became the LLM default
- Encoder-only is dominant
- MoE forces decoder-only
Decoder-only: simple objective, clean scaling, ICL, cheap KV cache — modern LLM default.
#architecture#fundamentals
Practise LLMs & GenAI
214 interview questions in this topic.
Related questions
- What is SwiGLU and why do LLMs use it in the MLP block?
- What is a large language model, in one paragraph?
- What is the context window and what tricks extend it?
- What is a Mixture-of-Experts (MoE) LLM?
- Walk through how one training step of an autoregressive LLM works.
- How does Byte-Pair Encoding (BPE) tokenization work?