PagedAttention — what problem does it solve?
hardAnswer
- Standard KV cache reserves contiguous memory per sequence for max length → 60-80% waste (short sequences reserve too much).
- Fragmentation prevents new sequences even with free memory.
- Paged attention: divide KV cache into fixed-size blocks (pages) similar to OS virtual memory.
- Sequence maps to non-contiguous pages via block table.
- Utilization → 90%+; throughput 2-4x.
- Enables efficient shared prefix caching (system prompt shared across users).
Check yourself — multiple choice
- Random
- Standard KV cache: contiguous per-sequence, reserves max length → 60-80% waste + fragmentation; paged: fixed blocks + block table (like OS virtual mem) → 90% util + 2-4x throughput + shared prefix
- Same as attention
- Not real
PagedAttention: paged KV cache; 90% util; shared prefix caching.
#serving#llmops
Practise MLOps & Data Quality
215 interview questions in this topic.