How do you serve many LoRA adapters efficiently?
hardAnswer
- Multi-LoRA serving (S-LoRA, vLLM, Punica): load one base model + hundreds of small LoRA adapters in memory.
- At request time, apply the requested adapter dynamically during matmul via a specialized kernel that fuses ΔW = B * A into the base W @ x.
- Adds only KB of weight per adapter.
- Enables per-tenant / per-task specialization at negligible extra memory.
- Standard for multi-tenant LLM SaaS with lots of fine-tuned variants.
Check yourself — multiple choice
- One model per adapter
- Load one base + many LoRA adapters; fused kernel applies ΔW per request → hundreds of adapters at negligible extra memory
- Only for training
- Impossible
Multi-LoRA serving: one base + many adapters via fused kernels → cheap multi-tenant.
#serving#peft#lora#inference
Practise LLMs & GenAI
214 interview questions in this topic.