How do you mitigate serverless cold-start latency?

medium

Answer

$(1) Provisioned concurrency: pre-warmed containers (Lambda).$
$(2) Snap start: pre-load runtime state (Java / .NET Lambda).$
$(3) Reduce model size (quantize, distill).$
$(4) Load model lazily on first request \to keep warm via periodic ping (warmer pattern).$
$(5) Container reuse: Lambda reuses container for successive invocations.$
$(6) Move heavy imports to global scope (loaded once).$
$(7) Consider always-on service if cold start unacceptable.$
$Modern: Cloud Run min-instances = 1.$

Check yourself — multiple choice

Just wait
Provisioned concurrency + snap-start + quant/distill smaller model + lazy load + warmer ping + container reuse + global imports; Cloud Run min-instances=1; if unacceptable → always-on
Random
Not real

Cold-start: provisioned + snap-start + smaller model + warmer + min-instances.

#serving#infrastructure

215 interview questions in this topic.