How can multiple models share a GPU?

hard

Answer

$(1) MPS (Multi-Process Service, NVIDIA): concurrent process access, no isolation.$
$(2) MIG (Multi-Instance GPU, A100+): hardware partition into 1-7 slices with dedicated compute + memory.$
$(3) Triton concurrent model execution: single process, multiple models on GPU.$
$(4) K8s device plugin: dev-plugin variants for fractional GPU (nvidia-device-plugin + MIG or MPS).$
$(5) Time slicing: rare, high overhead.$
$Rule: MIG for isolation / SLA, MPS for max utilization, Triton for cohabiting models.$

Check yourself — multiple choice

One only
MPS (concurrent no isolation) / MIG (hardware partition A100+, 1-7 slices dedicated) / Triton concurrent execution / K8s device plugin fractional / time slicing rare; MIG=isolation, MPS=max util, Triton=cohabit
Random
Not real

GPU sharing: MIG (isolate) / MPS (max util) / Triton (cohabit).

#infrastructure

215 interview questions in this topic.