How do you monitor GPU utilization?

medium

Answer

$(1) nvidia-smi real-time.$
$(2) dcgm-exporter for Prometheus (per-GPU metrics: SM utilization, memory, temp, power).$
$(3) PyTorch Profiler / TensorBoard for training-time analysis.$
$(4) NCU / Nsight for kernel-level.$
$Key metric: SM occupancy — not just GPU busy %.$
$Low occupancy = kernels not filling GPU (small batch, PCIe bottleneck, CPU bound).$
$Interview: '100% utilization' can hide inefficient kernels; profile to find idle SMs.$

Check yourself — multiple choice

Random
nvidia-smi real-time + dcgm-exporter Prometheus (SM util/mem/temp/power) + PyTorch profiler / TensorBoard training + NCU/Nsight kernel-level; key SM occupancy not just busy %; 100% util can hide inefficient kernels
Just busy%
Not real

GPU util: dcgm + Prometheus + profiler; SM occupancy > busy%.

#monitoring#infrastructure

Practise MLOps & Data Quality

215 interview questions in this topic.

Read the lesson Flashcards Take the quiz

Related questions