How does DP-SGD work?

hard

Answer

$Standard SGD with two modifications: (1) Clip per-example gradient norm to constant C (bounds sensitivity).$
$N(0, \;{\sigma}^{2}C^{2} \cdot I)$
$Provides (ε, δ)-differential privacy: presence/absence of any individual has bounded influence on output.$
$Tradeoff: more noise \to more privacy but worse accuracy.$
$Track privacy budget over training epochs (privacy accountant).$
$Standard in Google / Apple production ML.$

Check yourself — multiple choice

Random
Clip per-example gradient norm to C + add Gaussian $N(0, \;{\sigma}^{2}C^{2})$ to gradient sum; provides (ε,δ)-DP: bounded influence per example; track budget over epochs
Same as dropout
Not real

DP-SGD: clip + Gaussian noise; (ε,δ)-DP; track privacy budget.

#safety#advanced

Practise MLOps & Data Quality

215 interview questions in this topic.

Read the lesson Flashcards Take the quiz

Related questions