BYOL — how does it avoid the need for negatives?

hard

Answer

$Two networks: online (learnable) and target (EMA of online).$
$Feed different augmentations to each; online must predict target's representation via a predictor head.$
$No negatives, no contrastive loss — momentum + asymmetric predictor prevents collapse.$
$Surprising and empirical: BYOL matches SimCLR without negatives.$
$Successor SimSiam removes even the momentum target (relies only on stop-gradient).$

Check yourself — multiple choice

Same as SimCLR
Online net predicts target (EMA) representation across augmentations; no negatives, no contrastive loss; asymmetric predictor + stop-gradient prevents collapse
Random
Not real

BYOL: online predicts target (EMA); no negatives; stop-grad prevents collapse.

#representation-learning

214 interview questions in this topic.