What makes a good self-supervised pretext task?

hard

Answer

$It must be solvable only by learning something you actually want.$
$The failure mode is a shortcut: a task the network can solve with a superficial cue, such as detecting a rotation from a border artefact or matching two crops by their shared colour histogram, which yields a high pretext score and useless representations.$
$Good tasks force invariance to nuisance factors while preserving the semantics you need downstream, which is why contrastive augmentation design matters more than the loss function.$
$The task also has to be hard enough to require capacity but not so ambiguous that the target is unpredictable in principle.$
$The only real test is transfer: freeze the encoder and measure linear-probe performance on the downstream task, because pretext loss is not the objective you care about.$

Check yourself — multiple choice

Any task with low loss
One that cannot be solved by a shortcut, forces invariance to nuisances while keeping downstream semantics, and is validated by linear-probe transfer rather than pretext loss
The hardest possible task
Pretext tasks do not matter

Shortcut resistance and downstream transfer, not pretext loss, define a good pretext task.

#representation-learning#evaluation

214 interview questions in this topic.