Autoencoder variants — quick tour.

medium

Answer

$(1) Undercomplete AE: bottleneck < input dim \to compression.$
$(2) Denoising AE: reconstruct clean from noised input \to robustness / representation.$
$(3) Sparse AE: L1 or KL penalty on activations \to sparse code.$
$(4) Contractive AE: penalize ||\partialh/\partialx||_F \to local invariance.$
$(5) VAE: probabilistic latent, generative.$
$(6) Masked AE (MAE, He et al. 2022): mask 75% of patches, reconstruct — SOTA vision SSL.$

Check yourself — multiple choice

Only one
Undercomplete / Denoising / Sparse / Contractive / VAE / Masked AE — different regularizers for different goals; MAE = SOTA vision SSL
Random
Not real

AE variants: undercomplete / denoising / sparse / contractive / VAE / MAE.

#representation-learning#deep-learning

214 interview questions in this topic.