How does Latent Diffusion (Stable Diffusion) reduce compute?
hardAnswer
- Train a VAE / autoencoder that maps 512×512 images to an 8-16x smaller latent space (e.g., 64×64×4).
- Run the diffusion process in that latent space instead of pixels.
- Compute drops ~50x, memory drops ~64x, quality is comparable to pixel-space diffusion after the VAE decoder up-samples the final latent.
- Foundation of Stable Diffusion (Rombach 2022) — democratized text-to-image generation on consumer hardware.
Check yourself — multiple choice
- Same as pixel diffusion
- Compress image to a smaller VAE latent, run diffusion there → ~50× compute reduction with similar quality (Stable Diffusion)
- Removes the VAE
- Only for text
Latent Diffusion: VAE compresses image → diffuse in latent → cheap high-quality generation.
#diffusion#generative#architectures
Practise Deep Learning
214 interview questions in this topic.
Related questions
- What are the main ideas of StyleGAN?
- Describe the forward and reverse processes in diffusion models.
- How does DDIM speed up diffusion sampling?
- What is classifier-free guidance (CFG) in diffusion?
- What are normalizing flows and their trade-off vs GANs / diffusion?
- Why do residual (skip) connections help training deep networks?