Score matching — what does it estimate?
hardAnswer
- Rather than p(x), estimate the score s(x) = ∇_x log p(x).
- Loss (Hyvärinen): → doesn't need normalization constant → tractable for unnormalized density models.
- Foundation of energy-based models and diffusion models (score-based generative modeling, Song & Ermon; Ho et al. DDPM).
- Modern generative modeling paradigm shift.
Check yourself — multiple choice
- Random
- Estimate score s(x) = ∇log p(x) — no normalization needed → tractable for unnormalized densities; foundation of energy-based + diffusion models
- Same as MLE
- Not real
Score matching: fit ∇log p(x); foundation of diffusion / energy-based models.
#density#deep-learning
Practise Unsupervised Learning
214 interview questions in this topic.
Related questions
- Normalizing flows for density estimation — the idea.
- Gaussian Mixture Model vs k-means — what's the difference?
- What is an autoencoder and what are the common variants?
- HDBSCAN — how does it improve on DBSCAN?
- OPTICS — what does it produce and how do you use it?
- Mean shift clustering — mechanism and use case.