EasyDeepLearn
Unsupervised Learning · section 2 of 9

Advanced clustering & density

8 interview questions on advanced clustering & density, each answered in full. Free to read, no account needed.

Local Outlier Factor (LOF) — how does it work?

medium
  • For each point, compare its local density (via k-NN distances) to that of its neighbors.
  • LOF > 1: point is in a lower-density region than its neighbors → outlier.
  • Handles clusters of varying density — a big advantage over global thresholds.
  • Cost O(n2)O(n^{2}) or O(n log n) with indexes.
  • Weakness: sensitive to k.
  • Standard in tabular anomaly detection alongside Isolation Forest.
#anomaly-detection#densityPermalink & quiz →

Elliptic envelope / robust Mahalanobis distance — when to use?

medium
  • Fit a robust multivariate Gaussian (Minimum Covariance Determinant, Rousseeuw-Van Driessen) → flag points with high Mahalanobis distance.
  • Assumes elliptical / Gaussian normal cloud; robust to outliers via MCD.
  • Fast and interpretable.
  • Best for tabular data with roughly Gaussian normals (financial time series, sensor readings).
  • Fails for multimodal / non-Gaussian normals — use GMM or Isolation Forest instead.
#anomaly-detection#densityPermalink & quiz →

Kernel Density Estimation — mechanism.

medium
  • Non-parametric density estimator: p̂(x)  =  (1/(nhd))(x)\; = \;(1 / (\mathrm{nh}^{d})) Σ K((x  −  xi)/h)K((x\; - \;x_{i}) / h).
  • K = kernel (Gaussian typical). h = bandwidth (critical hyperparameter).
  • Uses: visualization (smoothed histogram), density-based anomaly detection, generative sampling.
  • Scales poorly to high d (curse of dimensionality — need exponentially more data).
  • Use cross-validation or Silverman's rule for h.
#density#anomaly-detectionPermalink & quiz →

How do you choose the KDE bandwidth?

hard
  • (1) Silverman's rule: h = (4/(d+2))^(1/(d+4)) * σ * n^(-1/(d+4)) — closed form, assumes Gaussian data.
  • (2) Scott's rule: similar but different exponent.
  • (3) Cross-validation of log-likelihood → most principled, more expensive.
  • (4) Adaptive bandwidth: h varies with local density.
  • Small h → overfit / spiky; large h → over-smooth.
  • Rule: start with Silverman, refine with CV if it matters.
#densityPermalink & quiz →

What is a copula and why use one?

hard
  • Copula separates marginal distributions from dependence structure: F(x,  y)  =  C(FX(x),  FY(y))F(x, \;y)\; = \;C(F_{X}(x), \;F_{Y}(y)).
  • Sklar's theorem: unique C for continuous margins.
  • Uses: multivariate density estimation when marginals are heavy-tailed but dependence is simpler (finance risk, joint failure modeling).
  • Gaussian copula, t-copula, Archimedean copulas (Clayton, Gumbel) — different tail dependence structures.
  • Foundational in quantitative finance.
#densityPermalink & quiz →

Normalizing flows for density estimation — the idea.

hard
  • Learn invertible neural network f_θ that maps simple base distribution (Gaussian) to complex data distribution.
  • Change of variables: log p(x) = log p(z)  +  log⁡∣det⁡p(z)\; + \;\operatorname{log} \mid \operatorname{det} JfJ_{f}^(-1)|.
  • Requires efficient Jacobian determinants → architectures: RealNVP, Glow, NICE (coupling layers), MAF, NSF.
  • Uses: exact density estimation, sampling, anomaly detection.
  • Modern: replaced by diffusion models for sampling but still competitive for exact-density needs.
#density#deep-learningPermalink & quiz →

Score matching — what does it estimate?

hard
  • Rather than p(x), estimate the score s(x) = ∇_x log p(x).
  • Loss (Hyvärinen): E[∣∣sθ(x)  −  ∇  log⁡  p(x)∣∣2]E[ \mid \mid s{\theta}(x)\; - \; \nabla \;\operatorname{log}\;p(x) \mid \mid ^{2}] → doesn't need normalization constant → tractable for unnormalized density models.
  • Foundation of energy-based models and diffusion models (score-based generative modeling, Song & Ermon; Ho et al. DDPM).
  • Modern generative modeling paradigm shift.
#density#deep-learningPermalink & quiz →

KL divergence — what it measures and pitfalls.

medium
  • KL⁡(P  ∣∣  Q)  =  Σ\operatorname{KL}(P\; \mid \mid \;Q)\; = \;{\Sigma} P(x) log⁡(P(x)/Q(x))\operatorname{log}(P(x) / Q(x)) = -H(P) + H(P, Q).
  • Not symmetric, not a metric.
  • Zero iff P = Q, infinite if Q(x) = 0 where P(x) > 0.
  • Two directions: forward KL⁡(P∣∣Q)\operatorname{KL}(P \mid \mid Q) 'mode-covering' (VAE default → q spreads over p); reverse KL⁡(Q∣∣P)\operatorname{KL}(Q \mid \mid P) 'mode-seeking' (VI → q concentrates on one mode of p).
  • Standard divergence in ML but rarely a metric; prefer Wasserstein / MMD when metric properties matter.
#density#theoryPermalink & quiz →

Practise Unsupervised Learning

All 9 sectionsFlashcardsQuiz