How do you estimate the intrinsic dimension of a dataset?

hard

Answer

$\operatorname{log}(\mathrm{number}\;\mathrm{of}\;\mathrm{pairs}\;\mathrm{within}\;{\varepsilon})$
$(2) MLE-based (Levina-Bickel): from distances to k nearest neighbors.$
$(3) TwoNN (Facco et al., 2017): ratio of first two neighbor distances.$
$(4) PCA scree-plot elbow.$
$Real datasets typically have intrinsic dim much smaller than ambient dim (MNIST ambient=784, intrinsic \approx 12-14).$

Check yourself — multiple choice

Same as ambient
Correlation dim / MLE (Levina-Bickel) / TwoNN / PCA scree; MNIST ambient=784 but intrinsic ≈ 12-14 — data lives on low-dim manifold
Random
Not measurable

Intrinsic dim: correlation dim / MLE / TwoNN / PCA scree; usually ≪ ambient.

#dimensionality-reduction#theory

214 interview questions in this topic.