Why does truncated SVD denoise?
hardAnswer
- Signal typically lies in low-rank subspace; noise spreads across all singular directions.
- Discarding small singular values throws away noise-dominant components while keeping signal.
- Foundation of PCA-based denoising, spectral clustering, and matrix completion (Netflix Prize).
- Also called optimal shrinkage — Gavish-Donoho gives an explicit optimal threshold for Gaussian noise.
Check yourself — multiple choice
- Random
- Signal is low-rank; noise spreads across all singular values → truncate small SVs → keep signal, drop noise; Gavish-Donoho gives optimal threshold
- Same as PCA
- Not real
Truncated SVD denoises: signal is low-rank, noise spreads.
#dimensionality-reduction
Practise Unsupervised Learning
214 interview questions in this topic.