EasyDeepLearn

What is a covariance matrix and its key properties?

medium

Answer

  • For random vector X ∈ RdR^{d}: Cov⁡(X)  =  E[(X−μ)(X−μ)]\operatorname{Cov}(X)\; = \;E[(X - {\mu})(X - {\mu})], a d×d matrix.
  • Diagonal: variances Var⁡(Xi)\operatorname{Var}(X_{i}).
  • Off-diagonal: covariances Cov⁡(Xi,  Xj)\operatorname{Cov}(X_{i}, \;X_{j}).
  • Properties: (1) symmetric, (2) positive semi-definite, (3) real eigenvalues ≥ 0.
  • Standardization: correlation matrix  =  D−1\mathrm{matrix}\; = \;D^{-1} Σ D−1D^{-1} where D is a diagonal matrix of SDs.
  • PCA is eigendecomposition of Σ; Mahalanobis distance uses Σ−1{\Sigma}^{-1}.
Check yourself — multiple choice
  • Non-symmetric
  • d×d symmetric PSD; diag = variances, off-diag = covariances; foundation of PCA / Mahalanobis / MVN
  • Only diagonal
  • Not real-valued

Cov matrix: symmetric PSD; used by PCA, Mahalanobis, MVN.

#variance#probability

Practise Statistics Fundamentals

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions