Principal Components Regression (PCR) — what does it do?
hardAnswer
- Regress on PCA components instead of raw features: (1) PCA → keep top k components, (2) OLS on those.
- Reduces multicollinearity + acts as regularization.
- Downside: PCA is unsupervised → top components may not correlate with Y.
- Alternative: PLS (Partial Least Squares) finds components maximizing covariance with Y — supervised DR + regression.
- PLS usually better for prediction with many correlated features.
Check yourself — multiple choice
- Random
- Regress on top PCA components → reduces multicollinearity + regularizes; PCA unsupervised so may miss Y-relevant dirs — PLS supervised is better
- Same as OLS
- Not real
PCR: OLS on top PCs; PLS is supervised alternative maximizing cov with Y.
#dimensionality-reduction#regression
Practise Unsupervised Learning
214 interview questions in this topic.