Give the OLS closed-form and its variance.

medium

Answer

$(XX)^{-1}$
$\operatorname{Var}({\beta}\; \mid \;X)\; = \;{\sigma}^{2}\; \cdot \;(XX)^{-1}$
${\sigma}^{2}$
$s^{2}\; \cdot \;(XX)^{-1}$
$When X'X is ill-conditioned (multicollinearity), variances blow up.$
$Solutions: ridge regression, principal components regression, or dropping correlated features.$
$This formula is the reason multicollinearity is so damaging: it inflates the variance of β̂.$

Check yourself — multiple choice

Random
β̂ = $(XX)^{-1}$ X'y; $\operatorname{Var}({\beta})\; = \;{\sigma}^{2}$ $(XX)^{-1}$ → ill-conditioned X'X blows up SEs (multicollinearity)
Only for logistic
Not closed-form

OLS: β̂= $(XX)^{-1}$ X'y; $\operatorname{Var} = {\sigma}^{2}(XX)^{-1}$ ; multicollinearity blows up variance.

#regression#theory

215 interview questions in this topic.