EasyDeepLearn
Statistics Fundamentals · section 2 of 15

Common distributions

5 interview questions on common distributions, each answered in full. Free to read, no account needed.

Log-normal distribution and its uses.

medium
  • X ~ LogNormal(μ,  σ2)\mathrm{LogNormal}({\mu}, \;{\sigma}^{2}) if log⁡(X)\operatorname{log}(X) ~ N(μ,  σ2)N({\mu}, \;{\sigma}^{2}).
  • Right-skewed, always positive.
  • Mean: exp⁡(μ  +  σ2/2)\operatorname{exp}({\mu}\; + \;{\sigma}^{2} / 2); Median: exp⁡(μ)\operatorname{exp}({\mu}).
  • Uses: incomes, prices, response times, file sizes — quantities that are strictly positive with right skew.
  • Products of many small independent factors (like the multiplicative CLT) tend log-normal.
  • In DS: use log-transform to make skewed features more Gaussian-like for linear models.
#distributionsPermalink & quiz →

What is a power-law distribution and how do you spot one?

medium
  • P(X > x) ∝ x^(-α) for large x.
  • Very heavy right tail — a few extreme observations dominate the mean.
  • Examples: word frequencies, city sizes, wealth, network degree distributions (Zipf's / Pareto's law).
  • Spot them by plotting log⁡(rank⁡)\operatorname{log}(\operatorname{rank}) vs log⁡(size)\operatorname{log}(\mathrm{size}) — a straight line indicates power law.
  • Consequences: sample means are unstable, need heavy-tailed methods (median, quantile regression, robust stats).
#distributionsPermalink & quiz →

Beta distribution: setup and use case.

medium
  • X ~ Beta(α, β): support [0, 1], flexible shape controlled by α, β.
  • E[X] = α / (α+β); Var⁡  =  αβ  /  ((α+β)2(α+β+1))\operatorname{Var}\; = \;{\alpha}{\beta}\; / \;(({\alpha} + {\beta})^{2}({\alpha} + {\beta} + 1)).
  • Very flexible: uniform (α=β=1), U-shaped (α, β < 1), bell (α = β > 1), skewed (unequal α, β).
  • Standard Bayesian prior for probabilities: conjugate to Bernoulli/Binomial — Beta(α, β) + k successes in n trials → Beta(α+k, β+n-k).
  • Used for A/B testing priors and click-through-rate modeling.
#distributions#bayesianPermalink & quiz →

Gamma distribution and when it appears.

medium
  • X ~ Gamma(k, θ): support (0, ∞).
  • PDF ∝ x^(k-1) * e^(-x/θ).
  • E[X] = kθ; Var⁡  =  kθ2\operatorname{Var}\; = \;k{\theta}^{2}.
  • Special cases: Exponential  =  Γ(1,  θ)\mathrm{Exponential}\; = \;\Gamma(1, \;{\theta}); Erlang  =  Γ(integer  k)\mathrm{Erlang}\; = \;\Gamma(\mathrm{integer}\;k).
  • Uses: waiting time until k events in Poisson process, survival times, Bayesian conjugate prior for Poisson rate and normal precision.
  • Common as a heavy-tailed regression target (Gamma GLM for positive continuous outcomes: durations, insurance claims).
#distributions#bayesianPermalink & quiz →

Dirichlet distribution — where does it show up in ML?

hard
  • Dirichlet(α1,  ,  αK)\mathrm{Dirichlet}({\alpha}_{1}, \;, \;{\alpha}_{K}): distribution over probability vectors on the (K-1)-simplex (positive components summing to 1).
  • Multivariate generalization of Beta.
  • Conjugate prior for the categorical / multinomial distributions.
  • Uses: topic models (LDA — Latent Dirichlet Allocation), mixture-model weights, softmax priors, RL policy priors.
  • Concentration parameter Σαi{\Sigma}{\alpha}_{i} controls how peaked (large α) vs uniform (small α) the samples are.
#distributions#bayesianPermalink & quiz →

Practise Statistics Fundamentals

All 15 sectionsFlashcardsQuiz