What is feature hashing (the 'hashing trick') and when is it useful?
mediumAnswer
- Hash each category / n-gram to an index in a fixed-size vector using a fast hash function.
- Handles unbounded / very high cardinality without an explicit vocabulary (users, URLs, streaming categoricals).
- Trades collisions for a fixed memory footprint — controllable by the number of hash buckets.
- Used in Vowpal Wabbit, online learning, and large-scale linear models with billions of features.
- Downside: model interpretability drops; two colliding categories become inseparable.
Check yourself — multiple choice
- Hash the target instead of features
- Hash categories/n-grams into a fixed-size vector — bounded memory, tolerates collisions
- Same as one-hot encoding
- Only used for computer vision
Hashing trick = fixed-size feature vector with tolerated collisions ⇒ great for very high cardinality.
#encoding#features
Practise Supervised Learning
215 interview questions in this topic.