How do you turn text into features for a classical model?
easyAnswer
- Bag-of-words (BoW): count each token per document, sparse count matrix.
- TF-IDF: down-weights common terms by document frequency — usually beats raw counts.
- N-grams: include word bigrams/trigrams to capture short phrases.
- Character n-grams (3-5 chars): robust to typos and language variation.
- All of these feed cleanly into linear models (log-reg / SVM), Naive Bayes, or gradient boosting.
- Modern deep alternative: use a pretrained transformer embedding (SBERT, E5) — but for many tasks TF-IDF + logistic regression is a very strong baseline.
Check yourself — multiple choice
- Only use raw text
- Bag-of-words / TF-IDF / n-grams (word or char) — solid classical baselines
- Only word2vec works
- Text can't be used with linear models
BoW / TF-IDF / n-grams — classical, sparse, model-agnostic and often surprisingly strong.
#text#features#feature-engineering#encoding
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Ordinal vs nominal encoding — how do you choose?
- What is target encoding and when is it useful?
- How do you encode cyclical features like hour-of-day or day-of-week?
- You have a feature with 50,000 unique category values. How do you encode it?
- When should you add interaction features to a linear model?
- How do decision trees handle categorical features?