19 interview questions on feature engineering & encoding, each answered in full. Free to read, no account needed.
Which models need feature scaling and which don't?
easy- Scale features for models that use distances or gradient descent: k-NN, k-means, SVM, PCA, linear/logistic regression with regularization, and neural networks.
- Tree-based models (decision trees, random forest, gradient boosting) don't need scaling because splits are threshold-based and invariant to monotonic transforms.
What is target leakage and how do you prevent it?
medium- Target leakage is when a feature contains information about the label that would not be available at prediction time (e.g., using post-outcome features, or fitting a scaler/encoder on the full dataset before splitting).
- Prevent it by fitting all transformations only on the training fold (use pipelines), splitting temporally when time matters, and reviewing features for future information.
Ordinal vs nominal encoding — how do you choose?
easy- Ordinal encoding maps categories to integers (0, 1, 2, ...) — makes sense only for ordered categories (education level, star rating), or when passing to a tree-based model that will find splits regardless of numeric order.
- Never use it for unordered categories with a linear/distance model — the integers imply an ordering that doesn't exist.
- One-hot encoding is the safe default for unordered (nominal) categoricals with any model; target encoding for high-cardinality categoricals.
What is target encoding and when is it useful?
medium- Replace each category value with a statistic of the target for that category — typically the mean of y within the category (mean encoding).
- Great for high-cardinality categoricals (zip codes, product SKUs, user IDs) where one-hot creates thousands of columns.
- Adds signal from the label into the feature.
- Big risk: leakage.
- Always compute the encoding on the training fold only and apply to validation, or use K-fold target encoding, smoothing (blend with global mean by count), and noise injection.
- CatBoost's ordered boosting bakes this in.
How do you prevent leakage in target encoding?
hard- Never compute the encoding on the same rows you'll evaluate.
- Techniques: (1) K-fold target encoding — for each fold, compute encoding from the other folds; (2) Leave-one-out encoding — for row i, exclude i from the statistic; (3) Smoothing — blend per-category mean with global mean by category count so tiny categories don't overfit; (4) Additive Gaussian noise on the encoded value; (5) CatBoost's ordered target encoding — the correct principled version.
- Test set is encoded using the final training statistics only.
What is frequency (count) encoding?
easy- Replace each category with its count (or frequency) in the training set.
- Works well for tree models: high-frequency categories often behave differently from rare ones, and this captures that in one column instead of one-hot's thousands.
- Doesn't leak the target so it's safer than target encoding for high-cardinality categoricals.
- Downside: two categories with the same count get the same encoding, so it's information-losing — often combined with other encodings.
What is feature hashing (the 'hashing trick') and when is it useful?
medium- Hash each category / n-gram to an index in a fixed-size vector (eg,218columns) using a fast hash function.
- Handles unbounded / very high cardinality without an explicit vocabulary (users, URLs, streaming categoricals).
- Trades collisions for a fixed memory footprint — controllable by the number of hash buckets.
- Used in Vowpal Wabbit, online learning, and large-scale linear models with billions of features.
- Downside: model interpretability drops; two colliding categories become inseparable.
What is Weight of Evidence (WoE) encoding?
hard- For a binary target and each category c: WoE(c)=ln(P(x=c∣y=1)/P(x=c∣y=0)) — the log-ratio of positive-class share to negative-class share within that category, offset by the global class ratio.
- Comes from credit scoring: after WoE, categories are on a monotonic log-odds scale that plugs cleanly into logistic regression.
- Also often reported alongside Information Value (IV = sum (P1 - P0) * WoE) to rank predictive features.
How do you encode cyclical features like hour-of-day or day-of-week?
medium- Naive integer encoding breaks cyclicity: hour 23 and hour 0 look very far apart, but they're adjacent.
- Encode as two columns: sin(2*pi*x / P) and cos(2*pi*x / P) where P is the period (24, 7, 12, 365).
- Now the distance between 23 and 0 is small in feature space.
- Trees don't need cyclical encoding (they'll find the split), but linear models, distance methods, and neural networks benefit a lot.
- Extract multiple periods too (hour + day + month).
How do you turn text into features for a classical model?
easy- Bag-of-words (BoW): count each token per document, sparse count matrix.
- TF-IDF: down-weights common terms by document frequency — usually beats raw counts.
- N-grams: include word bigrams/trigrams to capture short phrases.
- Character n-grams (3-5 chars): robust to typos and language variation.
- All of these feed cleanly into linear models (log-reg / SVM), Naive Bayes, or gradient boosting.
- Modern deep alternative: use a pretrained transformer embedding (SBERT, E5) — but for many tasks TF-IDF + logistic regression is a very strong baseline.
You have a feature with 50,000 unique category values. How do you encode it?
medium- One-hot is out — 50k sparse columns kill most models.
- Options: (1) target encoding (with K-fold + smoothing) for gradient boosting or linear models; (2) frequency encoding to distinguish common vs rare; (3) feature hashing to a fixed-size vector; (4) entity embeddings — a learned dense vector per category, trained with a neural net or via factorization machines; (5) group rare categories into 'Other' below a min-count threshold.
- Often combine target + frequency encoding.
When should you manually engineer interaction features?
medium- For linear / logistic models, always — they can't discover interactions by themselves.
- For deep nets and gradient boosting, only in cases where you know a specific multiplicative or ratio structure matters (BMI=weight/height2,price⋅volumeforrevenue).
- If a domain expert says 'this feature only matters when X is high', build the interaction explicitly.
- Trees find interactions automatically but they may need very many trees to capture them cleanly.
When is discretization (binning) a useful feature transformation?
medium- Binning turns a continuous feature into categorical buckets — great when (1) the relationship with y is highly non-linear and non-monotonic, so a linear model can't capture it; (2) you want to expose interactions between binned features cleanly; (3) domain knowledge suggests thresholds (age < 18, 18-65, > 65).
- Trees don't benefit — they discretize on their own.
- Choose bins by equal-width, equal-frequency, or supervised (decision-tree-based binning that maximizes mutual information with y).
StandardScaler vs MinMaxScaler vs RobustScaler — how do you choose?
easy- StandardScaler: mean 0 / std 1.
- Best for approximately Gaussian data and models sensitive to variance (linear + regularization, SVM, PCA, neural nets).
- MinMaxScaler: to [0, 1].
- Useful when features must be strictly positive (e.g., neural nets with sigmoid outputs, image pixels) or when you want a bounded feature range.
- RobustScaler: subtract median, divide by IQR.
- Best when features contain outliers you don't want to remove — outliers don't distort the scale as they would with mean/std.
Filter, wrapper, and embedded feature selection — how do they differ?
medium- Filter: rank features by a statistic (mutualinformation,χ2,ANOVAF,variance) — fast, model-agnostic, but ignores feature interactions and doesn't optimize for the downstream model.
- Wrapper: try different feature subsets, retrain the model, pick the best — most accurate but expensive (forward, backward, RFE).
- Embedded: the model itself selects features during training (L1 in linear models, tree importance, or LightGBM's built-in feature filtering).
- Embedded is usually the best cost/quality tradeoff.
How exactly does SMOTE generate synthetic minority-class samples?
medium- For each minority-class point x, find its k nearest minority-class neighbours (default k=5).
- Pick one at random; call it x'.
- Generate a synthetic point xnew=x+λ⋅(x−x) where lambda is uniform in [0, 1] — a random interpolation on the segment between two real minority points.
- Repeat until the desired oversampling ratio is met.
- Works only on continuous features (interpolation makes no sense for categoricals — use SMOTE-NC or SMOTE-N variants for mixed / categorical data).
What does ColumnTransformer do and when do you need it?
easy- Applies different transformers to different subsets of columns in the same DataFrame and stitches the outputs back together into a single feature matrix.
- Standard use: one-hot encode categoricals, standardize numerics, drop or passthrough others — all in one step, fit inside a Pipeline.
- Combined with makecolumnselector you can select columns by dtype or name pattern instead of hard-coding indices.
- The idiomatic way to handle mixed tabular data in scikit-learn.
You did StandardScaler().fittransform(X) then split into train/test. Why is this wrong?
easy- The scaler was fit on the full dataset including the test set, so mean and std leak information from test data into training preprocessing.
- Effect: test metric is slightly optimistic, sometimes a lot on small data.
- Correct order: split first, fit on train only, transform train and test with the same fitted scaler.
- In practice: wrap everything in a Pipeline and pass it to crossvalscore or GridSearchCV — the split-then-fit invariant is automatic.
Why is target encoding dangerous, and how do you do it safely?
hard- Target encoding replaces a category with the mean target for that category, so the target leaks directly into the feature.
- Fit it on the whole training set and the model memorizes rare categories, which look perfectly predictive in training and useless later.
- Do it safely with out-of-fold encoding: for each fold, compute the mapping from the other folds only.
- Add smoothing toward the global mean so a category with three rows is pulled hard to the prior.
- Keep the encoder inside the cross-validation pipeline, never as a preprocessing step applied before splitting.