EasyDeepLearn
Unsupervised Learning · section 1 of 9

Clustering basics

69 interview questions on clustering basics, each answered in full. Free to read, no account needed.

How does k-means work and what are its main limitations?

easy
  • Initialize k centroids (k-means++ is best).
  • Assign each point to the nearest centroid, then recompute centroids as the mean of assigned points; repeat until convergence.
  • Limitations: you must choose k, it assumes spherical clusters of similar size, it's sensitive to scale and outliers, and it can get stuck in local minima (mitigated by multiple restarts).
#clusteringPermalink & quiz →

How do you choose the number of clusters k?

easy
  • Use multiple signals, not one: the elbow method on inertia (look for the bend), the silhouette score (higher is better, up to 1), the gap statistic, and domain knowledge.
  • For density-based clustering (DBSCAN, HDBSCAN) you don't set k at all — you set density parameters instead.
#clustering#evaluationPermalink & quiz →

When would you pick DBSCAN over k-means?

medium
  • Pick DBSCAN when clusters are non-convex or of very different densities, when you don't want to specify k, and when you want an explicit notion of noise/outliers. k-means struggles with irregular shapes and forces every point into a cluster.
  • DBSCAN needs two parameters: eps (neighborhood radius) and min⁡samples\operatorname{min}_{\mathrm{samples}}.
#clusteringPermalink & quiz →

What does the silhouette score measure?

medium
  • For each point, silhouette = (b - a) / max(a, b), where a is the mean distance to points in its own cluster and b is the mean distance to points in the nearest other cluster.
  • Values close to +1 mean well-separated clusters, near 0 means overlapping, negative means misassigned.
  • The overall score is the mean across all points.
#clustering#evaluationPermalink & quiz →

What is hierarchical clustering and when is it useful?

medium
  • It builds a tree (dendrogram) of nested clusters, either agglomerative (bottom-up: merge closest clusters) or divisive (top-down).
  • You cut the tree at a chosen height to get k clusters.
  • Useful when you want to explore cluster structure at multiple scales, or when you don't know k in advance and want a visual.
  • Downside: O(n2)O(n^{2}) memory, so it doesn't scale beyond ~10k points.
#clusteringPermalink & quiz →

Gaussian Mixture Model vs k-means — what's the difference?

medium
  • k-means gives hard assignments and assumes spherical, equal-variance clusters.
  • A GMM models each cluster as a multivariate Gaussian and returns soft (probabilistic) memberships; it handles ellipsoidal clusters and different covariances.
  • GMM is fit with the EM algorithm.
  • Use GMM when clusters overlap, have different shapes, or when you need probabilities.
#clustering#densityPermalink & quiz →

What is linkage in agglomerative clustering?

hard
  • Linkage defines the distance between two clusters.
  • Single linkage = min pairwise distance (chains, elongated clusters).
  • Complete linkage = max pairwise distance (compact clusters, sensitive to outliers).
  • Average linkage = mean pairwise distance.
  • Ward linkage minimizes within-cluster variance (produces balanced, compact clusters — most common default).
#clusteringPermalink & quiz →

Why does clustering degrade in high dimensions?

medium
  • In high dimensions, pairwise distances concentrate — points look almost equidistant — so distance-based clustering loses discriminative power.
  • Also, irrelevant features add noise.
  • Fixes: dimensionality reduction (PCA/UMAP) before clustering, feature selection, or use models less reliant on Euclidean distance (e.g., subspace clustering).
#clustering#dimensionality-reductionPermalink & quiz →

How does k-means++ initialization work?

medium
  • Pick the first centroid uniformly at random.
  • For each subsequent centroid, sample point x with probability proportional to D(x)2D(x)^{2} where D(x) is the distance to the nearest already-chosen centroid.
  • Result: centroids spread out over the data → far better starting point than random.
  • Expected O(log k) approximation to optimal SSE.
  • Default in scikit-learn and Spark MLlib.
#clusteringPermalink & quiz →

When is mini-batch k-means preferred?

medium
  • Large datasets (millions of points).
  • Sample small random mini-batches (~1000 points), update centroids incrementally toward batch means with a learning rate.
  • Slightly worse SSE than full k-means, but 10-100x faster and streaming-friendly.
  • Standard for big-data clustering (Spark, scikit-learn's MiniBatchKMeans).
  • For tabular data > 100k rows, prefer it over standard k-means.
#clusteringPermalink & quiz →

k-medoids vs k-means — the difference.

medium
  • k-medoids uses actual data points as cluster representatives (medoids) instead of arithmetic means.
  • Robust to outliers (the median is more robust than the mean).
  • Works with any distance/dissimilarity — not just Euclidean (categorical, custom).
  • Cost: O(k(n−k)2)O(k(n - k)^{2}) per iteration (PAM algorithm) — slower than k-means.
  • Use for outlier-heavy data or non-Euclidean distances.
#clusteringPermalink & quiz →

What is fuzzy c-means?

hard
  • Each point has a membership degree uiju_{\mathrm{ij}} ∈ [0,1] for every cluster (rows sum to 1).
  • Centroids weighted by uijmu_{\mathrm{ij}}^{m} for a fuzzifier m > 1 (m=2 typical).
  • Update alternates between soft assignments and weighted means.
  • Use when clusters overlap naturally and hard assignment loses information.
  • Sits between k-means (m→1: hard) and uniform (m→∞).
#clusteringPermalink & quiz →

HDBSCAN — how does it improve on DBSCAN?

hard
  • Runs DBSCAN over all density levels and extracts the most stable clusters — avoids picking a single eps.
  • Handles clusters of varying density (DBSCAN's biggest failure mode).
  • Output: cluster labels + outlier scores + soft memberships.
  • Standard practice for real-world clustering when densities are heterogeneous (customer segments, sensor data).
  • Slightly slower but no eps tuning needed.
#clustering#densityPermalink & quiz →

OPTICS — what does it produce and how do you use it?

hard
  • Computes a reachability plot ordering all points by density-reachability.
  • Reveals cluster hierarchy visually as valleys in the plot.
  • Extract clusters at any density threshold from a single run.
  • Slower than DBSCAN (O(n log n) with index) but more flexible.
  • Used as an exploratory tool when you don't know cluster densities in advance.
#clustering#densityPermalink & quiz →

Spectral clustering — the core idea.

hard
  • Build similarity graph on data (k-NN or ε-neighborhood).
  • Compute graph Laplacian L.
  • Find eigenvectors of the k smallest eigenvalues → embed points into RkR^{k}.
  • Run k-means in this embedding.
  • Captures non-convex, manifold-shaped clusters that k-means can't handle.
  • Cost: O(n3)O(n^{3}) eigendecomposition or O(n2)O(n^{2}) with Nyström — scales to ~50k.
  • Modern uses: community detection in graphs, image segmentation.
#clusteringPermalink & quiz →

Affinity propagation — how does it work?

hard
  • Message-passing algorithm exchanging 'responsibility' and 'availability' messages between all pairs of points until exemplars emerge.
  • No k needed — 'preference' parameter influences the number of clusters.
  • Handles non-convex clusters, but O(n2)O(n^{2}) memory + slow.
  • Use for small-to-medium exploratory clustering when k is unknown and interpretable exemplars matter (e.g., customer archetypes).
#clusteringPermalink & quiz →

Mean shift clustering — mechanism and use case.

hard
  • Non-parametric mode-seeking.
  • For each point, repeatedly shift it toward the local mean within a bandwidth-radius kernel until convergence.
  • Points converging to the same mode form a cluster.
  • No k needed — clusters = number of density modes.
  • Bandwidth is the key parameter.
  • Uses: image segmentation, mode tracking.
  • Slow (O(n2)  per  iteration)(O(n^{2})\;\mathrm{per}\;\mathrm{iteration}) but robust to cluster shape.
#clustering#densityPermalink & quiz →

BIRCH — when to use it?

hard
  • Streaming / very large dataset clustering.
  • Builds a CF-Tree (Cluster Feature summaries: N, LS, SS) in a single pass over data — leaves become micro-clusters.
  • Then apply hierarchical or k-means clustering on the leaves.
  • Memory-efficient, near-linear in n, streaming-friendly.
  • Weakness: assumes spherical clusters (built-in radius threshold).
  • Standard for terabyte-scale exploratory clustering.
#clusteringPermalink & quiz →

EM for a GMM — one iteration explicitly.

hard
  • E-step: for each point xix_{i} and cluster j, γij  =  πj{\gamma}_{\mathrm{ij}}\; = \;{\pi}_{j} N(xi  ∣  μj,  Σj)  /  ΣkN(x_{i}\; \mid \;{\mu}_{j}, \;{\Sigma}_{j})\; / \;{\Sigma}_{k} πk{\pi}_{k} N(xi  ∣  μk,  Σk)N(x_{i}\; \mid \;{\mu}_{k}, \;{\Sigma}_{k}) (responsibilities).
  • M-step: update πj  =  Σi{\pi}_{j}\; = \;{\Sigma}_{i} γij  /  n{\gamma}_{\mathrm{ij}}\; / \;n, μj  =  Σi{\mu}_{j}\; = \;{\Sigma}_{i} γij{\gamma}_{\mathrm{ij}} xi  /  Σix_{i}\; / \;{\Sigma}_{i} γij{\gamma}_{\mathrm{ij}}, Σj  =  Σi{\Sigma}_{j}\; = \;{\Sigma}_{i} γij{\gamma}_{\mathrm{ij}} (xi  −  μj)(x_{i}\; - \;{\mu}_{j})(xi  −  μj)(x_{i}\; - \;{\mu}_{j})' / Σi{\Sigma}_{i} γij{\gamma}_{\mathrm{ij}}.
  • Converges to local max of log-likelihood.
  • Init sensitive → use k-means for starting points.
#clustering#densityPermalink & quiz →

GMM covariance types — which do you pick?

hard
  • Full: each cluster has its own general Σj{\Sigma}_{j} — most flexible, most parameters (p2)(p^{2}).
  • Tied: all clusters share one Σ.
  • Diagonal: axis-aligned ellipsoids (p params per cluster).
  • Spherical: Σ  =  σ2{\Sigma}\; = \;{\sigma}^{2} I (1 param) — closest to k-means.
  • Rule: use full for small p, tied or diagonal for high p to avoid overfitting.
  • Cross-validate log-likelihood or BIC to choose.
#clustering#densityPermalink & quiz →

How does BIC choose k in GMM?

medium
  • BIC = -2 log L + p log n where p = number of free parameters, n = dataset size.
  • Lower is better.
  • Balances fit (log L) against complexity (p log n).
  • For GMM, p grows with k and covariance type.
  • Standard practice: fit GMM at k = 1..KmaxK_{\mathrm{max}}, plot BIC vs k, pick the minimum (or the elbow).
  • More reliable than AIC for clustering.
#clustering#density#evaluationPermalink & quiz →

How does the gap statistic work?

hard
  • Compare observed within-cluster dispersion WkW_{k} to expected WkW_{k} under a null reference distribution (uniform in the bounding box).
  • Gap(k)  =  E⋅[log⁡  Wk⋅]  −  log⁡\mathrm{Gap}(k)\; = \;E \cdot [\operatorname{log}\;W_{k} \cdot ]\; - \;\operatorname{log} WkW_{k}.
  • Pick smallest k such that Gap(k) ≥ Gap(k+1)  −  sk+1\mathrm{Gap}(k + 1)\; - \;s_{k + 1}.
  • More principled than the elbow; handles cluster-vs-no-cluster case (returns k=1 when data is unstructured).
  • Costly (needs B ~ 20 reference simulations).
#clustering#evaluationPermalink & quiz →

Why can the elbow method fail?

medium
  • (1) No clear elbow — smooth curve, subjective.
  • (2) Different features scales dominate SSE → misleading k.
  • (3) Sees the biggest cluster count as always better (SSE monotonically decreases).
  • Use with silhouette or gap statistic as tiebreakers.
  • In practice, teams combine 3 metrics + domain knowledge; blindly trusting elbow SSE is a common bug in production clustering pipelines.
#clustering#evaluationPermalink & quiz →

Calinski-Harabasz index — what is it?

medium
  • CH(k)  =  [Tr(Bk)  /  (k−1)]  /  [Tr(Wk)  /  (n−k)]\mathrm{CH}(k)\; = \;[\mathrm{Tr}(B_{k})\; / \;(k - 1)]\; / \;[\mathrm{Tr}(W_{k})\; / \;(n - k)] — ratio of between-cluster dispersion to within-cluster dispersion (higher = better).
  • Analogue to an F-statistic for clustering.
  • Fast, no ground truth needed.
  • Bias: tends to favor higher k.
  • Complements silhouette.
  • Combined with domain, one of the standard internal indices.
#clustering#evaluationPermalink & quiz →

Davies-Bouldin index — how is it computed?

hard
  • DB = (1/k) Σi{\Sigma}_{i} max⁡j≠i\operatorname{max}_{j \ne i} [(si  +  sj)  /  dij][(s_{i}\; + \;s_{j})\; / \;d_{\mathrm{ij}}] where si  =  within−clusters_{i}\; = \;\mathrm{within} - \mathrm{cluster} scatter, dij  =  distanced_{\mathrm{ij}}\; = \;\mathrm{distance} between centroids.
  • Lower is better.
  • Rewards compact + well-separated clusters.
  • Fast, ground-truth-free.
  • Weak on non-convex clusters (assumes centroid representation).
  • Standard internal metric alongside silhouette.
#clustering#evaluationPermalink & quiz →

Adjusted Rand Index vs NMI — external cluster metrics.

medium
  • Both compare a clustering to ground-truth labels (when available).
  • ARI = (RI - E[RI]) / (max RI - E[RI]) → 0 for random, 1 for perfect match, can be negative.
  • Corrects for chance.
  • NMI = mutual information normalized to [0,1] — measures how much info the clustering carries about labels.
  • AMI = adjusted NMI (chance-corrected).
  • Rule: report both — they disagree in edge cases.
#clustering#evaluationPermalink & quiz →

How do you cluster mixed numeric + categorical data?

hard
  • (1) Gower distance — averages appropriate distances per feature type (Manhattan for numeric, matching for categorical).
  • (2) k-prototypes — extends k-means with categorical dissimilarity.
  • (3) Embed categorical as one-hot / target encoding, scale numerics, then k-means (loses interpretability).
  • (4) UMAP with metric='gower' → then cluster in embedding.
  • Most production tabular clustering uses Gower + PAM or k-prototypes.
#clustering#similarityPermalink & quiz →

Why do you scale features before k-means?

easy
  • Euclidean distance is scale-sensitive: a feature with 1000-range dominates one with 0-1 range.
  • Standardize (z-score) so features contribute equally.
  • Exception: if scale carries meaning (spatial coordinates in meters), don't scale.
  • Also: outliers stretch scale → robust scaling (median / IQR) can be better.
  • Missing this is one of the most common bugs in first-time clustering pipelines.
#clustering#similarityPermalink & quiz →

Cosine similarity vs Euclidean — when do you use cosine?

medium
  • Cosine similarity = x · y  /  (∣∣x∣∣  ∣∣y∣∣)y\; / \;( \mid \mid x \mid \mid \; \mid \mid y \mid \mid ) — measures angle, ignores magnitude.
  • Use when magnitude is irrelevant: TF-IDF vectors (doc length varies), sentence embeddings, user-item preferences.
  • Use Euclidean when magnitude matters (physical measurements, absolute values).
  • In practice: L2-normalize embeddings then use Euclidean → equivalent to cosine but works in standard k-means / FAISS indexes.
#clustering#similarityPermalink & quiz →

How do outliers affect k-means and how do you handle them?

medium
  • Outliers pull centroids toward themselves (mean is non-robust) → distort cluster boundaries.
  • Handle: (1) pre-cluster outlier removal via IsolationForest / z-score, (2) use k-medoids (median-like), (3) use DBSCAN which explicitly flags outliers as noise, (4) trim tails / winsorize before clustering.
  • Always plot the largest / smallest per-cluster distances after fitting — extreme values are usually outliers or mistakes in the pipeline.
#clustering#anomaly-detectionPermalink & quiz →

How do you assess cluster stability?

hard
  • (1) Bootstrap: resample data, re-cluster, measure Jaccard overlap between cluster memberships.
  • (2) Random-init variability: many k-means restarts, measure ARI between runs.
  • (3) Perturbation: add small noise, check whether cluster boundaries shift.
  • Stable clusters survive resampling / perturbation.
  • Standard sanity check before shipping a customer segmentation.
#clustering#evaluationPermalink & quiz →

What is consensus clustering?

hard
  • Run clustering many times (varying algorithm, k, subsample, init).
  • Build co-clustering matrix Mij  =  fractionM_{\mathrm{ij}}\; = \;\mathrm{fraction} of runs where i and j were in the same cluster.
  • Cluster M (usually hierarchical) → consensus assignment.
  • Standard in bioinformatics for gene expression subtype discovery.
  • More robust than any single run; helps identify natural k when peaks in M are stable across k.
#clustering#evaluationPermalink & quiz →

Online k-means — how does it work?

hard
  • For each incoming point, find nearest centroid and update it: μj{\mu}_{j} ← μj  +  η{\mu}_{j}\; + \;{\eta} (x  −  μj)(x\; - \;{\mu}_{j}) with decaying learning rate.
  • Streaming-friendly, constant memory.
  • Sensitive to arrival order and cluster drift.
  • Modern alternative: mini-batch k-means; production streaming systems (Kafka + Flink) prefer mini-batch for robustness.
  • Concept drift → reset centroids periodically.
#clusteringPermalink & quiz →

'Cluster then classify' — when does it help?

medium
  • Add cluster label as an extra feature to a supervised model.
  • Sometimes lifts performance on data with strong latent groups (customer segments, region clusters) — the label encodes a non-linear grouping the model wouldn't easily learn.
  • But: usually replaces with target-encoded categoricals and interaction features that a GBDT can learn directly.
  • Test with CV — often marginal or negative for modern tree models.
#clustering#applicationsPermalink & quiz →

How do you cluster time series?

hard
  • (1) Feature extraction: extract statistics (mean, variance, autocorrelation, FFT bands) and cluster the feature vectors.
  • (2) Dynamic Time Warping (DTW) distance + k-medoids / hierarchical.
  • (3) Shape-based: k-Shape (normalized cross-correlation).
  • (4) Embed with sequence autoencoder / TS2Vec then cluster embeddings.
  • Rule: DTW for shape similarity, feature-based for interpretability, embedding-based for large heterogeneous series.
#clustering#applicationsPermalink & quiz →

What is Dynamic Time Warping?

hard
  • Similarity between two sequences allowing non-linear alignment (stretch/compress).
  • Finds the best matching via dynamic programming through a cost matrix.
  • Robust to phase shifts and different lengths.
  • Cost O(mn) per pair — expensive for large series (mitigated  by  lower−bounds  like  LBKeogh,  sakoe−chiba  constraint  window)(\mathrm{mitigated}\;\mathrm{by}\;\mathrm{lower} - \mathrm{bounds}\;\mathrm{like}\;\mathrm{LB}_{\mathrm{Keogh}}, \;\mathrm{sakoe} - \mathrm{chiba}\;\mathrm{constraint}\;\mathrm{window}).
  • Standard for shape-based time-series similarity in speech, ECG, gesture recognition.
#clustering#similarityPermalink & quiz →

Community detection in graphs — main approaches.

hard
  • (1) Modularity optimization: Louvain, Leiden (greedy hierarchical) — scale to millions of nodes, standard default.
  • (2) Spectral: eigenvectors of the graph Laplacian → k-means.
  • (3) Label propagation (fast, non-deterministic).
  • (4) Stochastic Block Models (probabilistic).
  • Modern: GNN-based (DeepWalk / Node2Vec + clustering).
  • Leiden fixes Louvain's resolution-limit + connected-community issues — usually the default now.
#clustering#applicationsPermalink & quiz →

How do you interpret / visualize clusters?

medium
  • (1) 2D projection with UMAP / t-SNE colored by cluster.
  • (2) Feature-importance-per-cluster: mean of each feature per cluster vs global mean → describes each group.
  • (3) Decision-tree surrogate: fit a small tree predicting cluster label → rule-based description.
  • (4) Representative sample per cluster (medoid + top / bottom feature scores).
  • Standard deliverable for a customer segmentation report.
#clustering#visualizationPermalink & quiz →

How do you find nested or hierarchical structure?

medium
  • (1) Agglomerative clustering with a dendrogram → cut at different heights for different granularity.
  • (2) HDBSCAN's condensed cluster tree — natural hierarchy.
  • (3) Recursively apply k-means with k=2.
  • (4) Bayesian nonparametric methods (Dirichlet Process) let hierarchy emerge from data.
  • Useful for taxonomies, org charts, biological classifications, topic hierarchies.
#clusteringPermalink & quiz →

Dirichlet Process — how does it help clustering?

hard
  • Bayesian nonparametric: number of clusters is inferred rather than pre-specified.
  • New points can start new clusters with a probability depending on concentration parameter α (Chinese Restaurant Process metaphor: 'rich get richer' + occasional new table).
  • Fit via Gibbs / variational inference.
  • Automatic model complexity selection.
  • Foundation of DPGMM (Dirichlet Process GMM) in scikit-learn.
#clustering#densityPermalink & quiz →

One cluster dominates in k-means — what do you do?

medium
  • (1) Try higher k — the 'big' cluster may naturally split.
  • (2) Log-scale skewed features.
  • (3) Try k-medoids (robust to imbalanced density).
  • (4) Use HDBSCAN which handles varying densities.
  • (5) Balanced k-means variants that constrain cluster size.
  • (6) Sanity-check with silhouette per cluster — often the dominant cluster is a garbage-collector for outliers or non-standardized features.
#clusteringPermalink & quiz →

Interview: 'you're asked to segment 10M customers — how?'

hard
  • (1) Sample down to 1M for prototype.
  • (2) Feature engineering: RFM (recency, frequency, monetary), behavioral counts, categorical embeddings.
  • (3) Scale (log-transform skewed features, z-score).
  • (4) PCA/UMAP to 20-50 dim for compute.
  • (5) Mini-batch k-means or HDBSCAN across candidate k (silhouette + business interpretability).
  • (6) Profile clusters with per-feature means + decision-tree surrogate.
  • (7) Stability check via bootstrap.
  • (8) Deliver actionable segments with names + business hypotheses.
#clustering#interview#applicationsPermalink & quiz →

How do you monitor cluster drift over time?

hard
  • Track: (1) fraction of new points assigned to each cluster (should be stable).
  • (2) Distribution of intra-cluster distances (blow-up = drift).
  • (3) Silhouette on rolling window.
  • (4) Population Stability Index of cluster proportions.
  • (5) Re-cluster periodically, measure ARI/NMI vs previous partition (< 0.7 → cluster meaning shifted).
  • When drift detected, retrain segmentation and rename / merge / split clusters with business.
#clustering#monitoring#applicationsPermalink & quiz →

Mixture models for density estimation — beyond GMM.

hard
  • GMM assumes Gaussian components.
  • Alternatives: mixture of t-distributions (heavier tails, robust), mixture of Dirichlet (categorical), mixture density networks (NN outputs mixture parameters — good for heteroscedastic regression), Bayesian nonparametric DP-mixtures (infer k).
  • Choose component distribution based on data support (bounded → beta; positive → gamma; heavy tails → t).
#density#clusteringPermalink & quiz →

LDA vs NMF for topic modeling — which do you pick?

medium
  • LDA: probabilistic — documents = Dirichlet mixture of topics, topics = Dirichlet mixture over words.
  • Fit via variational Bayes or Gibbs.
  • Interpretable but sensitive to hyperparameters (α, β).
  • NMF: matrix factorization of TF-IDF, faster and often more coherent on short text.
  • LDA better for longer documents / traditional corpora; NMF better for short noisy text (tweets, reviews).
  • Both superseded by embedding-based topic models (BERTopic) on modern short text.
#nlp#text#clusteringPermalink & quiz →

How does BERTopic work?

medium
  • (1) Embed documents with sentence-transformers.
  • (2) Reduce dimensionality with UMAP.
  • (3) Cluster with HDBSCAN.
  • (4) Extract topic keywords via class-based TF-IDF (c-TF-IDF): TF-IDF where each 'document' is the concatenation of cluster texts.
  • Interpretable, handles short noisy text, supports topic modeling over time and hierarchical topics.
  • Modern default for exploratory topic modeling on real-world corpora.
#nlp#text#clusteringPermalink & quiz →

Applications of clustering in NLP.

medium
  • (1) Topic modeling / document grouping (LDA, NMF, BERTopic).
  • (2) Word sense discovery: cluster contextualized embeddings of ambiguous words.
  • (3) Sentence deduplication in training data (embed + cluster + keep centroids).
  • (4) User intent discovery in chatbot logs.
  • (5) Named entity clustering across languages.
  • (6) Query intent clustering for search.
  • All exploit unsupervised structure discovery.
#nlp#applications#clusteringPermalink & quiz →

How do you cluster images at scale?

medium
  • (1) Feature extraction: pass through pretrained CNN or ViT (DINOv2, CLIP) → embeddings.
  • (2) UMAP / PCA to ~50 dim for compute.
  • (3) HDBSCAN or mini-batch k-means.
  • (4) For huge scale (billions): FAISS + BIRCH or streaming k-means.
  • (5) Semi-supervised: use CLIP embeddings + text queries as prompts for prototype clustering.
  • Modern: DINOv2 embeddings + k-means or HDBSCAN is production default.
#applications#clusteringPermalink & quiz →

User behavior segmentation — feature engineering.

medium
  • (1) RFM: Recency (days since last activity), Frequency (activities per period), Monetary (spend).
  • (2) Behavioral counts: events per day/week.
  • (3) Categorical: preferred category, region, device.
  • (4) Sequences: transformer / GRU embed of event sequence.
  • (5) Time-decay weighted (recent behavior matters more).
  • (6) Log-transform skewed monetary.
  • Then z-score, PCA/UMAP → k-means or HDBSCAN.
  • Standard for marketing / product analytics segments.
#applications#clusteringPermalink & quiz →

Clustering in single-cell RNA-seq — the standard pipeline.

hard
  • (1) QC + normalize (log-CPM, SCTransform).
  • (2) HVG selection.
  • (3) PCA to 50 dim.
  • (4) Batch correction (Harmony, scVI).
  • (5) k-NN graph in PCA space.
  • (6) Leiden community detection.
  • (7) UMAP for visualization.
  • (8) Marker gene analysis per cluster to identify cell types.
  • Modern: scVI / scANVI for VAE-based normalization + integration; totalVI for multimodal (CITE-seq).
#applications#clusteringPermalink & quiz →

Interview: cluster 5M e-commerce customers for a marketing campaign.

hard
  • (1) Define business goal (targeting, retention, cross-sell?).
  • (2) Sample 500k for prototyping.
  • (3) Build RFM + behavioral features + demographic.
  • (4) Log-transform skewed features + z-score.
  • (5) PCA/UMAP to 30 dim.
  • (6) Try mini-batch k-means (k=5-10) and HDBSCAN.
  • (7) Metrics: silhouette + business KPI (per-cluster conversion / LTV).
  • (8) Profile clusters: mean features + tree surrogate + medoid examples + name each.
  • (9) Stability check.
  • (10) Deploy to full 5M via saved pipeline; retrain quarterly.
#interview#clustering#applicationsPermalink & quiz →

Interview: how would you group log messages from a large distributed system?

hard
  • (1) Parse log: extract template (Drain algorithm, LogPunk) → replace variables (timestamps, IPs, numbers) with placeholders.
  • (2) Vectorize templates via TF-IDF or sentence-transformers on remaining text.
  • (3) Cluster with HDBSCAN → template categories.
  • (4) Anomaly detection: rare template = novel event.
  • (5) Interactive drill-down: click cluster → see representative logs + timeline.
  • Standard in Splunk, DataDog logs, Elasticsearch's ML.
  • Log2Vec / LogBERT are modern DL variants.
#interview#applications#clusteringPermalink & quiz →

Interview: business asks 'why is customer X in Cluster 3?' — how do you explain?

medium
  • (1) Show customer X's features + Cluster 3 mean/std for each feature → highlight top-3 features driving assignment (largest z-score contribution).
  • (2) Show representative medoid / centroid of Cluster 3 alongside X.
  • (3) Decision-tree surrogate: rule-based description of Cluster 3 (e.g. 'high-frequency + high-monetary + monthly recency').
  • (4) SHAP values on the cluster assignment probability.
  • (5) Contrast: how would X have to change to belong to Cluster 2 instead?
  • Standard for stakeholder-friendly cluster interpretability.
#interview#applications#clusteringPermalink & quiz →

Interview: 'you have to pick k for the executive team — walk me through it.'

medium
  • (1) Business first: how many actionable groups can marketing / product operate?
  • Usually 3-8 in practice.
  • (2) Compute silhouette, gap statistic, BIC (if GMM) across k = 2-15.
  • (3) Plot metrics — find plateau / knee.
  • (4) Choose k that satisfies business AND metrics (e.g. silhouette > 0.3 acceptable, gap not too noisy).
  • (5) Stability check: does the top-k survive resampling?
  • (6) Present 2-3 candidates with pros/cons; let the exec team choose the interpretation.
#interview#clusteringPermalink & quiz →

Interview: after segmentation, one segment has 60% women but each region shows 40%. Why?

hard
  • Simpson's paradox — different regions have different sizes / proportions in that cluster.
  • Verify with contingency table: within each region, the segment may be 40% women, but the sample-size-weighted overall proportion is 60% because certain regions dominate the segment.
  • Fix by post-stratifying — report within-region proportions if that's what stakeholders care about.
  • Also check for sampling bias / SRM in segmentation.
#interview#causal-inference#clusteringPermalink & quiz →

Interview: how would you cluster 1B rows on a budget?

hard
  • (1) Sample 1M for prototype clustering.
  • (2) Feature-engineer + reduce (PCA / hashed features).
  • (3) Mini-batch k-means on the sample → get k centroids.
  • (4) Extend to 1B via 'assign nearest centroid' in a single map-reduce (Spark, Dask, Ray).
  • (5) Or: BIRCH streaming to build micro-clusters, then hierarchical on summaries.
  • (6) HDBSCAN via subsample-then-extend heuristic.
  • (7) Store cluster IDs alongside data.
  • (8) Iterate on 1M until quality is right; only re-run map-reduce on 1B when necessary.
#interview#applications#clusteringPermalink & quiz →

Interview: you cluster and 3 clusters look 'right', but 2 mix categories — what next?

hard
  • (1) Feature analysis: what's shared in each mixed cluster (maybe a real behavior spans your labels).
  • (2) Try higher k → often 'mixed' clusters split cleanly.
  • (3) Try alternate algorithm (HDBSCAN, GMM) — mixed clusters may be low-density boundary.
  • (4) Feature engineering: add features that separate the mixed classes.
  • (5) Semi-supervised: use partial labels to nudge clustering via constrained k-means / seeded k-means.
  • (6) Accept that categories don't fully align with unsupervised structure and communicate to stakeholders.
#interview#applications#clusteringPermalink & quiz →

Interview: 'no labels — how do you know your clustering is any good?'

hard
  • (1) Internal indices: silhouette + Calinski-Harabasz + Davies-Bouldin across k.
  • (2) Stability: bootstrap Jaccard, ARI between init runs.
  • (3) Downstream utility: does adding cluster label improve any adjacent supervised task?
  • (4) Domain sanity: do clusters agree with known heuristics?
  • (5) Visual: 2D projection colored by cluster — coherent groups?
  • (6) Business sanity: are cluster profiles actionable + interpretable?
  • Reality: unsupervised eval is always partial — combine 3-4 signals + stakeholder buy-in.
#interview#clustering#evaluationPermalink & quiz →

Interview: 'when should you NOT cluster?'

medium
  • (1) Data is truly unimodal / uniform — no structure to find.
  • (2) Very high dimensionality without reduction → distance concentration ruins similarity.
  • (3) The business question is regression / classification, not grouping.
  • (4) You need explanations, not groups (use decision trees or PDP).
  • (5) Sample size too small (< 100).
  • (6) You need causal answers → clustering can produce spurious segments.
  • Always sanity-check whether clustering is the right tool before running one.
#interview#clusteringPermalink & quiz →

Interview: 'how would you tune HDBSCAN's min⁡clustersize\operatorname{min}_{\mathrm{cluster}}\mathrm{size}?'

hard
  • (1) Business constraint: what's the smallest meaningful group in the domain?
  • (2) Sensitivity analysis: sweep min⁡clustersize\operatorname{min}_{\mathrm{cluster}}\mathrm{size} ∈ [5, 10, 20, 50, 100] → measure silhouette + noise-fraction + number-of-clusters.
  • (3) Stability: bootstrap Jaccard between runs.
  • (4) Choose the min⁡clustersize\operatorname{min}_{\mathrm{cluster}}\mathrm{size} that gives stable + interpretable + right number of clusters.
  • (5) Also tune min⁡samples\operatorname{min}_{\mathrm{samples}} (density conservativeness) and clusterselectionmethod\mathrm{cluster}_{\mathrm{selection}}\mathrm{method} ('leaf' vs 'eom').
  • Iterate.
#interview#clusteringPermalink & quiz →

Interview: 'discuss the tradeoffs between k-means, DBSCAN, and HDBSCAN.'

medium
  • (1) k-means: fast, scales to millions (mini-batch), but assumes spherical + equal-size clusters + fixed k + non-robust to outliers.
  • (2) DBSCAN: handles non-convex + detects noise, but needs eps tuning + fails with varying densities + O(n log n) with index.
  • (3) HDBSCAN: no eps tuning + handles varying densities + hierarchy + stability, but slower + memory-heavy for big data + hyperparameter min⁡clustersize\operatorname{min}_{\mathrm{cluster}}\mathrm{size} still matters.
  • Rule: k-means → baseline / large-scale; DBSCAN → known density; HDBSCAN → real-world exploration.
#interview#clusteringPermalink & quiz →

Interview: 'what's the biggest mistake you've seen in a real clustering project?'

medium
  • (Answering as a candidate) '(1) Not scaling features — one feature dominated distances → nonsense clusters.
  • (2) Trusting elbow / silhouette alone without stability check → non-reproducible clusters.
  • (3) Skipping outlier removal for k-means.
  • (4) Presenting clusters without profiling — stakeholders couldn't act on them.
  • (5) Not monitoring drift — clusters silently degraded over 6 months.
  • Lesson: cluster is 30% algorithm, 70% feature engineering + evaluation + stakeholder communication.'
#interview#clusteringPermalink & quiz →

Without labels, how do you convince a stakeholder your clustering is any good?

medium
  • Combine three kinds of evidence, because no single one is sufficient.
  • Internal measures such as silhouette or Davies-Bouldin say whether clusters are compact and separated, which is necessary but not meaningful on its own.
  • Stability says whether the structure is real: re-run on bootstrap samples or different seeds and measure how often pairs of points stay together, since a partition that changes every run is describing noise.
  • Then external usefulness, which is what actually persuades anyone: profile each cluster on variables you did not cluster on, and show that the groups differ in ways the business recognizes, or that using the cluster as a feature improves a downstream model.
  • Present the clusters as hypotheses to be validated, never as ground truth.
#clustering#evaluationPermalink & quiz →

You run k-means on customer data with age, income and number of purchases. What breaks?

easy
  • The scales.
  • Income in dollars ranges over tens of thousands while age spans a few decades, so squared Euclidean distance is essentially income distance and the other two variables contribute nothing.
  • Standardize or use robust scaling first, and understand that scaling is a modelling decision, because it declares each feature equally important.
  • Skew is the second problem: income is heavy-tailed, so a handful of high earners dominate the centroids, and a log transform usually helps.
  • If some variables are categorical, k-means is the wrong algorithm entirely, since a mean of a category is meaningless; use k-prototypes or a Gower distance with a method that accepts arbitrary distances.
#clustering#similarityPermalink & quiz →

The elbow plot has no elbow. How do you pick k?

medium
  • Accept that a smooth curve is telling you the data has no crisp cluster count, which is common and not a failure of method.
  • Look at the silhouette across a range of k and prefer a value that is locally best rather than globally optimal.
  • Use the gap statistic, which compares within-cluster dispersion against a uniform reference and gives a principled comparison.
  • Fit a Gaussian mixture and compare the Bayesian information criterion, which penalizes complexity explicitly.
  • Then apply the constraint that usually decides it in practice: how many groups the organization can actually act on differently.
  • Six segments a marketing team can staff beats a statistically optimal thirty-one nobody can use.
#clustering#evaluationPermalink & quiz →

DBSCAN labels almost everything as noise. What do you change?

medium
  • The neighbourhood radius is too small for the density of your data, or the minimum points is too large.
  • Set the minimum points from domain reasoning, often around twice the dimensionality, then choose the radius from the k-distance plot: sort every point's distance to its k-th nearest neighbour and pick the value at the knee, which is where density drops off.
  • Check scaling first, since the radius is a single global distance and meaningless if features have different units.
  • If clusters genuinely have very different densities, no single radius works and that is the algorithm's real limitation, so switch to HDBSCAN, which varies the density threshold and returns a hierarchy instead of forcing one global choice.
#clustering#densityPermalink & quiz →

Is it a good idea to run k-means on raw text embeddings?

hard
  • It works, with caveats worth stating.
  • Embeddings are high-dimensional and roughly isotropic, so Euclidean distances concentrate and everything looks similarly far apart, which weakens the contrast k-means depends on.
  • Use cosine similarity instead, which for normalized vectors is equivalent to Euclidean distance on the sphere, so normalizing your vectors and then running k-means is the usual fix.
  • Reducing to a few dozen dimensions with PCA or UMAP before clustering often improves both quality and speed.
  • Also remember that embeddings cluster by whatever the encoder was trained to emphasize, frequently topic and register, so if you need a different notion of similarity no clustering algorithm will recover it.
#clustering#representation-learning#nlpPermalink & quiz →

LDA or embedding-based topic modelling for a corpus of support tickets?

medium
  • Embedding-based clustering is usually the better modern default.
  • LDA treats a document as a bag of words with no notion of synonymy, which hurts badly on short texts like tickets, where there are too few words per document to estimate a topic mixture reliably.
  • Embedding the tickets, reducing dimensionality and clustering, then labelling each cluster by its most distinctive terms, handles paraphrase and short text far better.
  • LDA still has advantages: it gives each document a genuine mixture over topics rather than one assignment, it is cheap, and its probabilistic form is easier to defend statistically.
  • Whichever you pick, topic quality is judged by whether humans can name the topics, so run a coherence measure and then read samples.
#nlp#text#clusteringPermalink & quiz →

Your customer segments change completely when you re-run the pipeline monthly. Is that acceptable?

hard
  • No, because a segmentation the business acts on has to be stable enough to plan around, and instability means you are describing sampling noise rather than structure.
  • First check whether the instability is algorithmic: k-means with random initialization on data with no strong separation gives different partitions every run, which a fixed seed hides without fixing.
  • Measure it properly with the adjusted Rand index between consecutive runs, and if agreement is low the structure is weak.
  • Options are to reduce k until the solution is stable, initialize from the previous month's centroids so segments evolve rather than being reinvented, or move to a model with an explicit assignment probability so you can see which customers are genuinely borderline.
  • Report the stability number alongside the segments.
#clustering#interviewPermalink & quiz →

Practise Unsupervised Learning

All 9 sectionsFlashcardsQuiz