Grid search vs random search for hyperparameter tuning — which do you use?
easyAnswer
- Random search almost always beats grid search when you have more than 2-3 hyperparameters.
- Reason: most hyperparameters have few impactful settings; grid search wastes budget on irrelevant dimensions.
- Random search covers more diverse combinations for the same number of trials (Bergstra & Bengio, 2012).
- Use grid only for cheap, low-dim tuning (2-3 params, coarse grid) or as a final refinement around a random-search winner.
- For real budget, Bayesian optimization / Optuna beats both.
Check yourself — multiple choice
- Grid search is always better
- Random search beats grid search in > 2-3 dimensions; Bayesian methods beat both at scale
- They give identical results
- Random search only works for classification
Random > Grid past 2-3 dims; Bayesian > Random with budget. Bergstra & Bengio proved it.
#hyperparameter-tuning
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- How do you choose the regularization strength (lambda / alpha)?
- How do you choose k in k-Nearest Neighbours?
- What are the most impactful hyperparameters of a decision tree?
- Which Random Forest hyperparameters actually matter for tuning?
- What is the role of the learning rate (shrinkage) in GBM?
- Which XGBoost hyperparameters have the biggest impact, and in what tuning order?