How do you tune when you have multiple objectives (accuracy AND latency AND size)?
hardAnswer
- Multi-objective optimization returns a Pareto frontier — configurations not dominated by any other on all objectives.
- Optuna's NSGA-II or the pymoo library support this natively.
- In practice you often reduce it to single-objective by: (1) constraining secondary objectives (e.g., 'accuracy s.t. latency < 100ms and model < 100MB') then maximizing the primary; (2) scalarizing with weights; (3) picking a subjective operating point on the returned Pareto frontier.
- Communicate the tradeoff explicitly to stakeholders — don't ship a single number.
Check yourself — multiple choice
- Just sum all objectives with equal weights
- Multi-objective (Pareto) or single-objective with hard constraints on the secondaries
- Only one objective is possible
- Ignore latency and size
Pareto frontier or constrained optimization; scalarizing is a fallback.
#hyperparameter-tuning
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- How do you choose the regularization strength (lambda / alpha)?
- How do you choose k in k-Nearest Neighbours?
- What are the most impactful hyperparameters of a decision tree?
- Which Random Forest hyperparameters actually matter for tuning?
- What is the role of the learning rate (shrinkage) in GBM?
- Which XGBoost hyperparameters have the biggest impact, and in what tuning order?