Distributed tracing for ML inference — why?
mediumAnswer
- Single request spans: feature store fetch → preprocess → model call → postprocess → API response.
- Tracing (Jaeger, OpenTelemetry) shows per-span latency + errors + attributes.
- Debugging: p99 latency dominated by feature fetch (not model) reveals where to optimize.
- Correlated failure: one downstream service down → trace shows which span failed.
- Modern: LLM traces per prompt token / retrieval hop (LangSmith, Langfuse, Weights & Biases Traces).
Check yourself — multiple choice
- Random
- Single request spans: feature fetch → preprocess → model → postprocess → API; per-span latency + errors reveals bottleneck (e.g., feature fetch dominates p99); LLM traces LangSmith/Langfuse
- Not needed
- Just logs
Traces: per-span latency + errors; find bottlenecks + LLM per-prompt.
#observability#infrastructure
Practise MLOps & Data Quality
215 interview questions in this topic.
Related questions
- Why is high-cardinality label bad for Prometheus?
- Online vs offline feature store — architecture.
- How do you compute real-time features (streaming aggregates)?
- Orchestration tools for ML pipelines — Airflow vs Prefect vs Dagster.
- Why partition ML training tables by date?
- What is an 'asset' in Dagster and why is it useful for ML?