EasyDeepLearn

What is an 'asset' in Dagster and why is it useful for ML?

hard

Answer

  • Asset = data artifact (dataset, model, feature) with computation defined declaratively.
  • Dagster tracks: which upstream assets it depends on, when it was last computed, whether it's stale.
  • Enables: (1) materialization on demand ('compute only what's needed').
  • (2) natural ML thinking (features + labels + models as assets).
  • (3) automatic lineage graph.
  • (4) partitioned assets for date-based data.
  • Vs Airflow's task view: assets are the deliverables, tasks the how.
Check yourself — multiple choice
  • Random
  • Data artifact with declarative computation + tracked upstream deps + staleness; enables materialization on demand + ML-natural thinking + auto-lineage + partitions
  • Same as task
  • Not real

Dagster asset: declarative + deps + staleness + materialize on demand.

#pipeline#infrastructure

Practise MLOps & Data Quality

215 interview questions in this topic.

Related questions