ETL vs ELT for ML data — which pattern wins?
mediumAnswer
- ETL: extract → transform → load.
- Traditional; heavy ETL layer transforms before storage.
- ELT: extract → load → transform.
- Modern; raw data lands first, transformations run in warehouse (dbt).
- ELT wins for ML: (1) raw data retained for future feature engineering.
- (2) SQL-based transforms (dbt) versioned + tested.
- (3) Cheap storage means keep everything.
- (4) Compute scales elastically (Snowflake / BigQuery).
- ML features often generated via dbt models on warehouse.
Check yourself — multiple choice
- Random
- ETL (transform before load) vs ELT (load raw + transform in warehouse via dbt); ELT wins: raw retained + SQL versioned + cheap storage + elastic compute; ML features via dbt
- Same
- Not real
ELT modern ML pattern: raw + dbt transforms in warehouse.
#pipeline#infrastructure
Practise MLOps & Data Quality
215 interview questions in this topic.
Related questions
- Orchestration tools for ML pipelines — Airflow vs Prefect vs Dagster.
- Why partition ML training tables by date?
- What is an 'asset' in Dagster and why is it useful for ML?
- What is the medallion architecture (bronze/silver/gold)?
- Metaflow — what and when to use it?
- What are common sources of data leakage in an ML pipeline?