How do you compute real-time features (streaming aggregates)?
hardAnswer
- (1) Streaming engine (Flink / Spark Streaming / Materialize) consumes events (Kafka).
- (2) Windowed aggregation: tumbling / sliding / session windows.
- (3) Write result to online store (Redis / DynamoDB).
- (4) Handle late-arriving events with watermarks + allowed lateness.
- (5) Exactly-once semantics via checkpointing.
- Example: '(user)' updated as events flow.
- Latency: sub-second.
- Complexity: state management + backfill from historical events for training.
Check yourself — multiple choice
- Random
- Streaming engine (Flink/Spark/Materialize) + Kafka events + windowed agg + write to online KV + watermarks for late data + exactly-once checkpoints; sub-second latency + backfill for training
- Just batch
- Not real
Real-time features: streaming + windows + online KV + watermarks + exactly-once.
#features#infrastructure
Practise MLOps & Data Quality
215 interview questions in this topic.