Interview: 'you have to embed 100M documents monthly — cost strategy?'
hardAnswer
- (1) Only re-embed changed / new documents (change-data-capture).
- (2) Choose model by cost/quality tradeoff: OpenAI-3-small ($0.02/1M tokens) vs OSS bge-small hosted on your infra (initial GPU spend, then free).
- (3) Batch requests → economies of scale in API cost.
- (4) Truncate long docs after N tokens or chunk + aggregate.
- (5) Cache embeddings by content hash.
- (6) Consider Matryoshka embeddings (OpenAI-3 supports variable dim) → smaller dim for retrieval, full dim for reranking.
Check yourself — multiple choice
- Just pay
- CDC (only new/changed) + model tradeoff (OpenAI-3-small vs OSS on your GPU) + batching + truncation/chunking + content-hash caching + Matryoshka variable-dim
- Random
- Not real
100M/month embed cost: CDC + model choice + batch + cache + Matryoshka.
#interview#applications
Practise Unsupervised Learning
214 interview questions in this topic.
Related questions
- Interview: 'you're asked to segment 10M customers — how?'
- Interview: 'design an unsupervised fraud detection system'.
- Interview: cluster 5M e-commerce customers for a marketing campaign.
- Interview: production fraud detector — daily volume 10M, current FN too high.
- Interview: design a recommender for a new streaming service.
- Interview: your model's accuracy dropped 15% overnight — how do you diagnose?