EasyDeepLearn
Lesson

LLMs & GenAI

214 questions18 sections

Large language models: pretraining, fine-tuning, RAG, hallucinations, and evaluation.

Every questionFlashcardsTake the quiz

Introduction

Large language models are just transformers trained on very large amounts of text with a next-token prediction objective — then aligned with instruction tuning and preference optimization (RLHF or DPO).

What makes LLM interviews distinctive is how many *practical* concerns they cover: retrieval, context windows, prompt engineering, evaluation, safety, cost, latency. This chapter walks through everything a senior engineer is expected to have opinions about, from the shape of RAG pipelines to why quantization matters when you actually have to ship.

The 18 sections

Each section is a short read on one subject, with every answer written out. Work through them in order, or jump to the one you are weakest on.

  1. 01FundamentalsWhat is a large language model, in one paragraph?8 questions2 easy4 medium2 hard
  2. 02Tokenization & embeddingsWhat are text embeddings and how are they used?14 questions1 easy6 medium7 hard
  3. 03Pretraining & scalingWhat data goes into modern LLM pretraining?16 questions4 medium12 hard
  4. 04Architecture & attention variantsWhat is the context window and what tricks extend it?8 questions1 medium7 hard
  5. 05Fine-tuning & adaptationFine-tuning vs RAG — which do you use when?7 questions1 easy3 medium3 hard
  6. 06Alignment: SFT, RLHF, DPODescribe the full RLHF pipeline in three stages.20 questions5 medium15 hard
  7. 07Prompting techniquesWhat are the pillars of good prompt engineering?18 questions5 easy9 medium4 hard
  8. 08Decoding & samplingWhat do temperature and top-p do at generation time?16 questions6 easy4 medium6 hard
  9. 09Retrieval-augmented generationHow does Retrieval-Augmented Generation (RAG) work?29 questions16 medium13 hard
  10. 10Vector databases & hybrid searchHow do you handle content updates and stale embeddings in production RAG?1 questions1 medium
  11. 11Inference: caching, batching, servingWhat is quantization and what tradeoffs come with it?26 questions13 medium13 hard
  12. 12Long contextWhat is 'needle in a haystack' evaluation?1 questions1 medium
  13. 13Evaluation & benchmarksHow do you evaluate an LLM system?22 questions1 easy16 medium5 hard
  14. 14Hallucinations & reliabilityWhy do LLMs hallucinate and how do you reduce hallucinations?5 questions2 medium3 hard
  15. 15Safety, guardrails, red-teamingWhat are the main LLM safety and alignment concerns?9 questions6 medium3 hard
  16. 16Agents & tool useDescribe a basic ReAct / agent loop.10 questions7 medium3 hard
  17. 17Multimodal LLMsHow do multimodal LLMs handle OCR / document understanding?3 questions2 medium1 hard
  18. 18Production concernsHow do you estimate LLM API cost for a workload?1 questions1 medium