18 interview questions on prompting techniques, each answered in full. Free to read, no account needed.
What are the pillars of good prompt engineering?
easy- Be explicit about role, task, format, and constraints.
- Provide examples (few-shot) when the format is nontrivial.
- Ask for step-by-step reasoning when useful (chain-of-thought).
- Use structured output (JSON schema, tool schemas) when downstream code will parse.
- Iterate on real inputs and add negative constraints ('do not X') for common failure modes.
Few-shot vs zero-shot — when do you use each?
easy- Zero-shot: task description only.
- Best for well-known task formats or when tokens matter (long few-shot examples eat context).
- Few-shot: 1-5 example pairs.
- Best when: (1) output format is non-obvious and needs demonstration; (2) task is unfamiliar; (3) you want deterministic style.
- Modern instruction-tuned LLMs are strong zero-shot for common tasks, but few-shot still gives 2-10% gains on niche tasks.
- Rule: start zero-shot, add examples only if quality misses.
What is chain-of-thought (CoT) prompting?
medium- Instruct the model to 'think step by step' or provide few-shot examples that show intermediate reasoning before the answer.
- On multi-step tasks (math, logic, complex QA), CoT can improve accuracy 10-40% because it exploits the LLM's ability to condition on its own intermediate tokens.
- Introduced by Wei et al. (2022).
- Zero-shot CoT ('Let's think step by step') works well; few-shot CoT with worked examples works better on hardest problems.
How does self-consistency improve CoT?
medium- Sample K CoT traces at higher temperature (T=0.7-1.0), extract the final answer from each, and majority-vote (or weighted vote).
- Robust to sporadic reasoning errors: even if 3/10 traces are wrong, the correct answer wins the majority.
- Cost: K× more tokens generated.
- Gains: another 5-15% on math benchmarks like GSM8K on top of CoT (Wang et al., 2022).
- Standard in evaluation harnesses for reasoning benchmarks.
What is Tree of Thoughts (ToT) and when does it help?
hard- Instead of a linear CoT, the model explores a tree of intermediate steps: at each node, generate K candidate next steps, evaluate them (self-scoring or a value function), and expand the most promising ones (BFS/DFS with pruning).
- Helps on tasks with backtracking / long horizons like Game-of-24, creative writing plotting, planning.
- Cost: 10-100× a single CoT.
- Underlying idea powers modern agent frameworks that combine search + LLM.
Does role-play prompting ('You are a senior lawyer...') actually help?
easy- Modestly, on some tasks.
- Instruction-tuned LLMs adjust style, tone, and level of detail based on the assumed role.
- Bigger gains: (1) 'You are a strict grader — output PASS or FAIL' improves format adherence.
- (2) 'You are an expert in X' can nudge more domain vocabulary.
- Diminishing returns for frontier models — they already infer expected register from context.
- Effective for style / tone; smaller effect for capability.
How is the system prompt different from the user prompt?
easy- Chat-tuned LLMs use a chat template with roles: 'system' (developer instructions, persona, guardrails, style), 'user' (query), 'assistant' (response).
- RLHF training teaches the model to weight system instructions higher than user instructions in case of conflict — the primary defense against user jailbreaks and prompt injection.
- Best practice: put durable, safety-critical, and format instructions in the system prompt; put per-request context in the user prompt.
How do you force structured output (JSON) from an LLM?
medium- Options: (1) plain prompt asking for JSON — unreliable, parses fail.
- (2) JSON mode: model constrains output to valid JSON syntax (OpenAI, Anthropic, most APIs).
- (3) Constrained decoding: mask logits at each step to only allow tokens consistent with a schema (Outlines, XGrammar, LM Format Enforcer) — guarantees schema compliance.
- (4) Function calling / tool schemas: model outputs a JSON matching a declared function signature.
- (3) and (4) are the production standard.
How does function calling / tool use work in modern LLMs?
medium- Developer declares a set of tool schemas (name, description, JSON-schema parameters).
- At inference, the model may decide to emit a special toolcall token followed by a JSON matching one of the schemas instead of a text reply.
- The runtime executes the tool, appends the result as a toolresponse message, and re-invokes the model.
- Loop until the model emits a normal message.
- Foundation for agents, calculators, retrieval, code execution, and every non-trivial LLM app.
Does position in the context matter for what the LLM 'sees'?
medium- Yes — Liu et al. (2023) 'Lost in the Middle' showed LLMs use context near the start and end far more effectively than in the middle.
- On a QA task with 20 retrieved docs, accuracy drops 20+% when the answer is in doc 10 vs doc 1 or 20.
- Practical implications: (1) put the most important context near the beginning or end; (2) shorter, more relevant context beats stuffing everything in; (3) re-rank retrieved docs and put the top few near the query.
Do 'negative prompts' ('do not X') work in LLMs?
easy- Sometimes, sometimes counterproductive.
- Instruction-tuned LLMs generally follow 'do not' directives, but there's a well-documented 'ironic rebound' — mentioning the forbidden thing primes it.
- Better patterns: (1) positive rephrase ('respond only in JSON') vs negative ('do not respond in prose'); (2) explicit output schema; (3) format constraints via examples.
- Use negative prompts sparingly and pair with positive examples.
What is prompt chaining and why prefer it over one big prompt?
medium- Split a complex task into a chain of smaller LLM calls, where each call handles one sub-task and passes structured output to the next.
- Advantages: (1) easier to debug — inspect each intermediate output; (2) better latency for early user feedback; (3) can inject different tools / retrieval per step; (4) enables branching / retries per step; (5) each sub-prompt is simpler → less error compounding.
- Cost: more API calls, more moving parts.
- Building block for agents.
How do you use one prompt to extract multiple fields at once vs many separate prompts?
medium- Multi-field: single prompt asks for a JSON with all fields at once → 1 API call, cheap, but errors in one field can bleed into others; long output = more room for hallucination.
- Per-field: separate prompt per field → K API calls, higher cost + latency, but each prompt is simpler and mistakes are isolated.
- Best practice: multi-field for correlated fields (structured entity extraction), per-field for critical / independent fields (safety flags).
- Combine with constrained decoding for reliability.
What is Chain-of-Density (CoD) prompting?
hard- Iterative summarization technique (Adams 2023): the model produces a first sparse summary, then in successive rounds is asked to add more 'entities' from the source while keeping the total length constant.
- Each round adds ~2 missing entities and rewrites for cohesion.
- Result: progressively denser, information-rich summaries at fixed length.
- Beats plain 'summarize this' on entity coverage and coherence.
- Underlying pattern (iterative rewriting) applies broadly to constrained-length generation.
How do you get reliable table output from an LLM?
medium- Options ranked by reliability: (1) structured output / function calling with an array-of-objects schema — guaranteed.
- (2) Markdown table with explicit format instructions + few-shot example — mostly reliable but occasional format drift.
- (3) CSV with explicit delimiter — brittle around commas/quotes.
- Best practice: array-of-objects JSON via structured output, render as a table in the UI.
- Never rely on prose 'table' output for downstream code.
When does chain-of-thought HURT performance?
hard- (1) Well-known factual retrieval — 'What is the capital of France?' — CoT adds noise and error paths without any benefit.
- (2) Simple pattern completion / one-step arithmetic.
- (3) Tasks where the model has been RLHF-tuned to answer directly and CoT triggers style drift.
- (4) Recent work (Sprague 2024) showed CoT gains are concentrated in math / logic / symbolic tasks — on many NLP tasks, CoT is neutral or slightly hurts.
- Rule: use CoT for multi-step / verifiable tasks; not for retrieval / classification.
How should you write tool descriptions for reliable agent tool selection?
medium- (1) Descriptive tool name: searchcustomerorders not
f1. - (2) Rich description: what the tool does, when to use it, what to NOT use it for.
- (3) Full argument schema with descriptions per field.
- (4) Example calls in the description when the signature is subtle.
- (5) Constraints: 'this tool is idempotent', 'costs $0.05 per call'.
- (6) Fewer tools > many tools: 5-10 well-scoped is better than 30 overlapping ones.
- Prompt-engineering the tool description is often more impactful than fine-tuning.
Do agents plan explicitly or should we let them just react?
hard- Depends on task complexity.
- Short tasks (2-4 steps): pure ReAct works well; explicit planning adds overhead.
- Long / complex tasks (10+ steps): explicit planning helps — the LLM writes a plan first, then executes each step, replanning as needed.
- Frameworks: Plan-and-Execute (LangGraph), Reflexion, LATS (LLM tree search).
- Trade-off: plans can become rigid or hallucinated; hybrid = plan then re-plan every K steps.
- Reasoning models (o1, R1) do implicit planning inside their thinking tokens.