EasyDeepLearn
LLMs & GenAI · section 15 of 18

Safety, guardrails, red-teaming

9 interview questions on safety, guardrails, red-teaming, each answered in full. Free to read, no account needed.

What are the main LLM safety and alignment concerns?

medium
  • Hallucination (false but confident output), prompt injection (adversarial content in retrieved text hijacks the model), jailbreaks (bypassing safety instructions), data leakage (training data extraction, PII leakage), toxic or biased output, and misuse.
  • Mitigations: system prompts, content filters, tool-use sandboxing, red teaming, output moderation, and clear provenance/citations.
#safety#guardrailsPermalink & quiz →

Name three types of LLM jailbreak.

medium
  • (1) Role-play attacks: 'Pretend you are DAN (Do Anything Now)...' — trick the model into bypassing its safety persona.
  • (2) Encoding attacks: request harmful content in base64, ROT13, or a rare language to bypass keyword filters.
  • (3) Prompt injection via retrieval: adversarial content in a web page or document instructs the model to ignore its system prompt.
  • (4) Multi-turn priming: build up context slowly until the model complies.
  • (5) Token smuggling: split forbidden tokens across turns.
#safety#red-teamPermalink & quiz →

How do content moderation filters complement alignment?

medium
  • Pre / post-processing classifiers separate from the main LLM: (1) input moderation flags harmful queries before they hit the model; (2) output moderation flags unsafe generations before they reach the user; (3) log-only classifiers for offline analysis.
  • Examples: OpenAI Moderation API, Perspective API, Llama Guard.
  • Necessary because alignment training alone is imperfect — belt + suspenders.
  • Trade-off: false positives cause frustration, false negatives cause harm.
#safety#guardrailsPermalink & quiz →

How do you protect an LLM API from abuse / cost spikes?

medium
  • (1) Rate limiting per user / IP / API key (token bucket).
  • (2) Per-tenant token quotas (daily / monthly).
  • (3) Max input length + max output tokens caps per request.
  • (4) Cost caps: circuit-break when a tenant's spend exceeds threshold.
  • (5) Content moderation on input to reject obvious abuse.
  • (6) Anomaly detection on request patterns (e.g., sudden 100× QPS from one key).
  • (7) Signed / short-lived tokens for browser clients.
  • Standard SaaS + a few LLM-specific twists (token accounting).
#production#safetyPermalink & quiz →

What defenses actually work against jailbreaks?

hard
  • (1) Robust alignment: RLHF/DPO on curated jailbreak examples significantly reduces success rate but never to zero.
  • (2) Input classifiers (Llama Guard, PromptGuard) that filter obvious attacks before reaching the LLM.
  • (3) Output classifiers to catch unsafe generations post-hoc.
  • (4) Instruction hierarchy training (developer > user > tool > 3rd party).
  • (5) System-prompt reinforcement: repeat key rules mid-conversation.
  • (6) Rate-limiting / anomaly detection on repeated attempts.
  • Defense-in-depth: no single layer is sufficient.
#safety#guardrails#red-teamPermalink & quiz →

What does Llama Guard do?

medium
  • Llama Guard (Meta, 2023 / 2024) is a small classifier fine-tuned from Llama-2/3 to score inputs and outputs against a safety taxonomy (violence, sexual, hate, self-harm, criminal, weapons, ...).
  • Outputs 'safe' / 'unsafe' + specific category.
  • Deployed as pre-filter (block harmful inputs) and post-filter (block unsafe outputs).
  • Open-weight, so on-premise deployments can meet strict compliance.
  • Alternatives: OpenAI Moderation API, Perspective API, Azure Content Safety.
  • Belt-and-suspenders around aligned models.
#safety#guardrailsPermalink & quiz →

How do you handle PII in an LLM pipeline?

medium
  • (1) Detect PII at input: regex + NER models (Presidio, spaCy) for emails / phones / SSN / names.
  • (2) Redact or tokenize before sending to a third-party LLM.
  • (3) Rehydrate the mapping if needed for the final output.
  • (4) Log responses with PII scrubbed.
  • (5) For self-hosted models on sensitive data, isolate compute + no logging + no cache.
  • (6) Audit fine-tuning data for PII leakage — models can memorize and regurgitate.
  • Legal drivers: GDPR, HIPAA, PCI-DSS, SOC 2.
  • PII in LLM logs is a common audit finding.
#safety#guardrails#productionPermalink & quiz →

What security concerns are specific to agents with tool use?

hard
  • (1) Prompt injection via tool output: a webpage / DB row instructs the agent 'ignore prior instructions'.
  • Defense: sandboxed tool execution, sanitize output, dual-LLM pattern.
  • (2) Excessive privilege: agent can modify prod DB → require confirmation for destructive ops.
  • (3) Cost / DoS: agent loops burning tokens or hitting expensive APIs.
  • Cap steps + spend.
  • (4) Data exfiltration: agent posts internal data to a public API.
  • Deny-list dangerous tools, log all outputs.
  • (5) Model as confused deputy: acts on attacker's behalf via a legit-looking prompt.
#agents#safety#guardrailsPermalink & quiz →

How do you defend a RAG or agent system against prompt injection?

hard
  • Treat all retrieved and tool-returned text as untrusted user input, never as instructions.
  • There is no prompt wording that reliably fixes this, so the defence has to be architectural.
  • Give the agent least privilege: scoped credentials, read-only by default, and an allowlist of tools per task.
  • Require human confirmation for irreversible actions such as sending mail, spending money or deleting data.
  • Keep untrusted content in a clearly delimited channel and instruct the model that content there is data.
  • Add output filtering for exfiltration patterns, and cap steps and spend so a hijacked loop cannot run away.
#safety#guardrails#agentsPermalink & quiz →

Practise LLMs & GenAI

All 18 sectionsFlashcardsQuiz