EasyDeepLearn

How do you defend a RAG or agent system against prompt injection?

hard

Answer

  • Treat all retrieved and tool-returned text as untrusted user input, never as instructions.
  • There is no prompt wording that reliably fixes this, so the defence has to be architectural.
  • Give the agent least privilege: scoped credentials, read-only by default, and an allowlist of tools per task.
  • Require human confirmation for irreversible actions such as sending mail, spending money or deleting data.
  • Keep untrusted content in a clearly delimited channel and instruct the model that content there is data.
  • Add output filtering for exfiltration patterns, and cap steps and spend so a hijacked loop cannot run away.
Check yourself — multiple choice
  • Add 'ignore malicious instructions' to the prompt
  • Treat retrieved content as untrusted data and defend architecturally: least privilege, tool allowlists, human confirmation for irreversible actions, output filtering, step and spend caps
  • Only use trusted documents
  • Prompt injection is solved

Injection is not fixable by prompt wording; it needs privilege and confirmation boundaries.

#safety#guardrails#agents

Practise LLMs & GenAI

214 interview questions in this topic.

Related questions