Tools · Workbench
LLM application security
A three-part series on prompt injection defense: mapping trust boundaries, enforcing controls around the model, and regression testing with promptfoo.
- 3 lessons
- 35 min of reading
An LLM feature mixes instructions and untrusted content in one stream of text, so the model will sometimes follow text it should only have read. These lessons design the application so that when it does, nothing important happens.
Test your own application and its model providers within their terms of use.
Lessons
3 lessons
- 01Prompt injection defense, part 1: map where untrusted text enters the modelDraw the trust boundaries of an LLM feature, mark every place untrusted text reaches the prompt, and list what the model can do with tools, so defenses go where an injection would cause harm.11 min
- 02Prompt injection defense, part 2: controls that limit what injected text can doApply least privilege to tools, require confirmation for side effects, enforce limits in code rather than in the prompt, and restrict output rendering so an injected instruction cannot move money or leak data.13 min
- 03Prompt injection defense, part 3: regression tests with promptfooTurn the injected test documents from your threat map into a promptfoo suite that checks tool calls and output on every prompt or model change, and read a failing result.11 min