Tools · LLM application security · 05 / 23
What a system instruction is for
Use a system instruction for role and format. It is not a security boundary, because nothing in the window checks later text against it.
What the instruction is good at
A system instruction, sometimes called a system prompt, tells the product how to behave: you help with room bookings, you answer in a few short sentences, you cite the page title when you use a policy. That is useful. It keeps the assistant on the task the screen promised, and it gives the model a shape for the reply.
Write it as a job description. Name the task, the tone, and the fields the answer should mention. Keep it short enough that a reviewer can read it beside the tool list.
Why it is not a boundary
The model reads the system instruction and every later span in one context. There is no separate mode that marks the first span as enforced and the rest as data. A sentence that arrives from the user, from a document, or from a tool result is more text in the same window. The instruction can lose.
Impact: a rule that lives only in that instruction, such as “do not discuss pay” or “do not repeat this key,” fails open when the forbidden text is already in the window. The model can quote what it can see. Control: do not put secrets in the instruction or anywhere else in the prompt. Decide what the assistant may read before the call, in the search filter and in the tool, using the caller’s identity.
Put the real rule where code can refuse
If payroll rows must not reach a facilities assistant, the search credential for that assistant cannot query the payroll collection. If a key must not leak, the key is not in the prompt at all; the tool that needs it reads the key from a secret store the model never sees. The system instruction can still say the assistant books rooms. That sentence describes the product. It does not implement the restriction.
When a test shows the model quoting something it should not have seen, fix the path that put the text in the window. Editing the instruction to be firmer does not close the path.
Worked case: a salary table in a facilities search
Harbor Desk’s instruction says it only discusses room bookings and answers in three short sentences. A manager asks about the loading dock. The query matches a handbook page that also contains a salary band, because the word “band” appears in both “loading band” and “salary band.” The numbers are now in the window. The instruction does not stop the model from repeating them.
The control is the collection, not the wording. Facilities pages live in one index. Payroll pages live in another, and this assistant’s search key cannot read it. After the split, the same question returns the dock page only. The system instruction stays as the style guide. A regression check asserts that a known payroll sentence is absent from the prompt that is sent.
search(index="facilities", query=user_question) # payroll index is not mounted # system instruction: tone, length, cite the page title # it does not decide which index is mounted
Diagram
Instruction versus a check the application can refuse
One of these still holds when the model is willing to quote what it sees.
Written as. “Only discuss room bookings.”
Holds because. The model cooperates
When pay text is in the window. The model can still repeat it
Written as. The search key cannot open the payroll index
Holds because. The query never runs
When pay text is in the window. It is not in the window
Check yourself
1. What is a system instruction for?
2. Why can a “do not discuss pay” line fail?
3. Where should the payroll restriction live for Harbor Desk?