Where untrusted text enters an assistant

Three channels, three owners

An assistant does not read one clean instruction. It reads a stack of text from different owners. The person at the keyboard owns the user message. A wiki, a ticket, or a file in a search index owns the retrieved document. Another system owns the tool result that comes back on the next turn. If you only watch the chat box, you miss the other two.

The impact is the same shape on every channel: the model may treat that text as something to carry out. The control is different, because the application can see who produced the text and can refuse to let that producer pick a target, a recipient, or a tool.

The user message is a request, not a credential

The user message is the easiest channel to see and the easiest to over-trust. The signed-in person can ask for anything they can type. That does not make the sentence an authorization. The session already knows who they are, which tenant they belong to, and which buttons the product showed them.

Impact: a message that names a coworker, a room, or a mailbox can be wrong, curious, or hostile, and the model has no way to check it. Control: tools take identifiers from the session and from form fields the application rendered. The chat text can explain what the person wants. It does not supply the id the tool will act on.

A retrieved document was written by someone else

Retrieval pulls a passage the current user did not write. A vendor edited the wiki. A customer wrote the ticket. A shared drive holds an old note. The assistant is reading it so it can answer, which means those sentences sit in the same window as the product rules.

Impact: the answer can follow the passage, and a later tool request can be shaped by it, even though the user only asked a question. Control: keep the document id and title attached to the passage, show that source in the interface, and do not let a search hit add a tool or change the allowlist. The passage is evidence for the answer, not a configuration change.

Worked case: a tool result carries the third channel

Harbor Desk books rooms for office managers. It searches the facilities wiki, then may call book_room. The search tool returns a page that includes a long “notes” field a vendor filled in. On the next turn that field is just more text, so the model can request a booking the manager did not ask for.

The review changes the tool contract. search_wiki may return title, url, and a passage capped at 500 characters. It drops notes, comments, and any field the schema does not name. book_room accepts only a room id and a time from the booking form. The model can draft a sentence under the form. It cannot invent the room id. A trial in the test tenant uses a wiki page with an unexpected request in the notes field and checks that the field never appears in the tool call.

Snippet

search_wiki(query) -> { title, url, passage }   # notes and comments dropped
book_room(room_id, start)              # both values come from the form
# the model may explain the booking; it does not choose the room id

Diagram

Three ways text reaches the model

User message
The signed-in person typed it. Treat it as a request.
Control
```
authority: session, not the sentence
```
Retrieved document
Search returned a passage someone else wrote.
Control
```
keep the document id; do not add tools
```
Tool result
Another system answered, and that answer is text again.
Control
```
return only named fields, capped
```
Check
A test page with an unexpected request stays out of the tool call.
Test
```
notes field absent from book_room
```

Check yourself

1. Which channel is a tool result?

2. Why is the user message not enough to authorize a booking?

3. What should search_wiki stop returning?