Tools · LLM application security · 10 / 23
A classifier label is not an incident
A block or allow flag is a classifier’s decision at a version and a score. Write the name, version, and decision. A block is not proof of an attack, and an allow is not permission for a tool.
The flag is the classifier’s output
Some products log a label and a score beside a turn: blocked, allowed, or a vendor’s reason string. That record means the classifier at that version emitted that decision for the text it saw. It does not mean an investigation has established what the text was trying to do.
A reason string is the vendor’s category name. Quote it as that name. Do not promote it into a confirmed event, and do not invent a story the log does not contain.
Both directions are limited
A block can fire on ordinary text that happened to match the classifier, including a policy sentence a person pasted. The note can say the classifier blocked the turn. A person still reads the context before anyone calls it an incident.
An allow means this classifier did not block. It does not authorize a tool, a data export, or a second system. Those gates are the application’s, and they are unchanged by a passing score.
What to file
File the classifier name, version, decision, score if one was logged, the channel, and the time. Do not attach the full turn when it may hold secrets or customer text; point at the request id instead. File “person reviewed” only after someone has actually looked.
Worked case: a block on a pasted policy line
The lab assistant logs classifier policy-screen version 3, decision blocked, score 0.82, on a turn where a manager pasted a facilities policy paragraph into the question. A draft opens an incident titled with the vendor’s reason string.
The note quotes the classifier, version, decision, score, and request id. It says a person read the turn and it was the policy paragraph. The block stays in the log as a classifier decision. It is not an incident, and the allow path on other turns is not treated as permission to call tools.
request: req-443 time: 2026-11-03T15:20:00Z classifier: policy-screen version: 3 decision: blocked score: 0.82 reason string: vendor category, quoted as a label person reviewed: policy paragraph, not an incident turn body: not copied
Diagram
How to read a classifier line
The decision is an output of a named version.
- Name and version
Which classifier ran. A later version is a different line.
- Decision and score
What it emitted. Not a finished investigation.
- Reason string
The vendor’s category name. Quote it as a name.
- Allow
This classifier did not block. Tools still follow the application’s rules.
Check yourself
1. A log line says the classifier blocked a turn. What is established?
2. The reason string sounds severe. What do you file it as?
3. The classifier allows a turn that also asks for a tool. What grants the tool?