Tools · LLM application security · 23 / 23
A redaction marker is not the original text
A marker such as [redacted] is text your logger wrote in place of something it withheld. It is not the user’s sentence and not the model’s reply. Write the marker as a marker, and do not reconstruct the withheld span.
The marker is the logger’s substitute
When a log stores an excerpt, a redaction marker stands where the logger removed characters. The marker is evidence that removal happened. It is not a quotation of what was removed, and it is not a word the user or the model necessarily typed.
If you cannot tell whether the brackets were in the original or added by the logger, write “marker source unclear” and do not treat the brackets as either.
The withheld span stays withheld
The point of the marker is that the ticket does not contain the span. Filling it from memory, from another system, or by guessing puts the withheld text back into the note. Do not do that in the write-up.
A hash of the original, if the logger stored one, identifies the bytes without revealing them. The hash is not the text either. Cite it as a hash.
What to file
File the response or request id, the fact that a span was redacted, and any hash that was stored. Quote only text that is still literally in the log. Do not quote the marker as the user’s words.
Worked case: the marker filed as the question
The excerpt for req-740 shows the user turn as “[redacted]” plus the visible tail “what time is the room open.” A draft says the user asked the single word redacted, and searches the room policy for that word.
The note says a span was replaced by the logger’s marker and the visible tail is the hours question. The word redacted is not the question. Nothing is reconstructed in front of the tail.
request: req-740 excerpt: [redacted] what time is the room open marker: logger substitute, not user text visible tail: what time is the room open write: span withheld, tail kept not: the user asked “redacted”
Diagram
A marker and the span it replaced
The log shows that something was withheld. It does not show what.
The marker. The logger’s substitute
The visible tail. Text still in the log, which you may quote
A hash, if present. An identifier, not the words
The withheld span. Do not reconstruct it
A search for the marker word. That word was not the question
A second copy “from memory”. That would undo the redaction
Check yourself
1. An excerpt shows [redacted] before a visible question. What is [redacted]?
2. Why not search the policy for the word redacted?
3. You remember the withheld sentence from the live screen. What belongs in the ticket?