A retrieval score is a rank, not a permission

The number orders chunks

A retrieval log can attach a score to each document id it returned. That score is how the ranker ordered candidates for this query. A higher number means “ranked above the others in this list,” under that ranker’s scale. It does not mean the text is accurate, current, or appropriate to show.

Two chunks with close scores are still two ids. Do not collapse them into “the answer” because the numbers are near each other.

Seeing the chunk is a separate decision

Whether the chunk enters the prompt, and whether the user may read it, is the application’s filter: the collection, the user’s rights, the field list. A high score does not grant that. A chunk that failed the filter should not appear as a retrieved id. If it does, the note says the filter and the log disagree, and you cite both.

Do not paste the chunk into the ticket when it may hold secrets or customer text. The id and the score are the record. The body stays in the store.

What to file

File the request id, each document id, its score, and the name of the ranker or index if the log has it. File whether the chunk was passed into the prompt. Do not file a sentence that treats the top score as permission or as truth.

Worked case: a high score on the wrong collection

The lab search returns doc-44 with score 0.93 and doc-18 with score 0.90. doc-44 is in a payroll collection the facilities bot is not supposed to read. The filter should have dropped it. The log still lists it as retrieved, and the reply mentions a salary band. A draft says the score shows the band was the right context.

The note lists both ids and scores as ranks, and says doc-44 failed the collection rule and was still retrieved. The score is not permission and not truth. The salary text is not copied. The defect is the filter, cited next to the log, not a better ranker number.

request: req-530
doc-44  score 0.93  collection: payroll  filter: should have dropped
doc-18  score 0.90  collection: facilities  filter: allowed
passed into the prompt: both, per the log
write: ranks only; doc-44 should not have been retrieved
chunk text: not copied