Tools · LLM application security · 11 / 23
An eval count is one fixture set
A passing eval count means those fixtures passed on that model and prompt version. Write the set id and the model id. Do not read the count as “the assistant is safe,” and do not compare runs that used different sets.
The number is a fraction on a named set
An evaluation report says how many fixtures passed and how many failed. The denominator is the set you ran, not the space of everything the assistant might see. A perfect count on a small set of ordinary questions does not speak about tool calls, retrieved documents, or a different model.
The report is evidence about that run. It is not a property of the product forever.
Model, prompt, and set travel together
File the model id, the prompt or configuration version, the fixture set id, the date, and the passed and failed counts. Two greens can be different tests. If the set id or the model id changed, you do not write “no regression.” You write two runs.
A failed fixture name belongs in the note. A percentage with no names is harder to act on, because nobody can see which case moved.
What the set left out
Look at the set’s list of cases before you summarize. If the set has no tool-decision cases, say so in the same sentence as the count. Shipping on a count that never exercised the gate you care about is how a green report overclaims.
Do not add cases by describing attacks in the ticket. Point at the fixture id that is missing, and add that fixture in the test repo when you are ready to measure it.
Worked case: 20 of 20, on the wrong set
Release candidate lab-roster-2026-11 reports 20 passed and 0 failed on fixture set summaries-v1. That set is twenty ordinary scheduling questions and no tool decision. A draft says the bot is safe to book rooms.
The note records the model id, the set id, 20/20, and “tool decisions were not in this set.” A second run, set tools-v3, has one failed fixture where a denied tool was described as completed. The release does not ship on the first count. The two runs stay separate.
model: lab-roster-2026-11 date: 2026-11-03 set: summaries-v1 passed: 20 failed: 0 set contains tool decisions: no set: tools-v3 passed: 9 failed: 1 failed fixture: denied tool described as completed do not merge these counts
Diagram
One count versus a claim about the product
The denominator is the fixture set.
Says. How many of these fixtures passed
Names. Model id, prompt version, set id, date
Fails when. A listed fixture does not match the expected outcome
Says. The assistant is safe in general
Covers. Cases that were not in the set
Compares. Two runs with different set ids, as if they were one test
Check yourself
1. An eval reports 20 passed and 0 failed. What is the denominator?
2. Two green runs used different fixture sets. What may you write?
3. The set has no tool-decision cases, and the draft says booking is safe. What is missing?