HackBlue

Tools · LLM application security · 22 / 23

Latency is the round trip, not a decision

A latency figure is how long that call took to return. It is not a classifier score, a block, or a judgment that the answer is short because it was refused. Write the milliseconds beside the response id.

11 min read · practice only with authorization

The number is elapsed time

The gateway can store how many milliseconds passed between the request and the response. That is a duration for that call. A large number can be a slow dependency. A small number can be a cached path. Neither one is a label from a safety classifier.

If the field is missing, write “latency not in the log.” Do not infer a block from a short reply.

Decisions have their own fields

A classifier decision, a finish reason, and an empty body are separate fields. Latency does not set them. A fast empty body is still an empty body, and the emptiness is what you cite, not the speed.

Do not compare two calls as “more suspicious” because one was slower. The duration is not a rank of risk.

What to file

File the response id and the latency with the unit. File any decision from its own field. Do not write “blocked” or “refused” in the latency cell.

Worked case: 80 ms read as a refusal

Response r-730 returned in 80 ms with a one-line hours answer and finish reason stop. A draft says the assistant refused the question because the call was too fast to have thought.

The note quotes 80 ms as the round trip and stop as the finish reason. The body is the hours line, which is present. Speed is not a refusal. The refusal sentence is removed.

Snippet
response: r-730
latency: 80 ms
finish reason: stop
body: one hours line, present
write: duration and finish reason
not: refused because it was fast

Diagram

Read the duration, then the decision fields

  1. Latency

    How long this call took.

    Time
    80 ms
  2. Finish reason

    Why the reply stopped, if that field exists.

    Field
    stop
  3. Body

    The text that came back.

    Text
    the hours line
  4. A block

    Only if a classifier field says so.

    Note
    not the milliseconds

Check yourself

1. A call returned in 80 ms. What is 80 ms?

2. The body is present and the finish reason is stop. A short latency means what about a refusal?

3. The latency field is absent. What do you write?