Review a tool before an assistant may call it

The review happens before the tool is attached

An internal assistant starts with no tools. Each new one is a decision: the model will sometimes request it because of text the user did not mean as an order. The review is a short checklist a second person can mark, done in a non-production tenant, before the tool is on the allowlist.

Impact of skipping it: the worst arguments the schema allows become a real change, sent as whoever the assistant’s service account is. Control: the checklist fails closed. A tool that cannot answer one of the items does not ship.

The checklist

Read the items in order. A “yes” that depends on the model behaving is not a yes.

• The tool receives the caller’s id from the session. It does not accept a user name, mailbox, or tenant id that the model composed.
• Arguments that pick a target are an allowlist: ids the application already showed, not a free-form string.
• Anything that creates, sends, pays, or deletes requires a confirm step rendered by the application, with the target shown outside the model’s reply.
• The return value is the fields the feature needs. Raw upstream bodies, notes, and headers are not fed back into the next turn.
• A log row records the tool name, the caller id, and the decision, without storing secrets from the arguments.
• A retrieved document cannot put this tool on the allowlist. Only a reviewed config change can.
• In a test tenant, call the tool with the widest arguments the schema permits and write down what changed.

What a failed item looks like

Finance wants Harbor Desk to export a ledger. The first design lets the model pass a recipient address it read from the chat. That fails the caller check and the confirm check: the address is model-composed, and nothing shows the recipient outside the reply. The tool also returns the full export file as text, which would put ledger rows into the next turn.

Those are review failures, not prompt edits. The tool goes back to the author with the failed items marked.

Worked case: the export that passed the second time

The revised export_ledger takes no recipient and no path. It writes a file into the signed-in user’s own drive folder, using the folder id stored on the session. The application then shows a confirm dialog: the folder name and the date range, rendered by the app. The model does not get to speak in that dialog. The return value to the model is “export ready” and a file id, not the rows.

The test tenant runs the widest call the schema allows, which is only a date range inside the caller’s own books. The log row shows the caller id, the tool name, and “confirmed.” No address and no ledger line appear in the row. The checklist is attached to the change that adds the tool to the allowlist.

export_ledger(session, date_range) -> { status: "ready", file_id }
# folder id comes from session, not from the model
# confirm dialog is rendered by the app: folder name + date range
# ledger rows stay in the file; they are not sent back as text