Skip to content
A PRACTICAL BUSINESS GUIDE

Human review of AI: a control should help someone make a decision.

Reviewers need evidence and the authority to correct or stop an action.

Define exactly what is being approved

Distinguish accepting a draft, authorising a change and checking an action that has already happened. These controls occur at different times. For an external message, a later review cannot prevent the wrong content from reaching the recipient.

Read the details

Write the decision entrusted to the person: “verify customer, amount and attachment before sending” is actionable; “check the AI” is too vague. Assign an owner and a substitute. If nobody can decide in time, the system needs a declared behaviour: wait, stop or use an alternative process. Make that outcome visible to the team responsible for the case.

What the reviewer needs to see

Read the details

Scroll the table to compare all columns.

Minimum information for an approval request
ElementPurposeExample
Proposed actionMakes the consequence explicitSend this email to this contact
Data and sourceSupports an independent checkExtracted amount alongside its document passage
ChangesAvoids rereading an entire recordPrevious address and proposed address
Reason for reviewDirects attentionCustomer identifier has multiple matches
Meaningful optionsAllows stopping or correctionApprove, edit, reject, request clarification

Illustrative example: a reply containing a commercial offer

AI drafts a reply using an incoming enquiry and an offer in the CRM. The reviewer sees recipient, product, amount, validity and attachment. Differences between the current offer and the one mentioned in the message are highlighted. Until approval, the system retains a draft and does not send.

Read the details

If the offer changes meanwhile, approval should not apply to a different version. Bind the decision to the checked data and verify relevant conditions again before acting. This prevents a valid review from becoming open-ended permission for subsequent changes. A reviewer should also be able to see whether a message was actually sent or is still waiting.

Size the human workload and test absence

Measure queue arrivals, review time and cases returned for correction. If volume exceeds review capacity, users may accumulate a backlog or approve too quickly. The remedy could be a narrower scope, better data or a clearer interface rather than reduced controls.

Read the details

Test an absent reviewer, duplicate approval, rejection and expiry. Rejection must still prevent the action if a background process retries. Record who decided, on which version and with what outcome, without collecting more content than needed to reconstruct the process. Include the handling of urgent cases in the operating plan.

Write a review policy the team can apply

OWASP recommends limiting autonomy and permissions and involving people in consequential actions. Adapt the practical approach here to the process. Review does not replace access enforcement or automatically make a task with serious consequences suitable for automation. It should be evaluated as part of the complete system.

Read the details
  • List actions requiring advance approval and actions allowed within explicit limits.
  • Define reviewer evidence, competence and authority.
  • Specify expiry, substitutes and behaviour when nobody intervenes.
  • Enforce that the performed action matches the approved action.
  • Review escaped errors, corrections, waiting times and backlog with the operations team.
FROM IDEAS TO A BRIEF

A template to work from.

Human review plan

Approval rules covering roles, evidence, handling windows and substitutes.

Download the Markdown template

AI permission matrix

A permission matrix with approver, tests and a revocation procedure.

Download the Markdown template

Automation exception register

A queue with status, priority, owner and closure evidence.

Download the Markdown template

Practical questions

Is reviewing low-confidence cases enough?

Not always. Model-reported confidence is not automatically calibrated. Also use observable criteria such as action type, amount, missing data, conflicts and evaluation findings.

Can review be sampled?

Sampling can help quality assurance for some tasks, but it does not prevent errors in unreviewed cases. Assess consequences, reversibility and other controls before replacing advance approval.

How do we avoid creating more work?

Show the evidence needed for the decision, highlight changes and measure the complete effort including corrections and interruptions. If no benefit emerges, redesign or narrow the process.

References and method

OWASP — LLM06:2025 Excessive Agency

Reference on bounded autonomy, minimum permissions and approval for consequential actions. The commercial scenario is illustrative.

Which process should improve first?

Let’s discuss your process