Human-in-the-loop: designing AI agents your team can trust

A chatbot answers questions. An AI agent goes further: it can look things up, make decisions and take actions, such as updating a CRM record, creating a ticket or sending an email. That is where the real value is, and also where the real risk is.

Trust is what decides whether a team actually uses an agent. These are the five rules we design every agent around.

Give the agent a narrow, written job

An agent should have a job description as clear as a new employee would get: what it is responsible for, what it must never do, and when it should ask for help. Narrow agents are easier to test and far more reliable than "do everything" assistants.

Put humans at the right checkpoints

Decide in advance which actions need approval. A useful rule is to look at how reversible and how costly an action is:

• Low risk and reversible (tagging a ticket, drafting a reply): the agent acts on its own.

• Medium risk (sending an email to a customer): the agent prepares, a person approves.

• High risk or irreversible (refunds, deleting data): the agent recommends, a person decides.

Grant the minimum access needed

Agents should use their own accounts with the smallest set of permissions that lets them do their job. If an agent only needs to read orders, it should not be able to edit them. This limits the damage of any mistake.

Log every step

Every input, decision and action should be recorded in a way a non-technical manager can read. When something looks wrong, you should be able to answer "what did the agent see, and why did it do that?" in minutes, not days.

Roll out gradually and measure

Launch in "shadow mode" first, where the agent suggests actions but people still do the work. Compare its suggestions with what people actually did. Only switch on automatic actions for the cases where it is consistently right.

The goal is not to remove people. It is to let people spend their time on the work that genuinely needs them.

Designed this way, agents become a dependable part of the team instead of a black box everyone is nervous about.