Agent stop rules: escalate high-risk, sample low-risk, approve fast

In the Bristol property-management pilot, a gas-smell report reached the out-of-hours duty manager in 12 seconds. The agent's job is to stop and hand the case to a human. Set the same stop rule before your agent ships: stop on high-risk actions, low confidence, failed approval, edge cases, and unresolved tasks.

The rule is an approval gate. It tells the agent when to pause, what to send, and who decides. The operator gets a decision. The agent gets a repeatable check.

Make the stop rule explicit

A Bristol property-management pilot's AI agent handled 70% of incoming maintenance calls without human involvement. Routine volume is not a reason to remove the human. The stop rule decides which calls are routine and which need a person.

Most agent failures are control failures. The model can answer, route, draft, and retry. The system must decide when the answer is not enough. Put the decision in code, not in a prompt. Run it before the next action, after the model output, and before the side effect.

Escalate on five triggers

Each trigger is a test. If the answer is yes, the agent stops and hands the case to a person with a compact record of what happened.

• High-risk action. Stop when the action can cause harm, legal exposure, or irreversible loss.
• Low confidence. Stop when the confidence signal falls below the threshold you set for that task.
• Failed approval. Stop when a prior approval fails, times out, or returns an unclear answer.
• Edge case. Stop when the request is outside documented scope, policy, or known failure modes.
• Unresolved task. Stop when the agent has retried the same step without progress.

The triggers are the minimum escalation set for production agents: harm, uncertainty, broken consent, out-of-scope work, and stalled progress.

Set the confidence threshold per task. A routine maintenance call can tolerate a lower bar than a gas-smell escalation. If the threshold is too high, the agent stalls. If it is too low, it acts on guesses. Tune it from sampled low-risk reviews, not from a single bad case.

Sample low-risk actions to catch drift. Review enough to compare outcomes, not everything. The sample should be random enough to avoid cherry-picking and structured enough to show which actions are failing. If a low-risk action starts failing, raise its tier.

Failed approvals need a path. A timeout or unclear answer is not approval. The agent pauses, preserves the state, and re-escalates with the same evidence pack plus the failed attempt. It does not retry the action and hope for a different reviewer.

Edge cases need a documented exit. The agent should know what is out of scope. When it hits one, it labels the case, attaches the evidence, and routes it to the owner who can decide.

Unresolved tasks need a retry limit. The agent stops after repeated attempts without progress. The handoff includes what was tried, what failed, and what the next human should check.

The gas-smell call is the test. The Bristol tenant's flat was made safe within two hours after the gas-smell escalation. The stop rule's job is to move the case to the human owner.

Approve by risk tier

A stop rule without tiers creates a queue. Every action asks for a human, and the human becomes the bottleneck. Approve by risk, not by volume.

LiveKit's Human-in-the-Loop pattern assigns 90% of work to the AI and 10% to humans for judgment, empathy, or regulatory sign-off. LiveKit's Human-in-the-Loop architecture requires approval for 100% of high-risk actions and samples 5% to 20% of low-risk actions for review. That split keeps humans on the cases that need them. The sample is the feedback loop that tells you when a low-risk action has become high-risk.

In LiveKit's Human-in-the-Loop architecture, a complete evidence pack reduces approval decisions to 10 to 30 seconds instead of minutes. The pack is the handoff. It should let the reviewer decide without reconstructing the case from logs, replayed conversation, or more context from the agent. Include the request, the proposed action, the risk tier, the confidence signal, the relevant policy, and the last few steps the agent took.

Build the stop rule before the agent touches production. Put the triggers in code, the risk tiers in the workflow, and the evidence pack in the handoff. The reviewer sees the evidence, the agent stops at the boundary, and the case moves.