The confidence threshold that decides AI answer or human handoff

The handoff moment: where an AI agent's confidence threshold decides whether a person takes over.

An AI support agent should hand off when its confidence in a grounded answer falls below a floor you set by risk, not when a vendor default says so.

For general questions, I'd start near 0.7. Most teams treat handoff as a switch. It works better as a dial. A handoff confidence threshold is the score below which the agent stops answering and routes the conversation to a person. Set it low and wrong answers leak through; set it high and people field questions the agent could have closed.

According to Userflow, whose escalation guidance was published on September 24, 2026, the right starting point depends on what an answer can break: low-risk answers, medium-risk changes and high-risk actions each get a different rule, and the riskiest get no confidence cutoff at all. The same guidance treats a model's stated confidence as weak evidence on its own. That is why I treat the dial as retrieval confidence, checked against other signals, rather than the model's self-report.

The stakes are also shifting from commercial to legal. California's AB 1609, the Right to Human Customer Service Act, turns an explicit request for a person into a compliance question for large businesses rather than a tuning choice. The published evidence does not yet show how accuracy changes by confidence bucket for support agents, so treat the range below as a starting point to test against your own transcripts.

What confidence threshold should you start with?

Start general support answers at a 0.7 confidence floor after a grounding check, raise it to 80-90% or higher for regulated topics, and give high-risk actions no dial at all.

An analysis of 12 sources shows no agreement on a single universal number, but clear agreement that the floor should rise with risk. According to Userflow, low-risk answers should escalate below 0.4 plus one corroborating signal (based on the Rasa default), medium-risk changes should ask the user to confirm below 0.7 (based on a Rasa NLU example), and high-risk actions get "None; always gated or human" (based on Salesforce per-action guidance). Practitioner guidance from the support outsourcing side lands in the same place: a 60-70% floor for general support and 80-90% or higher for regulated topics.

I call this the risk-tiered dial: one threshold per risk tier, each with its own floor, instead of one global slider. It is also the first setting I'd check when comparing help desk software with live chat. Zazachat's reviews are written and edited by our editorial team, led by Daniel Calloway, and in my view a handoff dial the buyer cannot inspect is a gap any review should name. A common misconception is that handoff is a yes/no switch. The reality is that it works like a ladder, and each rung carries a different cost of being wrong.

Why start at 0.7 rather than the bottom of the band? A wrong low-risk answer, in Userflow's words, "costs a follow-up question." A wrong general answer costs more, because the customer may act on it. In my view, starting at the top of the general band and loosening it only after transcript review is the safer direction to tune from. Loosening a strict floor is a reversible experiment. Winning back a customer who acted on a confident wrong answer is not.

Why does the grounding check come before the number?

The sequence matters as much as the setting. The confidence floor is a second gate, applied only after the answer has been grounded against the knowledge base, and an answer with no supporting passage should escalate whatever its score. Userflow defines the low-confidence trigger the same way: "No matching article retrieved, plus one corroborating signal."

1. Hard rules first. Explicit human requests and high-risk actions exit before any score is read.
2. Grounding second. No retrieved passage means no autonomous answer.
3. Confidence floor third. Only grounded answers are compared against the tier's threshold.

The first gate is no longer only a design choice. According to Quarles & Brady, California's AB 1609 requires businesses with more than $500 million in gross annual revenue that make a customer service chatbot available to people in California to let customers request a live human agent. For those businesses, a request for a person cannot be scored away.

The broader principle is to escalate when the expected cost of continuing exceeds the cost of involving a person, which is exactly what a tiered floor encodes. In practice, the dial is a ladder, not a switch. The takeaway: the more damage a wrong answer can do, the higher the floor sits.

What will matter most for AI handoff in the next 12-24 months?

Over the next 12-24 months, handoff will move from one global cutoff to risk-tiered thresholds, with explicit requests for a person and high-risk actions bypassing the score entirely.

Three signals carry most of the weight in my view. Each is visible today, even if it has not yet reached most vendor settings pages.

One nuance keeps the second prediction from going too far. Self-assessment is useful when it is situational. In one clinical example, an AI reports that its confidence in a diagnosis is lower than usual because of an atypical presentation, and it strongly recommends human review. That is the behavior I'd want from a support agent: a flag tied to why the case is unusual, not a bare number.

Two developments would change this forecast. If models' stated confidence came to track their real accuracy, a single cutoff could work again. And if platforms began publishing accuracy by confidence bucket, the tuning debate would move from judgment to evidence.

What most buyers miss: the threshold number is not where most handoffs fail. Benchmarking shows first-contact resolution runs about 19% lower for customers who are transferred, and 74% of customers are frustrated when they have to repeat information they've already given. A perfectly tuned dial still loses if the person on the other end starts from zero.

Why can't you take the confidence score at face value?

Because models overstate their own certainty. A threshold read at face value passes more wrong answers than the number implies, so the score needs calibrating before the dial means anything.

Every range in the previous section assumes the score is honest. In practice, it often is not. According to Userflow, which cites Xiong et al. (2023), GPT-4's stated confidence across question-answering benchmarks produced an average AUROC of 62.7 percent for vanilla verbalized confidence. AUROC measures how reliably a score ranks correct answers above incorrect ones, so a middling result means the score is a loose sorter rather than a precise gate. In other words, the model's self-reported number tells you something, but not enough to hand it the handoff decision on its own.

The miscalibration also runs in one direction. According to EverHelp, an ICLR 2024 study by Xiong and colleagues found that LLMs tend to be overconfident when they state their own confidence, with accuracy in each confidence band landing "well below" the number the model reported. EverHelp's illustration is blunt: a bot claiming 90% certainty "can be closer to 75% right." What this means for the dial is specific. If you set your floor against raw self-reported scores, part of what clears the bar is less accurate than the setting promises.

This is why I treat handoff as a retrieval-confidence dial, not a self-report dial. The score worth gating on reflects whether the agent found and grounded a source for its answer, rather than the model's own sense of certainty. Low-confidence retrievals are where I expect wrong answers to concentrate, and that is the zone a higher floor removes first.

What does a mis-set threshold cost in each direction?

Both directions carry a real cost, and they land on different people.

The loose side is the one I would worry about most. In a survey of 1,000 US and Canadian chatbot users, 62% named the chatbot misunderstanding them as the root cause of escalation, ahead of underperformance at 22% and emotional triggers at 16%. Misunderstanding is exactly what a low floor lets through. The damage compounds quickly: 80% of users lose confidence after one or two incorrect responses. The takeaway: a loose threshold does not avoid escalations. It delays them until trust is already spent.

The opposite failure is quieter but no less damaging. In logged WhatsApp conversations, a user typed exactly "talk to a human" and "I need to speak to a person." Each time, the bot returned "the same rigid menu of options" and never transferred, and the user's frustration escalated until they insulted the bot or left the channel. Published design documents and patents describe "advanced concepts for detecting user frustration and automatically escalating to a human." The actual logs show different behavior. In practice, a handoff rule is only as good as its behavior in live transcripts, which is where I would test any threshold before trusting it.

So the recommended range holds, but only when the number feeding it is honest and the exit behind it works. Getting both right is an implementation problem, and it is the one the next section takes on.

How do you implement and tune the handoff dial?

Pair the score with corroborating signals, let hard rules override it, and build a transfer that carries state. Then tune each intent from real transcripts.

A single number should never decide alone. According to Petronella Technology Group, which organizes AI-to-human handoff logic into seven named trigger categories, the first is low-confidence or ambiguous intent: the agent can't confidently classify the request, or multiple intents compete. That second clause is the one most setups miss, because a leading intent can clear the floor while a rival sits right behind it.

Which signals should back up the score?

I'd recommend pairing the score with four corroborating signals. Any one of them should be able to pull a passing answer back to a person.

• Competing intents: a near-tie between two intents means the top score overstates certainty, so treat it as a low-confidence result.
• Missing required parameters: if the answer depends on an order number, plan or account detail the agent has not collected, a confident retrieval is answering the wrong question.
• Contradictory cues: a customer who confirms a fix and then repeats the problem is telling you the answer did not land.
• Fallback spirals: repeated rephrase or fallback turns mean the conversation is looping, and the next turn should be a handoff.

In practice, the score sets the floor and the signals decide the close calls.

Which rules should bypass the score entirely?

Some requests should never reach the dial at all. I treat three as hard routes.

1. An explicit request for a person. According to Quarles & Brady, California's AB 1609, the "Right to Human Customer Service Act," requires covered businesses to disclose chatbot use and to let customers request a live human agent, with a good faith effort to connect them within 15 minutes. Outside its coverage, the logic still holds: a customer who asks for a person has already answered the handoff question.
2. Requests that need legal, financial or policy sign-off. Petronella lists these as their own trigger category. An agent cannot grant an exception it has no authority to grant, however strong its retrieval.
3. High-risk actions. As set out earlier, these stay gated or human whatever the score says.

For everything in between, the cleanest decision rule is cost-based. Hand off when the expected cost of continuing exceeds the cost of involving a person.

What makes a transfer actually carry the conversation?

A handoff that announces itself but moves nothing is worse than no handoff. Practitioners describe bots that tell the customer a human agent is on the way, after which nobody joins. The stated root cause is structural: the escalation logic was built to send a message, not to hand off state. The workflow ends at the "transferring you" reply, with no mode change, no context passed, and no task created for a human agent.

The fix mirrors the failure. A working transfer does three things:

• It switches mode, so the bot stops replying once the handoff fires.
• It passes the transcript, the detected intent and any collected parameters to the agent.
• It creates a routed task in the human queue, so the conversation has an owner.

Finally, tune per intent rather than globally. Start each new intent at the conservative end, review its transcripts, and lower the floor only where answers hold up. Autonomy should be earned one intent at a time. The takeaway: a dial you can set per intent or risk tier is worth more than a smarter default.

What should you change about your handoff threshold first?

Move it off the vendor default, tie it to risk, and raise the floor for general answers, because the weakest retrievals are where wrong answers gather.

The case rests on one uncomfortable fact. A model that calls itself nine-in-ten sure can be right only about three times in four, so a dial read at face value is always looser than it looks. According to Userflow, the practical response is tiered: the more an answer can break, the less the score alone should decide.

The number is the easy part. A threshold nobody has checked against real transcripts is a guess, and a transfer that drops context wastes even a well-chosen one. For large businesses serving California, AB 1609 already takes one decision away from the dial entirely. An explicit request for a person goes to a person.

So here is the next step I'd take. Pull last month's AI-answered conversations, group them by confidence score, and have a reviewer mark each answer right or wrong. The first bucket where the wrong marks start piling up is where your floor belongs.