What Questions Should an AI Concierge Answer vs Escalate?
Most AI concierge programs fail a quieter test than their dashboards admit: a chat that never reaches a person is scored as success, even when the customer's problem did not end. Containment and resolution are not the same number. Ada's deployments show about 72% containment against about 52% automated resolution — a 20-point gap between a conversation that looks finished and one that is finished (AI Resolution Rate Statistics (2026) — Macha). A telecom chatbot posted 78% containment and, once repeat contacts were counted, closer to 41% true resolution (78% Containment, 41% True Resolution — Ender Turing).
Customers already treat that gap as the product. In a March 2026 Ada and NewtonX study of 2,000 consumers, only 24% said their most recent AI customer service interaction was fully resolved by AI alone; the other 76% needed a human, got a partial fix, or left (AI Resolution Rate Statistics (2026) — Macha). Operator-cited SurveyMonkey figures put the expectation just as plainly: 89% say a brand should always offer a way to reach a person (AI voice support vs human — Ringly). Design the concierge for that majority, not for a 100% automation target.
The workable rule is fail-closed. Answer when the question is repeatable, low-stakes, and grounded in an approved page or connected record — product facts, policies, hours, shipping, order status, and returns that follow a written rule. Clarify once if the intent is unclear. Hand off on an explicit request for a person, and on refunds, billing disputes, legal or security issues, safety, policy exceptions, VIP accounts, and anything the knowledge base does not cover. The rest of this guide is that decision tree, with starting thresholds you can calibrate on your own transcripts.
- Answer only repeatable, low-stakes questions you can ground in an approved page or connected record.
- Containment is not resolution: Ada's gap is about 72% contained versus 52% actually resolved.
- Only 24% of consumers in a March 2026 Ada/NewtonX study said AI fully resolved their latest service chat.
- Hard-escalate refunds, disputes, legal, security, safety, exceptions, and missing knowledge — confidence does not override those topics.
- Clarify once, then hand off with a case packet instead of looping the customer.
Containment Is the Wrong Score
The scoreboard problem is the definition, not only the gap. A confirmation, a closed chat, or a stretch of silence can all be marked resolved.
In that March 2026 Ada and NewtonX study, the customers who were not fully resolved by AI alone blamed comprehension failures, capability gaps, and repeated unhelpful answers.
Intercom's Fin averaged 76% across 12,000+ customers, counting confirmation or 24 hours of silence as resolved. A definition that treats silence as success will overstate what the concierge actually finished. Use that as a caution when you read any resolution rate, not as a number to paste into an allow-list.
Wrong score — Judge a resolution rate by what it counts as finished—silence and a closed chat are not the same as a problem that stayed solved.
Containment vs True Resolution
Questions Worth an Answer, Not a Guess
Refusing full automation only helps if the allowed work is already visible in the call mix. Repeatable lookups are the band a page can carry; emotion, exceptions, and missing knowledge are not.
Cited, in practice, means the reply quotes or points to the page that says it. If the page does not say it, the concierge does not answer. How that contract is enforced belongs on the website AI assistant pillar, not in a training how-to here.
On DTC support lines the split is stable: about 70% of calls are repeatable lookups, and about 30% carry emotion, exceptions, or missing knowledge. Bot CSAT of 78–84% on transactional queries versus 42–58% on troubleshooting and billing disputes draws the same line.
Read the published rates as ceilings, not failures. Gorgias reports the median brand resolves 45% of AI-touched tickets end to end, while 55% end in a handoff. Crisp found 40% of conversations resolved without a human among companies with an AI agent, versus 11% without, and 44% for one-agent teams.
The cost gap—about $0.62 for an AI resolution versus about $7.40 for a human, voice near $1.18—justifies answering that safe band, not stretching into disputes. Catalog-aware buying help is covered in the Shopify AI shopping assistant article.
The line — Read end-to-end resolution in the mid-40s as the ceiling for that safe band, not as a shortfall against full automation.
What Must Escalate No Matter the Confidence Score
Once an intent sits in the answer band, the next cut is stricter. Some questions leave that band even when the model is sure, and they should be hard-routed regardless of confidence: an explicit request for a person, a refund or billing dispute, a legal or security issue, a safety concern, a formal complaint, an accessibility request, a maintenance emergency, a policy exception, a VIP account, and anything the knowledge base does not cover.
Hospitality practice already treats refunds, safety, formal complaints, legal or accessibility requests, and maintenance emergencies as topic rules with no confidence override. Apply the same logic to SaaS account, billing, and security exceptions. Liability and judgment decide the handoff, not a high score.
Keep a visible path to a person. SurveyMonkey figures operators cite put the preference plainly: 79% strongly prefer a human, 84% think humans are more accurate, and 89% say a brand should always offer a way to reach a person. Gartner's survey of 5,728 customers, published August 2024, found only 14% of issues fully resolved in self-service, and only 36% of issues customers called very simple. Simple is not a safe automation label.
Plan with ranges, not targets. Escalation averages 32–45% across industries, with ecommerce and retail lower at 28–38% and B2B professional services at 55–70%. Match the range to the kind of work, not to a SaaS label those figures do not name. The chatbot-versus-concierge article covers the deploy choice; this rule is about what the chosen tool is allowed to finish.
No override — Topic rules and uncovered questions escalate even when confidence is high. Keep a visible human path, and plan an escalation share in the industry range rather than chasing containment.
Clarify Once, Then Answer or Escalate
With hard routes already set, clarification is not a second containment tactic. Reserve it for an in-scope question with one ambiguous referent: which order, which plan, or which published policy. Ask once, then answer or escalate.
A practical starting floor is to route any response scored below 70–75% confidence, tighter on billing, legal, security, or account changes even when those topics mostly hard-route. After two or more clarification rounds, 64% of customers report dissatisfaction, versus 71% satisfaction when the bot resolves on the first message. Paste this, then calibrate on your transcripts: Answer is a grounded reply; Clarify once is one specific in-scope question, then Answer or Escalate; Escalate is the floor, the two-strike rule, or any hard route. The floor is a starting line, not a law.
One question — Clarification is one disambiguation of an in-scope referent, then a grounded answer or a handoff—not another loop to protect containment.
Send the Case Packet and Log the Refusal
Once that single clarification is spent, the handoff is the work. The case packet holds only the transcript, a short summary, the escalation reason, the sources already cited, and what was already tried. Tell the customer a person has the case and what is already known, so they do not repeat themselves.
Every refusal and escalation stores the exact question text, the missing page or record, and the escalation reason. Answering from general knowledge to avoid a log line is a failure. That shortcut puts an ungrounded reply where a gap should have been written down.
Leave clustering, brief templates, and content automation out of this flow. The website AI assistant article is where logged gaps become the next brief. Stop when the packet is complete and the refusal is stored.
Log the gap — A finished handoff names what was tried and what the knowledge base lacked. Inventing an answer to skip the log is the failure, not the escalation.
Key Takeaways
Map your live intents against the answer, clarify-once, and hard-escalate rules before you raise the automation target.