Concierge

When Flows Should Refuse a Query Instead of Guessing

Read Time8 Min Read
Length1,793 Words
CategoryConcierge
When Flows Should Refuse a Query Instead of Guessing
AI Generated

A Flows concierge earns trust only when every answer is tied to a page you published. The dangerous failure is not a blank reply. It is a fluent one that fills a gap the knowledge base does not cover. A responsible document-grounded assistant should recognize that gap and decline to guess, rather than generating a fluent-sounding answer that happens to be wrong, as Hyperleap puts it in its handoff design guidance.

Those stops cannot be left to the model at runtime. Write them as if-then holds before launch, and give the assistant three outcomes rather than one handoff button: cite and answer, refuse and log, or escalate to a person. Hyperleap treats a 15–25% escalation rate as a healthy range for many deployments, and warns that a rate meaningfully below 10% can mean the assistant is answering questions it should have stopped on. A very low handoff rate is a warning, not a win.

The card that follows is built for that discipline. It covers a missing citation, a question outside the product map, an irreversible or regulated next step, and an explicit request for a human on the first ask. A clean refusal that routes onward is a successful turn. Guessing is the defect the rules exist to prevent.

Summary
  • Write escalation rules as if-then holds before launch; do not leave the model to decide when it should stop.
  • Use three outcomes: cite and answer, refuse and log, or hand off to a person.
  • Hard-refuse when there is no retrieved source, the question is off the product map, or the next step is irreversible, regulated, or a complaint.
  • Honor an explicit request for a human on the first ask, without another bot retry.
  • Treat a handoff rate meaningfully below 10% as likely guessing; several support guides treat roughly 15–25% as a healthy band.

When an AI assistant should refuse a query instead of answering

Diagram showing when AI assistant escalation rules should refuse a query instead of answering from an empty source.

Start with a refuse, not a guess. An AI assistant should decline a query instead of answering when retrieval has no current article that contains the answer, when the ask sits outside the two-job product map, or when the next step is irreversible, regulated, a complaint, or a billing dispute. Write those as if-then holds before launch. Do not leave the model to decide when it should stop. Copy the three outcomes, not a single handoff button:

01
Cite and answer
Only when a retrieved page actually contains the answer.
02
Refuse and log
No current article, off the product map, or any reply that would still be a guess.
03
Escalate to a person
Irreversible, regulated, complaint, or billing steps — a human approves before anything is done.

Never guess on those topics even if the model sounds sure. High-impact actions need a person to approve them before anything is done; that hold lives in the workflow, not in the model's self-check, which is the control OWASP describes for excessive agency. A fluent wrong answer is worse than a hold: a document-grounded assistant should decline to guess rather than sound certain and be wrong. Benchmark misses are a reason to refuse, not a rate to quote as your concierge's error rate — Stanford HAI's 2026 AI Index still has agents failing roughly one structured attempt in three. What Flows can help with belongs on the product map. Do not republish an allow-list of question classes.

Key Takeaway

Hold, don't guess — No source, off the product map, or a step that needs a person — refuse or hand off. Confidence is not a citation.

Sources

Refusing a query versus escalating it to a human

Two paths contrasting refusing a query and logging it versus escalating it to a person, with confidence not controlling policy.

The difference is who closes the loop. Refuse and log when there is no current page to cite, or the ask sits outside the product map. Escalate when a person must decide: the visitor asked for one, frustration is repeating, or the topic is policy-restricted.

Aloware names that split Can't, Won't, and Shouldn't. Can't with no source is a refusal plus a log. Won't is an immediate handoff on a human request or repeated frustration — do not retry the bot first. Shouldn't — billing disputes, complaints, regulated issues, deletions, and permission changes — is a hard escalate, not a cleverer answer. Hyperleap treats those policy topics as an always-escalate rule, separate from confidence tuning.

Those holds must not depend on a confidence score. IrisAgent's common band — act above 0.85, escalate below 0.75, and clarify in between — is only for retrieval uncertainty. Approval rules stay in the workflow: a persuasive prompt can talk a model out of asking.

Which questions to answer versus escalate is the sibling allow-list. This section only draws the outcome split.

Key Takeaway

Outcome split — Refuse and log when nothing safe can be cited; escalate on the first human request, repeating frustration, or any policy-restricted topic — and never let a confidence score override that hold.

Sources

How do you stop a grounded assistant from answering outside its scope?

Grounded assistant staying inside product scope by naming the missing cited page instead of answering from general knowledge.

The score band never licenses an out-of-map answer. Bind the concierge to the two-job map—automated SEO publishing and the on-site concierge—and to pages it can cite. If the ask is not on that map, refuse and point to the map. Do not invent adjacent services.

If retrieved passages do not contain the answer, use one line the concierge can repeat: there is no current page that answers this, so it will not guess. Offer the handoff or the log. Do not fill the gap with general knowledge. Userflow recommends that exact abstention when passages are empty; write Flows' own sentence rather than pasting a vendor script.

Copy the pattern, not the wording. Gorgias documents that its AI will not speculate beyond connected sources and hands over instead of guessing when it cannot find a relevant answer. That sentence is what blocks invented tools: in τ-bench, Yao et al. (2024) found GPT-4o generated 0.46 non-existent ID tool calls per task across simulated retail and airline support.

Key Takeaway

Scope hold — If the ask is off the two-job map or missing from a cited page, refuse in one Flows sentence and hand off or log—never guess, and never invent a service.

Sources

Honor a human request immediately—don't wait for a miss quota

A direct request for a human honored immediately, and a failed clarify followed by handoff rather than another bot retry.

Those scope holds stop guessing. A direct request for a person is a different rule, and the answer is yes: honor it on the first ask. Do not retry the bot, clarify, or offer another article first. An explicit request to speak with a human should always be honored immediately, without the assistant attempting another resolution first.

There is no healthy multi-miss quota either. After one clarify that still has no cited answer, or when frustration repeats, hand off. A second guessed answer is the failure, not a recovery.

Handoff situations include an explicit human request, detected frustration, sensitive topics, and actions that need approval, along with low confidence or a knowledge gap when a person can actually take the case. A knowledge gap with no one waiting is still refuse and log, not a fake queue ticket. The fail-closed sibling already uses answer, clarify once, or hand off—follow that hold, and do not restate its use-case list.

Key Takeaway

First ask — Honor a human request on the first ask, and hand off after one unanswered clarify. Guessing again is the failure, not a recovery.

A healthy chatbot escalation rate, and when a low one is a red flag

Escalation-rate gauge with a healthy chatbot escalation band and a too-low zone marked as a guessing red flag.

After the first human ask is honored, the rate is the next reading. A healthy chatbot escalation rate for support is roughly 15–25%, the band Hyperleap treats as healthy for many deployments, inside IdeaPlan’s wider customer-support range of about 15–30%. Internal copilots sit lower, about 5–15%—do not copy that number for a site concierge. These are published support ranges, not a Flows SLA.

15–25%
Healthy support band many deployments use
15–30%
Wider published customer-support range
5–15%
Internal copilots—do not copy for a site concierge

A rate meaningfully under about 10%, or near zero, is a guessing signal: the assistant is answering questions it should refuse or escalate. Past about 30–40%, triggers are too sensitive or scope is too wide. Lower is not the goal.

Handling is not resolution. One benchmark had chatbots handle 73.8% of chats in 2024 while resolving 45.8%. Judge transfer success, re-ask rate, and post-handoff resolution, not deflection.

A clean refusal that routes honestly is a success. An assistant that never says it is unsure is unsupervised. Guessing an uncited claim hurts citations more than a refusal: answer engines can quote a real page, not a fluent gap. No study in this source set says a clean refusal reduces AI visibility.

Key Takeaway

Red flag — A very low escalation rate is a warning, not a win. Refuse or hand off when there is no source; do not invent a citeable answer.

Handled chats vs actual resolution

Sources

What context must travel with an AI-to-human handoff?

Handoff packet carrying transcript, trigger, and citations to a person, with after-hours refusals logged instead of a new answer.

A rate in that band only helps if the person receiving the chat already knows why it arrived. Send the transcript, the trigger that fired, and any citations retrieved—including an explicit none—so the visitor does not repeat the issue. Name the trigger in the packet as Can't, Won't, or Shouldn't. IrisAgent finds that customers who have to repeat their issue to a human rate the experience 30 to 40 points lower than those who do not, and a clean handoff with full context cuts handle time on transferred conversations by 40% to 60%.

If no person is available, do not reopen the model to fill the silence. Confirm the refusal, leave the product-map page or a contact path, and keep the same log. A fallback answer is still a guess.

Key Takeaway

Packet — Every handoff carries the transcript, a named Can't, Won't, or Shouldn't trigger, and citations or an explicit none; if no one is free, confirm the refusal and do not guess.

What to log when the assistant refuses, and how to test the rules first

Refusal log clustering repeated intents into one content brief beside a short pre-launch test list of hold scenarios.

Log the query, the outcome—refuse or escalate—the trigger, citations retrieved or none, and the transcript. A one-off miss stays a log line. The same intent repeating becomes a brief or a page patch, not a new pillar. Point that weekly cluster at the unanswered-question workflow you already run; this only defines the fields and the repeat rule.

Before the concierge is on, write the holds as if-then rules instead of leaving the model to decide when to stop: no retrieved page, a question outside the product map, billing or a complaint, an irreversible action, an explicit person request, and a second miss after one clarify. Confirm a persuasive follow-up cannot talk the assistant out of the hold, and that the handoff packet still carries transcript, trigger, and citations. Do not turn it on because a demo answer sounded sure.

Key Takeaways

[01]
Three outcomesA cited-page Flows concierge should cite and answer, refuse and log, or hand off to a person, and should not guess when it has no source, the stakes are too high, or the reader asked for a human.
[02]
Can't, Won't, Shouldn'tRefuse and log a no-source or out-of-map ask; escalate on the first human request or repeated frustration; hard-escalate billing disputes, complaints, regulated issues, deletions, and permission changes.
[03]
Scope bindingOut-of-scope answers are stopped by tying the assistant to the two-job product map and citable pages, and by abstaining when passages are empty instead of inventing an answer.
[04]
Immediate honorAn explicit request for a person is handed off on the first ask, with no retry or miss quota, while a knowledge gap with nobody waiting stays a logged refusal rather than a fake ticket.
[05]
Escalation bandA healthy support escalation rate sits roughly in the mid-teens to mid-twenties; rates near zero often mean the assistant is guessing, and a clean refusal counts as a citation-safe success.
[06]
Packet, log, and testEvery refusal or handoff should carry the query, outcome, named trigger, citations or an explicit none, and the transcript, and the holds should be script-tested—including a persuasive override—before launch.

Write those if-then holds into the workflow, bind them to the product map, and test the packet before the concierge answers live queries.

Frequently Asked Questions

When should a Flows concierge refuse instead of answering?
Refuse when no retrieved page contains the answer, when the question sits outside the product map, or when the next step is irreversible, regulated, or a complaint. Hyperleap’s rule is to decline to guess rather than produce a fluent wrong answer, and to log the miss so it can become a brief or a page patch.
What are the three outcomes an escalation card should allow?
Cite and answer, refuse and log, or hand off to a person. A single handoff button hides the difference between a grounded answer, a knowledge-base gap, and a case a person must own.
Should an explicit request for a human be retried by the assistant first?
No. Hyperleap says an explicit request to speak with a human should be honored immediately, without the assistant attempting another resolution first.
What escalation rate suggests the assistant is guessing?
Hyperleap warns that a rate meaningfully below 10% can mean the AI is answering questions it should escalate. It treats 15–25% as a healthy range for many deployments. Measure transfer success, re-ask rate, and post-handoff resolution, not deflection alone.
Why do policy-restricted topics need a hard rule?
A model can sound confident while giving advice it should not give. Hyperleap separates policy-restricted topics—billing disputes, complaints, and regulated issues—from confidence tuning, and requires a hard always-escalate rule.
What must travel with a handoff so the reader does not repeat themselves?
Pass the transcript, the trigger, and any citations. IrisAgent reports that customers who have to repeat their issue rate the experience 30 to 40 points lower, and that a clean handoff with full context cuts handle time on transferred conversations by 40% to 60%.
Sources

You Might Also Like

View All Articles