AI Hallucinations FAQ
Why language models invent facts, how to catch it, and the production controls that reduce it — not a prompt that says “don’t hallucinate.”
An AI hallucination is a fluent claim that is not supported by the provided sources or by the world the system is supposed to answer from. Models hallucinate because they sample likely tokens, not because they “lie”; you reduce it with grounding, refusal when retrieval is empty, citations, and claim-level evals — you do not switch it off.
What hallucinations are
If you only need one test: could a sceptical reader find this claim in the sources you gave the model? If not, it is a hallucination.
What are AI hallucinations?
An AI hallucination is a fluent statement that is not supported by the evidence the system was supposed to use — retrieved documents, tool results, or a defined knowledge cut-off. It includes invented citations, wrong figures, and policies that sound official and are not in the handbook. The model is doing what it was trained to do: continue text. Production systems constrain that continuation or refuse.
Why does ChatGPT hallucinate?
Because next-token sampling favours fluent, typical text, not verified text. When the prompt lacks sources, when sources conflict, or when the model’s parametric memory is stale, it still produces an answer. Consumer ChatGPT is also willing to fill gaps unless you constrain it. Enterprise systems add retrieval, refusal, and evals; they do not rewrite the sampling process. Larger models reduce some errors and invent more sophisticated ones.
What types of hallucinations show up in production?
Factual: a wrong number or date. Citation: a real-looking source that does not contain the claim, or a URL that does not exist. Faithfulness: the retrieved policy says X and the answer says Y. Intrinsic: the answer contradicts itself. Tool: the model invents an API result it never got. Each type needs a different check. A single thumbs-down score hides which one you have.
Is a hallucination the same as outdated knowledge?
No. Outdated knowledge is a fact that was true in training data and is now wrong. A hallucination can be about something the model never knew. RAG addresses both only if the index is fresh and the prompt forbids answering from memory when sources exist. Treating every wrong answer as “hallucination” makes the fix bag useless; freshness SLOs and grounding checks are different jobs.
Does RAG stop hallucinations?
It reduces them when retrieval is right, the prompt forbids ungrounded claims, and empty retrieval refuses. It does not stop them when the wrong chunk is fetched, when the model ignores context, or when you stuff low-quality drafts into the index. Hallucination audits catch claim-level failures against the source. The fix is better retrieval plus policy, not a larger generator alone.
Can you eliminate hallucinations completely?
No, not while the model is sampling language. You can drive grounded-task error rates down to a number you will sign for, with refusal and a human queue on the rest. Claiming zero hallucinations is a red flag. We will quote a faithfulness target on your golden set; we will not promise a model that never invents a clause.
How do you stop hallucinations in production?
Four controls together: hybrid retrieval with a re-ranker, refuse-when-empty, citations that must align to spans, and a faithfulness eval in CI. Add tool-result binding so the model cannot invent API output, and a human path for high-stakes claims. The how-to on reducing hallucinations walks the playbook. A system prompt that only says “be accurate” is not a control.
How you detect and reduce them
You cannot manage a failure you have not labelled. Start with real questions and the passages that should support the answer.
How do you detect hallucinations?
Label claims in the answer and check each against retrieved spans or tool results. Automatic faithfulness judges help at volume; humans still review a slice and every high-severity miss. NLI-style entailment, citation alignment, and known-negative questions (no source on purpose) are the usual trio. If you only spot-check demos, you will ship fluent errors.
What is a faithfulness score?
A number for whether the answer stays inside the provided evidence — typically the fraction of claims supported by retrieved text, scored by a rubric judge and spot-checked by people. Keep it separate from retrieval recall and from helpfulness. A fluent, off-source answer can look helpful and still fail. CI should fail the release when faithfulness drops on the golden set.
How do citations help with hallucinations?
They force the system to point at a document and heading path, which gives you a handle for alignment checks and gives the user a way to verify. Fake citations are themselves a hallucination class you must eval. Citation without a refuse-when-unaligned rule is decoration. Production answers carry the URL and the span, or they do not ship.
What if retrieval returns the wrong passage?
The model may faithfully summarise the wrong thing — that is a retrieval failure, not a generation failure, and a single “quality” score will hide it. Measure recall of the right chunk, then faithfulness to whatever was retrieved. Fixes: hybrid search, filters, re-ranking, better chunking. Do not punish the generator for a missed SKU if BM25 never ran.
Do larger models hallucinate less?
On some knowledge probes, yes; on grounded enterprise tasks, not reliably, and they can produce more convincing wrong citations. A smaller model with retrieval and a refuse path often beats a larger ungrounded one on faithfulness. We pick models with the eval suite, not with a parameter count. Size is a cost and latency choice once the grounding stack is in place.
How do you audit an existing chatbot we already shipped?
Sample live or historical traces, extract claims, and check them against the sources the bot had — or should have had. Report rates by intent, severity, and whether a citation was real. That is a hallucination audit, not a redesign. Teams use it to decide whether to add refusal and RAG or to turn the bot off on the failing intents. We run this as a scoped service.
Who owns the hallucination eval set?
You do. Golden questions, verified answers, known-empty questions, and the scoring rubric are handed over with the system. Failures from production become new cases. Thirty days of on-call covers the pipeline we shipped. If the set only lives in a vendor UI, you do not own your quality bar.
Hallucinations in live systems
The incident is not the invented sentence. The incident is that nothing in the system was allowed to say “I don’t know.”
What happens when the agent hallucinates in production?
High-stakes or low-confidence outputs route to a human queue. The trace — prompt, retrieval, tools, model version — is stored and becomes an eval case. The same claim shape should then fail automatically. User-facing correction is a runbook step, not a hope that the next sample is better. We do not silently rewrite history in the customer’s thread without an operator.
Can we put a confidence score on answers?
You can expose a calibrated signal based on retrieval strength, disagreement between samples, or a judge — not the model’s own “I’m 90% sure,” which is poorly calibrated. Use it to route to a person, not to decorate the UI with a fake probability. If the score does not change actions (refuse, escalate, confirm), it is not a control.
Do agents hallucinate tool calls too?
Yes. Invented arguments, skipped tools, and fabricated results are tool-use hallucinations. Defenses: strict schemas, executing tools in your runtime (never “trust the model’s pretend output”), idempotency keys, and evals that score tool choice separately from the final sentence. Function-calling systems that skip execution traces cannot tell a hang from a lie.
How long does it take to add grounding controls to a bot we already have?
A refuse path, citations, and a 50+ case faithfulness suite can land inside a four-week implementation on one workflow. Rewriting a multi-channel bot with no traces takes longer because you must first see what it currently says. Shadow mode lets you compare grounded vs current answers before customers see the change.
Will the model provider’s safety filters stop hallucinations?
No. Safety filters target abuse, self-harm, and similar policy classes. They do not know your refund table. Grounding is your retrieval, your refuse rule, and your evals. Provider filters are complementary guardrails, not a faithfulness system. Relying on them for policy accuracy is a category error.
Do anti-hallucination checks make the system slower or more expensive?
Yes, some: extra retrieval, a citation alignment pass, and judges all spend tokens and milliseconds. You budget them on customer-facing and high-stakes paths, not on every internal autocomplete. Caching retrieval and skipping judges on already-passing intents keeps the bill sane. Measure the cost of a wrong answer (refund, incident) against the cost of the check.
Where hallucinations are unacceptable
The model does not know your risk tolerance. You have to encode it as refusal and review.
When is a hallucination acceptable?
On low-stakes drafts a person will edit — brainstorming, first-pass summaries — if the UI does not present the text as fact. It is not acceptable on policies, prices, medical or legal advice, or anything that triggers a write in a system of record. If you cannot say which bucket a workflow is in, treat it as high-stakes until you can.
How do you handle hallucinations in legal or medical workflows?
Mandatory citations, refuse-when-silent, and a human checkpoint before anything customer-facing. The model does not reconcile conflicting policies or diagnose. Architecture is HIPAA-aware or privilege-aware as required, inside your perimeter. Missed-escalation rate is a target. We will not ship a “legal answer bot” that has no lawyer in the loop on novel questions.
What if a support bot invents a policy?
Treat it as a P1 grounding failure: turn that intent to retrieval-only or to a human, add the case to the golden set, and check whether the index was missing the real policy. The agent must never invent a promise. The customer-support FAQ covers the workflow; this is the quality bar those agents fail most often.
Are hallucinations the same as prompt injection?
No. Hallucination is unsupported generation. Prompt injection is an attacker steering the model through untrusted text. They can combine: injected content becomes “evidence” the model faithfully repeats. Defenses differ — grounding vs trust boundaries — and both belong in the suite. Collapsing them into “the model went crazy” delays the fix.
What usually fails in the first 90 days of a “grounded” bot?
Indexing drafts, no known-empty test questions, citations that are not checked, and write tools opening before faithfulness holds. Stale indexes then serve last week’s price as if it were retrieved truth. The RAG FAQ and the reduce-hallucinations how-to are the antidotes. Rescue work almost always starts by measuring claim-level error on live traces.
Hallucinations are unsupported claims, not a personality flaw. They show up as invented citations, wrong numbers, and confident policy that is not in the handbook. Causes include empty or wrong retrieval, prompts that allow guessing, ignored tool results, and stale parametric knowledge. Production controls are hybrid retrieval, refuse-when-empty, span-level citations, faithfulness scores, and a human queue. ReinforcedX builds those controls into agents and runs hallucination audits on systems you already shipped. The how-to on reducing hallucinations is the playbook; this FAQ is the decision layer.
Keep reading
Ready to refine
your workflow?
Share your current process. We’ll help you identify what can be automated — and where efficiency can be reclaimed.