How to Add Citations to RAG Answers
Add RAG citations that cite sources at the span level: grounded answers with links, refusal when retrieval is empty, UI highlights, and faithfulness checks.
To add citations to RAG answers, retrieve chunks with stable IDs and offsets, require every factual claim to point at one of those IDs, refuse when nothing supports the claim, render grounded answers with links and span highlights, and verify faithfulness so the model cannot invent sources.
What you’ll build
- Every factual sentence points at a chunk id, URL, and span — or the turn is a refusal
- Span-level highlights in the source pane, not a footnote to a 40-page PDF
- A verifier that drops or flags claims the retrieved text does not support
- No fabricated URLs, page numbers, or quote marks around paraphrases the source does not contain
- Faithfulness scores on a weekly sample, with thumbs-down routed to missing-source tickets
Before you start
- 01A retriever that returns chunk id, source URL, and character (or token) offsets — not only raw text
- 02A generator that can emit structured citations, or a post-hoc linker you will accept as v1
- 03A golden set of questions with gold passages, including some that must be refused
- 04A UI that can open a source and highlight a span, even if v1 is just a link
- 05Agreement that an answer without a source is a defect, not a style choice
Key takeaways
- 01
RAG citations are a contract: no unsupported sentence ships. The UI is how users audit that contract.
- 02
Pass allowed source IDs into a structured output schema; never let the model mint a new one.
- 03
Span-level offsets beat document-level footnotes; quote only when the text matches.
- 04
Post-hoc citation (answer first, link later) is a v1 — pair it with a verifier or you will decorate hallucinations.
- 05
Empty retrieval and failed verification are refusals, not hedging.
Cite sources in LLM answers as a contract, not a footnote
A RAG citation exists so a reader can audit a sentence. That means a stable source ID, a URL they can open, and a span they can see. Filename footnotes, 'according to internal docs,' and markdown links the model dreamed up do not meet that bar. If you cannot highlight the supporting text, you do not have RAG citations — you have decoration.
The contract is simple to state and easy to violate: every factual claim either points at retrieved evidence or the system refuses. Opinions, chitchat, and 'I don't know' are exempt. Policy numbers, dates, prices, and names are not. Write that in the product spec before you argue about superscript vs hover cards.
Span-level citations vs document-level links
Document-level citations ('source: Travel Policy') send the reader hunting. Chunk-level is better. Span-level is what users trust: start and end offsets into the chunk, highlighted on click. Store those offsets at ingest. If you only stored embeddings, you will be regex-matching the quote at request time and failing on harmless whitespace.
Multi-hop answers need more than one span. Split the completion into claims (sentence-level is a decent default) and map each claim to one or more spans. 'Parental leave is 16 weeks and starts after 6 months' is two claims, possibly two documents. A single footnote under the paragraph hides a miss on the second number.
- Legal citation object: chunk_id, url, title_path, page, start, end
- Inline marker or structured claims[] — pick one and parse it in code
- Verbatim quote: character match after whitespace fold, or do not use quote marks
- Unknown id in the completion: drop the claim, do not look the id up in the corpus
Constrained generation vs post-hoc linking
Constrained generation is the production default: the schema lists allowed source IDs, the model must attach them, and the parser rejects extras. Inline markers like [S12] work if S12 was in the pack. Do not ask the model to emit raw URLs; it will autocomplete a plausible path.
Post-hoc linking (generate freely, then retrieve quotes that support each sentence) is faster to prototype and worse at honesty. It can staple a real link onto an invented fact if the overlap is coincidental. If you start there, add a verifier that requires the cited span to entail the claim, and still refuse when retrieval was empty before generation.
Refusal without sources, and the UI
Empty retrieval, permission-filtered-empty, and 'we retrieved three chunks and none entail the claim' are the same user-visible outcome: we do not have a source, here is who to ask. Do not distinguish 'no document' from 'a document you cannot see' — that is an existence leak. Do not hedge ('it might be 16 weeks'). Hedging is still an unsourced number.
The UI should make the contract visible. Click a sentence, see the span. Show a source pane. If verification failed and you still show a draft to an operator, mark it unverified in a way a customer never sees. Grounded answers with links are a product surface, not an afterthought in markdown.
Faithfulness checks that catch fake citations
Three failure modes: unsupported claim (no span), contradictory span (the source says the opposite), and fabricated citation (id or URL not in the pack). The first two need a textual entailment check — overlap heuristics catch copies, a small NLI or judge catches paraphrases. The third is a set-membership check in code and should be zero.
Put those checks in CI on a golden set that includes unanswerable questions and look-alike policies. Weekly, sample live traces: citation coverage (claims with a valid id), verifier fail rate, and user clicks on sources. If users never click, your highlights may be wrong — or the answers may be too vague to audit.
What we implement
The existing RAG overview on this site stops at 'require citations.' This page is the rest: offsets, claim mapping, verifier, refusal, UI. ReinforcedX implements that layer in your stack on top of the retriever you already have, or as part of a four-week RAG build.
You own the index metadata, evals, and runbooks. Work stays in your VPC. Model-agnostic, no token markup, 30 days on-call after handover. We will not ship a footnote list that the model is free to invent.
Step-by-step build
- 1
Make retrieval citation-ready
Return chunk_id, document_id, URL, title path, page, and start/end offsets on every hit; if the index cannot store offsets, fix the index before you touch the prompt.
- 2
Number the evidence pack
Give the generator a short list of sources with those IDs and a truncated quote; the only legal citations are IDs from that list.
- 3
Require per-claim citations in the schema
Have the model emit claims[] with text plus source_ids[], or inline markers that you parse; drop any claim whose IDs are missing or unknown.
- 4
Verify before render
Run a faithfulness check (NLI, overlap, or a small judge) from claim to cited span; strip or refuse claims that fail, and never display a URL the retriever did not return.
- 5
Render links and highlights
In the UI, attach each sentence to a clickable source and highlight the span; show quote marks only on verbatim matches.
- 6
Refuse and measure
On empty retrieval or total verification failure, return a refusal plus 'ask the owner'; track citation coverage, fabricated-id rate, and faithfulness weekly.
Common pitfalls
The mistakes that show up in real deployments — each one costs a week if you learn it the hard way.
Footnotes to the whole document
Citing 'HR policy.pdf' after a three-sentence answer is theater. The user cannot see the clause. Store offsets and open the span.
Letting the model invent citation IDs
If the prompt says 'cite [1][2]' and the model has no constrained list, it will cite [3] from a document you never retrieved. Pass the allowed IDs in the schema and reject anything else in code.
Quote marks around a paraphrase
Users treat quotation marks as a verbatim guarantee. If the span is a paraphrase, do not quote it. If you quote, the characters must match the source after whitespace fold.
Citing after a synthesis with no mapping
Multi-source answers need per-claim citations. A pile of links at the bottom does not tell you which number came from which table.
No refusal path
When retrieval is empty or the verifier fails, the model should say it cannot find a source — not 'answer anyway with low confidence.' Low confidence still ships a sentence.