GraphRAG Systems
GraphRAG system design: entity and relation extraction, community summaries, query-time graph traversal, hybrid fallback, failure modes, evals, and when chunks beat graphs.
- Read time
- 17 min
- Level
- Advanced
- Updated
- 2026-08-25
- Components
- Entity/relation extractor · Graph store · Community summarizer
A GraphRAG system extracts entities and relations from a corpus into a knowledge graph, builds hierarchical community summaries over that graph, and at query time traverses local neighborhoods or global summaries instead of (or in addition to) returning isolated text chunks. It outperforms chunk RAG when the question needs aggregation, multi-hop relations, or a map of “who is connected to what,” not a single passage.
Key takeaways
- 01
Build a graph when questions are relational or corpus-wide (“how do these vendors connect?”); keep chunk RAG when questions are “find the paragraph that says X.”
- 02
Extraction quality is the product: noisy entities and untyped edges make every later hop more wrong, not more insightful.
- 03
Community detection plus hierarchical summaries is what makes global questions cheap; local k-hop traversal is what makes entity-centric questions precise.
- 04
Always keep a hybrid chunk index as a fallback and as a citation source — graphs answer structure, chunks still hold the wording you must quote.
- 05
Evaluate extraction, retrieval, and generation separately: entity F1, hop accuracy, and groundedness against the source documents, not against the graph alone.
When a graph beats a chunk index
Chunk RAG retrieves local text. It is the right architecture for “what is our parental-leave policy” and “what does error 5042 mean.” It is the wrong architecture for “which suppliers share a parent with a sanctioned entity,” “summarize how incident IR-440 spread across teams,” or “what themes connect last quarter’s board papers.” Those questions need edges, not nearer neighbors.
GraphRAG earns its cost on three query classes: multi-hop (A relates to B relates to C), aggregation (themes, communities, coverage across many documents), and identity (the same person, SKU, or account under different strings). If your golden set is mostly single-hop factoid lookup, a graph is overhead. If analysts already draw boxes and arrows on the whiteboard, you are in graph territory.
Components and ingest data flow
Ingest still starts with documents, but the primary artifact is not a chunk table. An extractor — usually an LLM with a constrained schema, plus rules for identifiers — emits entities (typed nodes with canonical ids), relations (typed, directed, evidenced by a span), and optional claims. A resolution step merges “Acme Corp,” “ACME,” and the ticker into one node. The graph store holds nodes, edges, evidence pointers back to document spans, and timestamps. A community detector (Leiden or similar) partitions the graph; a summarizer writes descriptions bottom-up so each community has a text the retriever can embed.
Re-extract incrementally. Full-corpus re-runs are how graphs rot: new documents never attach to old entities, and deleted documents leave ghost edges. Store evidence as (doc id, span, extractor version). When the extractor prompt changes, version the graph rather than silently mutating edges that answers already cited.
- Extractor: typed entities and relations with source spans
- Resolver: canonical ids, aliases, and conflict rules
- Graph store: nodes, edges, evidence, time, ACL
- Communities: hierarchical clusters plus generated summaries
- Chunk index: fallback retrieval and quotation source
Query-time traversal: local vs global
Local search starts from entities mentioned in the query (or retrieved by a first-pass chunk search), walks a bounded neighborhood (k-hop, with type and time filters), and packs the subgraph plus evidence spans for the generator. This is the path for “what do we know about vendor V and contract C.” Cap hops and node count; unbounded traversal is a denial-of-wallet against your own graph.
Global search does not walk from a seed entity. It retrieves community summaries — often via embeddings over the summary text — at a hierarchy level that matches the question’s scope, then descends into child communities only where the summaries look relevant. This is the path for “what are the dominant risk themes in this corpus.” Many production systems route: if the query names resolvable entities, local; if it asks for themes or coverage, global; if both fail, hybrid chunk RAG.
Failure modes
Extraction errors compound. A missed relation drops a hop. A duplicated entity splits the neighborhood so local search never sees the full picture. An over-eager extractor invents edges the source text does not support; the generator then cites the graph as if it were evidence. Community summaries drift from the underlying documents and become a second, ungrounded corpus. None of these look like HTTP 500s.
Query-time failures are usually budget and leakage. Traversal that ignores ACL will walk through a node the user cannot read and still emit its neighbors. Summaries built without permission context can leak facts from restricted documents into a “theme” the user is allowed to see. Global search that always pulls the top-level summary answers every question with the same paragraph. Treat graph text as derived data: the user-visible answer must still ground in source spans the user is allowed to read.
- Unresolved aliases splitting one real entity into many nodes
- Hallucinated edges with no source span
- Stale communities after incremental ingest
- Unbounded k-hop expansion (cost and noise)
- Community summaries leaking restricted facts
Evals
Split the eval. Extraction: entity and relation F1 against a labeled document sample, plus alias-resolution accuracy. Retrieval: for local queries, did the gold entities and edges appear in the packed subgraph within the hop budget; for global queries, did the selected communities contain the gold documents. Generation: faithfulness to the cited source spans, not to the graph summary. A fluent answer that cites only a community summary is ungrounded even if the summary happened to be right.
Hold a slice of questions that chunk RAG already solves. GraphRAG should not regress them. Hold a slice that requires two or more hops; that slice is the reason the system exists. Track extractor cost per document and traversal p95 separately from generation cost — graphs fail first as bills, then as quality.
When not to use GraphRAG
Do not build a graph to look modern on a corpus of FAQs, runbooks, and single-policy lookup. Hybrid chunk retrieval with citations will beat it on hit rate, latency, and operational cost. Do not build a graph when you cannot name the entity types and relation types in a one-page schema — an unconstrained “extract everything” graph is a second messy corpus. Do not skip the chunk index; you need it for quotations, for extractor misses, and for the day the graph is wrong.
Prefer GraphRAG when analysts already think in entities (customers, parts, incidents, legal parties), when questions routinely mention two named things and a relation, or when leadership wants corpus-level maps rather than document Q&A. In those cases the graph is the index, the summaries are the skim layer, and the source spans remain the system of record.