When to evolve from RAG to GraphRAG

A concrete decision framework for upgrading a retrieval system when multi-hop questions become dominant.

Why it matters

Many teams jump to GraphRAG too early or too late. A quantified transition rule prevents wasted complexity and avoids quality plateaus.

GraphRAG adds an extraction-and-graph-build pipeline on top of RAG. It pays for that complexity only under specific workloads — so the transition should be a measured decision, not a trend-driven one.

Measure three things before deciding:

1. Query shape. Sample real queries and label them: single-hop factoid, multi-hop relational, or global/thematic ('summarise all themes'). If multi-hop + global dominate (say >35%), naive top-k will keep failing structurally.
2. Corpus signal. Do the same entities recur across many documents, with relationships between documents? GraphRAG's value comes from connecting facts that live in different chunks — no recurring entities, no payoff.
3. Failure transcript. Pull answers that were wrong and ask: would a 2–3 hop traversal have fixed this? If the answer is repeatedly 'yes', the graph is earning its keep.

transition_rule:
  if multi_hop_ratio >= 0.35 and recurring_entities: add graph extraction + hybrid retrieval
  else: invest in better chunking, re-ranking, and query rewriting first

Why not always GraphRAG? The build pipeline (extraction, canonicalisation, community summaries, incremental refresh) is real, ongoing engineering. For a 20-page handbook answered by single-hop questions, it is pure overhead. The honest progression is: exhaust cheap RAG improvements, then upgrade when the data proves multi-hop demand.

Moving to GraphRAG is upgrading from a phone book to a detective's pin-board. A phone book is perfect for 'what's Acme's number?'. The pin-board with red string between photos earns its wall space only when your questions are 'who connects this supplier to that competitor?' — buy the string when the cases demand it, not before.

• Adopting GraphRAG on hype. Symptom: months of build for no measurable lift. Fix: gate on measured multi-hop ratio.
• Adopting too late. Symptom: a recall ceiling no chunking fixes. Fix: watch for structural multi-hop failures.
• Skipping cheap wins first. Fix: re-ranking + query rewriting often close most of the gap at a fraction of the cost.
• No build-cost estimate. Fix: price the extraction + refresh pipeline before committing.