When to evolve from RAG to GraphRAG

A concrete decision framework for upgrading your retrieval system — before multi-hop questions quietly become the majority of your traffic.

Why it matters

GraphRAG (Level 2) is strictly more expensive to build and operate than naïve RAG: it needs an extraction pipeline, a graph store, and a more complex query planner. So the real skill isn't knowing GraphRAG exists — it's knowing when the upgrade pays for itself. Two failure directions are equally common:

• Jumping too early: building a full graph pipeline when 90% of user questions are simple single-document lookups. You pay GraphRAG's latency and engineering cost for RAG's problem.
• Waiting too long: staying on naïve RAG while users increasingly ask questions like 'which supplier of ours was also mentioned in the vendor-risk report from last quarter?' — questions that require composing facts that live in different documents. No amount of better chunking or more k fixes this, because the answer genuinely doesn't exist inside any single chunk.

A practical transition rule: log every user query, tag whether the correct answer required combining facts from more than one source, and track that percentage over time. When multi-hop questions cross roughly 15-20% of traffic — or when you notice support tickets citing 'the bot gave a half-right answer that mixed up two different things' — that's the quantified signal to invest in GraphRAG, rather than reacting to a single embarrassing demo failure.

GraphRAG adds an extraction-and-graph-build pipeline on top of RAG. It pays for that complexity only under specific workloads — so the transition should be a measured decision, not a trend-driven one.

Measure three things before deciding:

1. Query shape. Sample real queries and label them: single-hop factoid, multi-hop relational, or global/thematic ('summarise all themes'). If multi-hop + global dominate (say >35%), naive top-k will keep failing structurally.
2. Corpus signal. Do the same entities recur across many documents, with relationships between documents? GraphRAG's value comes from connecting facts that live in different chunks — no recurring entities, no payoff.
3. Failure transcript. Pull answers that were wrong and ask: would a 2–3 hop traversal have fixed this? If the answer is repeatedly 'yes', the graph is earning its keep.

transition_rule:
  if multi_hop_ratio >= 0.35 and recurring_entities: add graph extraction + hybrid retrieval
  else: invest in better chunking, re-ranking, and query rewriting first

Why not always GraphRAG? The build pipeline (extraction, canonicalisation, community summaries, incremental refresh) is real, ongoing engineering. For a 20-page handbook answered by single-hop questions, it is pure overhead. The honest progression is: exhaust cheap RAG improvements, then upgrade when the data proves multi-hop demand.

Moving to GraphRAG is upgrading from a phone book to a detective's pin-board. A phone book is perfect for 'what's Acme's number?'. The pin-board with red string between photos earns its wall space only when your questions are 'who connects this supplier to that competitor?' — buy the string when the cases demand it, not before.

• Adopting GraphRAG on hype. Symptom: months of build for no measurable lift. Fix: gate on measured multi-hop ratio.
• Adopting too late. Symptom: a recall ceiling no chunking fixes. Fix: watch for structural multi-hop failures.
• Skipping cheap wins first. Fix: re-ranking + query rewriting often close most of the gap at a fraction of the cost.
• No build-cost estimate. Fix: price the extraction + refresh pipeline before committing.