Hierarchical community summaries

Pre-generating executive summaries across macro and micro thematic clusters, so 'global' questions don't require reading everything at answer-time.

Why it matters

Chunk-based RAG is built to answer local questions — 'what does clause 4.2 say?' — by finding the one or two chunks that contain the answer. It fails badly on global questions like 'what are the overarching themes across these 500 documents?', because no single chunk (or even top-20 chunks) contains a corpus-wide summary — that information only exists in aggregate, across thousands of chunks the LLM can never fit in one prompt.

Hierarchical community summaries fix this by doing expensive synthesis offline, once, instead of live for every query. After community detection (previous lesson) has grouped entities into clusters, an LLM is run once per community to write a short summary of what that cluster is about — and communities are typically organised at multiple levels: fine-grained 'micro' communities (a handful of tightly related entities) roll up into coarser 'macro' communities (broad topical areas), much like a table of contents has chapters and sub-sections.

At query time, a global question no longer needs to touch raw text at all: the retriever can pull the small number of macro-level summaries, and only drill down into micro-summaries or raw chunks if the query needs specifics. This is the key trade: pay the LLM synthesis cost once, offline, amortised across every future query, instead of paying it (and the latency) fresh on every request.

Hierarchical community summaries pre-compute a table of contents for meaning. You summarise each leaf community, then summarise groups of leaf summaries into parent themes — a tree built offline, before any query arrives.

leaf_summary:   {input: chunks+edges of one community, output: [claims, entities, open_questions]}
parent_summary: {input: related leaf summaries,         output: [theme, conflicts, key_sources]}

Why offline. Synthesising themes at query time would mean reading thousands of chunks per question — slow and expensive. Doing it once, ahead of time, turns a global question into a cheap read of a few summary nodes. It is the classic latency-vs-precompute trade: pay once at build, save on every query.

The non-negotiable rule: keep source pointers attached at every level. A summary that has lost its links to the underlying chunks becomes an ungrounded rumour — the model will happily generate from it with no way to verify. Each summary node must carry the ids of the claims and source spans it was built from, so answer-time generation can still cite primary evidence rather than 'a summary said so'.

Conflict handling. Two communities may make contradictory claims. A mature summary schema records the conflict explicitly (conflicts: [...]) instead of silently averaging them, so the agent can surface the disagreement rather than hide it.

Hierarchical summaries are a book's index plus chapter abstracts. You don't re-read the whole book to answer 'what's the theme of Part II' — you read the abstract. But a good abstract still cites page numbers, so a sceptic can verify. Drop the page numbers and the abstract is just hearsay.

• Summaries without source links. Symptom: ungroundable answers. Fix: attach chunk/claim ids at every level.
• Generating from summary alone. Fix: require primary-source evidence at answer time.
• Averaging away conflicts. Fix: record contradictions explicitly in the summary schema.
• Never refreshing summaries. Symptom: stale themes. Fix: invalidate summaries when their community changes.