What is GraphRAG, and when is it worth it?

What GraphRAG is

Plain RAG cuts documents into chunks, finds the chunks closest to a question and hands them to the model. Microsoft Research's announcement names three places where that falls short: it "struggles to connect the dots" when an answer needs facts from separate documents linked by something they share; it does poorly at understanding summarized themes across a large collection; and it struggles with questions that need information aggregated across the whole dataset, such as the top themes in it.

GraphRAG changes the indexing step. In the paper by Darren Edge and co-authors, an AI reads the source documents and extracts an entity knowledge graph, then pre-generates summaries for communities, which are groups of closely related entities. At question time, each community summary gives a partial answer and those are combined into the final one. The authors report more comprehensive and more diverse answers than a plain RAG baseline on whole-collection questions over datasets of about one million tokens.

Microsoft's open-source GraphRAG offers several ways to ask: global search for questions about the whole collection, local search for questions about one entity and its neighbours, DRIFT search that mixes the two, and basic search, which is plain RAG for questions it already answers well.

Why a small team should care

• The questions founders ask are often global. "What do lost deals have in common?" or "Which features do enterprise prospects keep asking about?" No single passage answers them.
• Relationships are where the value hides. A graph shows that three churned accounts shared the same integration partner, which nobody wrote down in one place.
• It is not free. Every document is read by the model up front, and the graph needs rebuilding when documents change. Try it on one collection before committing.

A worked example

Illustration: a made-up business. Loop Analytics, a nine-person startup, has 120 call notes from sales and customer success. The CEO asks a plain RAG assistant, "Why do mid-size customers churn?" It returns three notes that contain the word "churn" and summarizes them. The answer is true for those three calls and misses the pattern.

A teammate runs an entity and relationship extraction over the notes: each customer, its size, the integrations it uses, the problems raised, and who raised them, with a quote as evidence for every link. The resulting table shows that 11 of 14 churned mid-size customers used the same CRM connector, and that every one of them reported sync errors in the month before cancelling. Neither fact was in any single note. The team now knows what to ask engineering, and what a GraphRAG index over all call notes would be worth.

How to try it without a data team

1. Write five questions your current search answers badly, ideally ones about patterns across documents.
2. Pick 20 to 50 documents that should hold the answers, such as call notes or interview summaries.
3. Extract entities and relationships into a table with a source quote for each relationship, and merge duplicate names (Acme, Acme Inc., ACME).
4. Read the five questions against the table. If the graph answers them and search didn't, GraphRAG is worth a proper build.
5. Agree the entity types (customer, product, problem) before scaling; an ontology is the formal version of that list.

Common confusions

• "GraphRAG replaces RAG." Microsoft's own implementation keeps a basic search mode for questions plain RAG answers well.
• "We need a graph database first." Not to find out whether the idea helps. A table of subject, relationship and object, with a quote each, is enough to test.
• "The graph is always right." An AI extracted it. Keep the source quote and confidence for every link, and check the ones a decision rests on. See knowledge graphs for how they are built by hand and by AI.

Questions

What is the difference between RAG and GraphRAG?

Plain RAG finds the passages that match a question and answers from them. GraphRAG first has an AI extract the people, products, places and relationships in all the documents into a graph, groups related entities into communities and summarizes each group, then answers from those summaries and links. It does better on questions about the whole collection and on connecting facts spread across documents.

Who made GraphRAG?

Microsoft Research. It described the method in a blog post on 13 February 2024 and a paper by Darren Edge and co-authors submitted in April 2024, and publishes an open-source implementation.

Does a small team need GraphRAG?

Usually not at first. If your questions are about one fact at a time (a price, a policy), clean documents and plain RAG are enough. GraphRAG pays off when people ask about patterns across many documents, such as themes in 300 customer interviews.