GraphRAG: When Knowledge Graphs Beat Vector Databases | Arpit Mishra

Standard RAG works because embedding-based similarity search is genuinely good at one thing: finding text that means something close to what you asked, even when the wording is completely different. Ask "how do I get a refund" and it will happily surface a chunk titled "Cancelling your subscription," because the two land near each other in vector space. For the large majority of question-answering tasks, that's exactly the right tool, and it's why vector databases became the default backbone of RAG.

But similarity search has a shape, and that shape has a blind spot. This article is about that blind spot, what a knowledge graph does about it, and why the practical answer usually isn't "graph instead of vectors" but "graph and vectors."

The question vector search can't really answer

A vector store treats every chunk as an island. Each chunk gets embedded on its own, scored against the query on its own, and returned on its own merits. That's fine when the answer to your question lives entirely inside one chunk, or can be assembled from a handful of chunks that are each independently similar to the question. It breaks down when the answer isn't in any single chunk — it's in the relationship between facts scattered across many documents.

Take a question like: "Which suppliers does Company A depend on that its competitors also depend on?" No single chunk in your corpus is "about" that question. The answer requires knowing Company A's suppliers (maybe from a procurement report), Company A's competitors (maybe from a market analysis), and each competitor's suppliers (scattered across several more documents) — then intersecting all of that. Embedding the question and searching for "similar" chunks will surface documents that mention suppliers, or competitors, or Company A, but it has no mechanism for the multi-hop join that actually answers the question. You'd get plausible-looking chunks and a model doing its best to guess at the connection, which is a polite way of describing a hallucination.

What a knowledge graph actually is here

In this context, a knowledge graph is a much more literal structure than the phrase might suggest: nodes are entities — people, companies, products, concepts — and edges are explicit, typed relationships between them. "Company A" —supplies→ "Component X"; "Component X" —used_by→ "Company B." The relationship is data, not prose you have to re-infer every time.

These graphs get built one of two ways. A small, high-value domain (an org chart, a product catalog, a regulatory taxonomy) might be modeled by hand, because precision matters more than coverage. Most real deployments instead build the graph automatically during ingestion: an LLM reads each document, pulls out the entities it mentions and the relationships asserted between them, and writes those out as graph triples alongside the usual chunking-and-embedding step. The graph grows as a byproduct of the same pipeline that feeds your vector store.

Retrieval by traversal, not just similarity

Once relationships are explicit edges instead of implicit text, a different retrieval strategy becomes possible. Rather than only asking "which chunks look like this query," you can ask "starting from this entity, what is it connected to, and what is that connected to?" A two- or three-hop traversal — Company A → its suppliers → those suppliers' other customers — answers exactly the kind of question vector search struggled with, because the connective structure is walked directly instead of hoped for.

This is also why graph retrieval tends to shine on "what connects X and Y" questions specifically. Finding the shortest path (or the set of paths) between two nodes in a graph is a well-understood traversal problem; finding the same connection by similarity search means hoping some chunk happens to mention both X and Y together, which for a genuinely indirect relationship, it usually doesn't.

GraphRAG: the hybrid pattern, not a replacement

It's tempting to frame this as graphs versus vectors, but almost no serious system actually throws away its vector index. GraphRAG, as the pattern is commonly used, is a two-stage retrieval: vector search still does the job it's good at — turning a fuzzy natural-language query into a handful of relevant entry-point entities or chunks — and graph traversal then takes over from there, expanding outward from those entry points along explicit relationships to pull in connected context the query never mentioned directly.

So a query about Company A's supplier risk might first use vector search to identify "Company A" and "supply chain" as the relevant entities, then hand off to the graph to walk from "Company A" through its supplier edges, competitor edges, and those competitors' supplier edges — context that no amount of query-chunk similarity would have assembled on its own. The vector store finds where to start; the graph finds what else matters from there.

A graph is a cost, not a default upgrade

Building an accurate knowledge graph is a genuinely harder problem than chunking and embedding text. Automated entity and relationship extraction is error-prone — an LLM will miss relationships, invent spurious ones, and represent the same real-world entity under slightly different names unless you invest in resolution and cleanup. And once the graph exists, multi-hop traversal queries bring their own performance and complexity considerations that a flat vector index simply doesn't have. Reach for GraphRAG when your actual queries require connecting facts across entities — not as a blanket replacement for a vector store that's already answering most of your questions just fine.

Deciding which problem you actually have

The practical test is simple: look at the questions your system keeps failing on. If the failures are "the right chunk existed but retrieval didn't surface it," that's a chunking, embedding, or re-ranking problem, and a graph won't fix it. If the failures are "no single chunk contained the answer because it depended on connecting two or three separate facts across documents," that's the specific shape of problem a knowledge graph is built to solve — and it's worth the added engineering cost precisely because vector similarity structurally cannot do that join.

Key takeaways

Vector search excels at finding chunks that are individually semantically similar to a query — that's most of what RAG needs.
It's structurally weak at questions whose answer requires connecting facts across multiple documents through an explicit relationship, not just finding similar text.
A knowledge graph makes those relationships explicit: entities as nodes, typed relationships as edges, built by hand or extracted automatically during ingestion.
Graph traversal turns multi-hop 'what connects X and Y' questions into a walk across edges instead of a hope that one chunk mentions both.
GraphRAG is typically hybrid: vector search finds the entry-point entities, graph traversal expands outward from them — not graph instead of vectors.
An automatically extracted graph is error-prone and traversal has its own cost; add a graph when your queries genuinely need multi-hop reasoning, not by default.