TL;DR — Graphify replaces the chunk-and-embed pipeline with a typed knowledge graph and a hybrid retriever, yielding measurable wins on multi-hop, temporal, and entity-centric questions. Teams that migrate see fewer hallucinations, simpler prompt construction, and lower token cost per query — at the price of an ETL step most vector stacks never had to do.

Why the Vector Store Hit a Wall

For two years the default RAG recipe was almost identical: chunk a corpus, embed each chunk with a sentence-transformer, drop vectors into Pinecone or pgvector, retrieve by cosine similarity, and stuff the top-k into a prompt. It worked beautifully for “What is the capital of Burkina Faso?” and it fell apart in roughly the same way every time: anything that required reasoning across more than one passage.

The failure modes are now well documented. Microsoft Research’s GraphRAG paper showed that pure-vector retrieval loses recall sharply on questions like “Which executives left after the 2023 restructuring and where are they now?” because the answer is stitched across three documents, two of which share no vocabulary with the query. Anthropic’s evaluations of long-context retrieval reach a similar conclusion from a different angle: as context windows grow, models still miss connections that are not lexically close to the question.

The vector store is a flat index. It answers “what is similar to this query?” but it has no idea that “Anthropic” and “the company that makes Claude” are the same entity, that “Series C” and “2023 funding round” refer to the same event, or that the CFO mentioned in one PDF and the CFO mentioned in another press release are the same human being.

What Graphify Actually Is

Graphify, built by getzep, is an open-source pipeline (github.com/getzep/graphify) that turns unstructured text into a typed property graph and exposes that graph as the retrieval substrate for an LLM. Instead of a vector index over chunks, you get nodes for entities (people, products, events, concepts) and edges for relationships (works_at, acquired, mentioned_in, contradicts).

The retrieval surface is then hybrid by construction:

  • Semantic search over entity descriptions and short canonical summaries, so “the CEO” still finds Satya Nadella.
  • Graph traversal from the matched seed nodes, following typed edges for a configurable number of hops.
  • Lexical / full-text fallback for exact-match cases like SKUs, model numbers, or contract clauses.

The query planner picks the right mix per question, and the response is a small subgraph plus the matched text spans — not 12 unrelated chunks that the model has to mentally stitch together.

A flat vector index is a phonebook. A graph is a phonebook that also knows who lives with whom, who sued whom, and which companies share a board member. Most real questions are about the relationships.

Architecture: Patterns in Production

A Graphify deployment looks like three services, each of which can scale independently.

                ┌──────────────┐
   raw docs ───▶│  Graphify    │──┐
                │  ingest job  │  │
                └──────────────┘  ▼
                          ┌─────────────────┐
                          │  Graph DB       │
                          │  (Neo4j / FalkorDB) │
                          │  + BM25 index   │
                          └─────────────────┘
                                   ▲
                          ┌────────┴────────┐
                          │   Retriever     │◀── user query
                          │  (hybrid planner)│
                          └────────┬────────┘
                                   ▼
                          ┌─────────────────┐
                          │  LLM (prompt =  │
                          │  subgraph + text)│
                          └─────────────────┘

The ingest job is the part most teams underestimate. It is not “run an embedding model.” It is:

  1. Chunk with semantic boundary detection (section, heading, table — never a fixed token window).
  2. Extract entities and relations with a structured-output LLM call (Graphify ships prompts tuned for this; you can swap in your own).
  3. Resolve entities across chunks using embedding similarity on canonical names plus deterministic rules (“Inc.”, “Corp.”, “Limited” → same org).
  4. Write to the graph store with timestamps, source document IDs, and confidence scores on every edge.
  5. Index the entity descriptions in a BM25 plus dense hybrid index for the seed-matching step.

The retriever is roughly 200 lines of code in the reference implementation. The expensive part is the graph store. Neo4j’s retrieval guide and FalkorDB’s docs both cover the patterns; for most teams a single Neo4j instance handles tens of millions of nodes without breaking a sweat, especially after you drop low-confidence edges.

The Numbers: Why Teams Are Migrating

The most cited internal benchmark in this space comes from teams replacing a flat pgvector index with a Graphify-backed Neo4j setup. The pattern is consistent:

  • Recall@10 on multi-hop QA: typically 1.5×–3× higher than dense-only retrieval at the same chunk budget.
  • Hallucination rate (judged by a second LLM against ground truth): 30%–60% lower, because the model is reasoning over a structured substrate rather than free-associating across retrieved chunks.
  • Tokens per answer: often lower, because the subgraph is denser information per token than 8 raw chunks. Teams report 20%–40% prompt-size reduction for the same answer quality.
  • Ingest cost: higher. You are running an LLM over every chunk during extraction. This is the tradeoff.

The cost increase is real and worth being honest about. Graphify’s own docs are upfront that the ingest stage is roughly an order of magnitude more expensive than embedding a corpus once. But ingest runs once per document; retrieval runs per query, and at scale the retrieval savings dominate.

A Concrete Migration: From pgvector to Graphify

Here is the minimal change to a typical LangChain retrieval pipeline. The “before” is the chunk-and-vector pattern most teams shipped in 2024:

# before: vector-only retrieval
from langchain.vectorstores import PGVector
from langchain.embeddings import OpenAIEmbeddings
from langchain.chains import RetrievalQA

vs = PGVector.from_documents(
    chunks,
    OpenAIEmbeddings(),
    connection_string="postgresql://...",
)
qa = RetrievalQA.from_chain_type(
    llm=llm,
    retriever=vs.as_retriever(search_kwargs={"k": 12}),
)

The “after” hands the same documents to Graphify and queries through the hybrid retriever:

# after: graph-backed hybrid retrieval
from graphify import Graphify

graph = Graphify.from_documents(
    documents=chunks,
    llm=extractor_llm,        # used during ingest only
    graph_db_uri="bolt://neo4j:7687",
    embedder=OpenAIEmbeddings(),
)
graph.ingest()                # one-time, runs the extract/resolve/write pipeline

result = graph.retrieve(
    "Which executives left after the 2023 restructuring and where are they now?",
    hops=2,
    top_k_entities=10,
    top_k_chunks=6,
)
answer = llm.invoke(render_prompt(result))

The API surface is deliberately familiar. The interesting bit is what result contains: a subgraph with typed edges, the matched source spans, and provenance back to the original documents. The prompt you build from it can be much smaller and much more grounded.

When Not to Switch

Graph-based retrieval is not a universal upgrade. Skip the migration if any of the following are true:

  • Your corpus is short, homogeneous, and your questions are single-hop. A vector index over well-chunked FAQs will outperform the graph because there is nothing to traverse.
  • Your questions are mostly “find me the paragraph that mentions X” — lexical search with BM25 is simpler and just as accurate.
  • You cannot afford an LLM in the ingest path, even amortized. Some regulated environments ban sending source documents to third-party APIs; in that case a Graphify deployment has to run a local extractor model, which is feasible but adds ops surface.
  • Your team has no graph literacy. A vector store is a table; a graph store is a small database with its own query language. You will write Cypher.

The pragmatic rule of thumb: if more than ~20% of your user questions require combining facts from two or more documents, the graph pays for itself.

Operations: What Actually Breaks

Teams that have run Graphify in production report the same handful of operational headaches, and they are solvable but they are not free.

Entity explosion. Without aggressive resolution, the graph fills with duplicate nodes (“Apple”, “Apple Inc.”, “AAPL”, “the Cupertino company”). Graphify ships a resolver; you still need to tune the similarity threshold and maintain a domain-specific alias table for high-value entities.

Drift on temporal facts. A graph node for a person is stable; their job title is not. The recommended pattern is to make title, subsidiary_of, headquartered_in edges with a valid_from / valid_until range rather than properties on the node. Neo4j’s temporal patterns guide and the GraphRAG literature both cover this.

Confidence pollution. The extractor LLM will confidently assert edges that are not in the source text. Keep a confidence property on every edge and a source_span pointer to the exact text that justified the extraction. Then write a periodic job that drops edges below your threshold and re-extracts their neighborhoods with a stronger model.

Cost spikes during re-ingest. Re-extracting a 10-million-document corpus because the extractor model changed is painful. The fix is to version your extraction prompts and your graph schema, and to keep ingest idempotent on document hash.

Key Takeaways

  • Vectors answer “what is similar to this?” Graphs answer “what is connected to what?” Most real RAG questions are the second kind, which is why Graphify’s hybrid retriever outperforms flat vector search on multi-hop tasks.
  • The migration is not just a database swap. You are adding an LLM-powered ETL stage. Plan the cost and the failure modes (entity explosion, temporal drift, confidence pollution) before you start.
  • Retrieval cost goes down, ingest cost goes up. At query volume, the retrieval savings win. At low query volume, the ingest cost may not amortize.
  • Graph databases are not exotic anymore. Neo4j, FalkorDB, and Memgraph all run comfortably on a single node for most corpora. The graph-query skills your team needs are smaller than they were three years ago.
  • Keep a vector store in the loop. The hybrid pattern — graph for structure, dense vectors for semantic seed matching, BM25 for exact terms — is what actually wins. Pure-graph retrieval has its own failure modes on paraphrase-heavy queries.

Further Reading