Helm7

Which Database for LLM Memory? A Plain Comparison

The Helm7 team · Published 14 November 2026 · Checked 16 November 2026 · 8 min read

llm memory: a row of vintage metal filing cabinets in black and white
Photo by Magda Ehlers on Pexels

LLM memory sounds like it needs special infrastructure, and the market is happy to sell you some. In practice it is a few things: find similar text, filter by who it belongs to, sometimes follow links, and delete it when asked. We compared four families of database against those jobs using each project's own documentation, and worked out how much space the vectors really take. Most teams are fine on the database they already run.

In short

  • LLM memory needs similarity search, exact filters, keyword match, sometimes relationships, and reliable deletion. Judge databases on those, not on raw search speed.
  • Postgres with pgvector does exact search by default and approximate search with HNSW or IVFFlat. It lets one query filter, rank and join in a single transaction.
  • A dedicated vector database such as Qdrant is built for filtered search at large scale. The cost is another system and a second copy of your filter data.
  • Use a graph database when relationships are the main event. For two or three hops over foreign keys, a recursive query in Postgres is enough.
  • One million memories at 512 half-precision dimensions is about 1 GB of vectors. Plan from arithmetic, not fear.

What does LLM memory need from a database?

Strip away the branding and llm memory is a set of records an application writes now and reads later, in a different session, often on behalf of a specific person. That gives a short list of requirements.

  • Similarity search: find the stored items closest in meaning to a new message.
  • Exact filters: only this user's memories, only live ones, only a certain kind. These must be applied properly, not as an afterthought.
  • Keyword match: names, codes and rare terms that embeddings blur.
  • Relationships: sometimes you want the customer, their tickets and the last decision, not one lonely row.
  • Updates and deletion: facts change, and people ask to be forgotten.
  • Operations: backups, access control and something your team already knows how to run.

Score each database on those, not on a benchmark of nearest-neighbour speed over a million random vectors. Speed matters, but only after the rest is right.

How do the four families compare?

Four database families against the needs of LLM memory, using each project's documentation as of 16 November 2026
NeedPostgres with pgvectorDedicated vector database (Qdrant as example)Graph database (Neo4j as example)Key-value store (Redis as example)
Similarity searchExact by default, HNSW or IVFFlat for approximateBuilt around it, HNSWVector indexes using HNSWFLAT or HNSW vector index
Filters with the searchOrdinary SQL WHERE on any columnPayload indexes extend the HNSW graph, one traversalCheck the productText, tag, numeric and geo filters
Keyword matchtsvector and GIN, in the same queryCheck the productCheck the productFull-text filters in the same index
RelationshipsForeign keys, joins, recursive queriesNot the pointThe whole pointNot the point
Delete one person's dataDELETE in a transactionCheck the productDelete nodes and edgesDelete keys
Extra service to runNo, if you already run PostgresYes, or a managed cloudYes, or a managed cloudYes, or a managed cloud

We filled the cells only from documentation we read. Where a page did not say, the cell says to check the product. Treat the table as a map of where to look, not a verdict.

When is Postgres with pgvector enough?

For llm memory, more often than the marketing suggests. The pgvector README describes the extension as open-source vector similarity search for Postgres. It adds four types: vector for single precision, halfvec for half precision, bit and sparsevec. It offers cosine distance with the <=> operator, and it does exact nearest-neighbour search by default, with perfect recall. Add an HNSW or IVFFlat index and search becomes approximate.

Dimension limits matter for planning. A vector column can be indexed up to 2,000 dimensions, and halfvec up to 4,000. Storage allows up to 16,000. Common embedding sizes fit comfortably.

The big win is that your memory table is an ordinary table. A query can filter by user, check a validity date, rank by meaning and join to the customer record, in one statement and one transaction. Postgres row level security can restrict, per user, which rows a query returns, so the memory store inherits the access rules you already wrote. Our walkthrough of the retrieval side is in GraphRAG on plain Postgres.

It is not free. A busy memory table adds write load, index builds take memory and time, and the database you share with your app now has another reason to need tuning. Backups are on your side too, though that is the same job you already do. Our guide to backing up and restoring PostgreSQL covers it.

When does a dedicated vector database earn its place?

Qdrant is a good example to read about. Its documentation describes a client-server system with official clients in six languages, HTTP and gRPC interfaces, and four ways to run it: managed cloud, hybrid cloud, private cloud and open-source self-hosting.

Its documented strength is filtering during the search. Payload indexes on specific fields extend the HNSW graph so a filter and a nearest-neighbour search happen in a single traversal. Qdrant recommends building those indexes before you load data. If your workload is huge numbers of vectors, heavy concurrent search and little else, a system built for exactly that can be the better answer.

For llm memory at modest scale, the costs are an extra system to run, a second copy of whatever metadata you filter on, and a deletion and consistency story that spans two stores. That is a fair trade at large scale, and a poor one for a product with a few thousand memories per user.

Do you need a graph database for LLM memory?

For llm memory, only if relationships are the main event. Neo4j's documentation says its vector indexes use HNSW graphs, provided through Apache Lucene, for approximate nearest-neighbour search. The dimension setting accepts 1 to 4096, and cosine is the default similarity. So a graph database can store embeddings, and it can follow links natively with a query language built for it.

That is the right tool when your memory is a knowledge graph by nature: many entity types, deep chains of links, graph algorithms. It is more than you need when your links are foreign keys and you walk two or three hops. A recursive query handles that in Postgres, and we show how in recursive CTEs as graph traversal.

Note that several memory frameworks use graphs underneath, so you may end up with one without choosing it. The comparison of agent memory frameworks lists which ones.

What about a key-value store like Redis?

Key-value stores are the quickest way to keep recent conversation state: a session id in, a blob out. For llm memory that has to be searched by meaning, you need more than keys. Redis now supports that.

“Redis stores vectors and metadata in hashes or JSON objects.”
Redis documentation, Vector search concepts

The Redis documentation offers FLAT and HNSW vector indexes, and it advises FLAT for small datasets, under about one million vectors, or when perfect accuracy matters more than latency. It supports filters over text, numeric, geospatial and tag fields alongside the vector search. It also points to its own agent memory feature for recalling long-term memories.

Where it fits: fast, short-lived working memory, caches of recent retrievals, and teams already running Redis. Where it does not: the durable record of what the agent has learned, which most teams want in a database with transactions and backups they trust.

How much space does LLM memory take?

Less than people fear for llm memory. A vector is its dimensions times its bytes per number. A single-precision vector uses four bytes per dimension and a half-precision one uses two. The table is our arithmetic, for the vectors alone, before indexes, text and row overhead.

Vector storage by memory count, calculated from bytes per dimension (vectors only, before indexes and text)
Memories512 dimensions, half precision (1,024 bytes each)1,536 dimensions, single precision (6,144 bytes each)
100,000about 0.1 GBabout 0.6 GB
1,000,000about 1 GBabout 6 GB
10,000,000about 10 GBabout 61 GB

A single product that stores a few thousand memories per user rarely reaches millions of rows. At that size, an exact scan in Postgres is often fast enough that you do not need a vector index at all. The Helm7 search feature, for example, builds no vector index below 50,000 rows and switches to HNSW above that, with a half-precision 512-dimension column that works out at about 1 KB a row. That is a design choice of ours, not a rule.

How should you decide?

  • Start your llm memory on the database you already run. If it is Postgres, add pgvector and one table. You can move later, because a memory table is easy to export.
  • Add a dedicated vector database when your measurements say so: millions of vectors per index, heavy concurrent search, or latency you cannot tune out.
  • Add a graph database when your questions are really about paths and structure, and a recursive query has stopped being enough.
  • Use a key-value store for working memory and caches, not as the only record.
  • Whatever you choose, test deletion. Pick a user, delete them and confirm their memories are gone from every store, including indexes and backups you rely on.

We sell the Postgres route, so weigh our view accordingly. On the databases page, Helm7 can switch on semantic search and relationships inside your own database, with nothing copied out. If another option fits your workload better, use it.

Common questions

  • What database should I use for LLM memory?

    If you already run Postgres, start there with pgvector and a single memory table. Move to a dedicated vector database only when measured scale or latency demands it, and to a graph database only when path queries are central.

  • Is pgvector good enough for production?

    For many applications, yes. It supports exact search, HNSW and IVFFlat indexes, and several vector types, and it sits inside a database that already gives you transactions, backups and row level security. Measure your own workload.

  • Do I need a vector database for a chatbot with memory?

    Usually not at first. A chatbot storing a few thousand memories per user can use exact search in Postgres. A dedicated vector database makes sense at much larger scale or with very heavy concurrent search.

  • How much storage does LLM memory need?

    A single-precision vector takes four bytes per dimension, so 1,536 dimensions is 6,144 bytes. A half-precision 512-dimension vector takes 1,024 bytes. Indexes and text come on top.

Sources

  1. pgvector on GitHub
  2. Qdrant documentation: overview
  3. Neo4j Cypher manual: vector indexes
  4. Redis documentation: vector search concepts
  5. PostgreSQL documentation: row security policies

Run your entire app stack, end to end, from $12 a month.

App, Postgres, auth, storage, jobs and domains on one platform, planned and priced before anything changes.

More from the blog