Every RAG pipeline leans on one underneath. Here's what it's actually doing, and whether you need a dedicated one or a Postgres extension will do.
A vector database stores embeddings (long lists of numbers that represent the meaning of a piece of text, an image, or anything else a model can encode), and finds the ones closest to a given query embedding. That's the whole job: turn "find things similar in meaning to this" into a fast lookup, instead of a linear scan through every row comparing meaning by hand. It's the retrieval half of RAG, and increasingly the memory layer behind AI agents that need to recall relevant context instead of re-reading everything.
An embedding model turns a chunk of text into a vector, typically a few hundred to a few thousand floating-point numbers, positioned so that semantically similar inputs end up close together in that space. Comparing two vectors' "closeness" uses a similarity metric: cosine similarity (the angle between them) is the default for most text embedding models, with dot product and Euclidean (L2) distance used by models specifically trained for them. Get the metric wrong for your model and results degrade quietly, with no error to warn you.
Comparing a query vector against every stored vector, exact nearest-neighbor search, works fine at a few thousand records and falls over well before a million. Vector databases instead build an approximate nearest-neighbor (ANN) index, most commonly HNSW (Hierarchical Navigable Small World), a graph structure that lets a query jump toward the right neighborhood in roughly logarithmic time instead of scanning everything. The trade is a small, tunable amount of recall for search times that stay fast as the collection grows into the millions.
In practice, most systems combine this with metadata filtering (narrowing to a tenant, a date range, or a document type before or during the similarity search), because "closest in meaning, among the records you're actually allowed to see" is the real query most applications need, not pure similarity search in isolation.
Purpose-built systems (Pinecone, Weaviate, Qdrant, Milvus) are built around the vector workload from the ground up: distributed ANN indexes, multi-tenant isolation, hybrid keyword-plus-vector search, and horizontal scaling designed in rather than bolted on. They're the right call when vector search is a primary, high-scale workload, or when you need features a general-purpose database doesn't offer.
The alternative that covers most real applications: pgvector, an extension that adds a vector column type and HNSW indexing directly to PostgreSQL. If you're already running Postgres, and most teams are, this means one database, one backup strategy, one place to run a query that joins vector similarity with ordinary relational filters, instead of keeping two systems in sync. For the RAG pipeline behind a typical internal tool or product feature, pgvector with a properly tuned HNSW index handles millions of vectors with strong recall, and the operational simplicity of not running a second database system is worth more than the marginal performance a dedicated store would add at that scale.
You're already running Postgres and want vector search alongside relational data, not a second system to operate.
Your workload is a typical RAG pipeline or agent memory store: millions of vectors, not billions.
Simplicity and one fewer moving part matter more than squeezing out the last bit of query latency.
Vector search is the primary workload at very large scale, not a feature bolted onto an existing app.
You need multi-tenant isolation, hybrid search, or distributed scaling that a single Postgres instance won't give you.
You're building a product where vector search performance is the differentiator, not incidental infrastructure.
No, pgvector on a Postgres instance you likely already run, or even an in-memory library for a few thousand vectors, is enough to prove out an idea. Reach for a dedicated system once you have a real workload to measure against, not before.
No, it's a complement. You still need a system of record for structured, transactional data: the vector store (or vector-capable extension) sits alongside it, indexing embeddings derived from that data or from unstructured content.
Exact search means comparing a query vector against every stored vector: it doesn't scale past a small dataset. Approximate nearest neighbor (ANN) algorithms like HNSW trade a small amount of recall for search times that stay fast as the collection grows into the millions.
Cosine similarity is the default for most text embedding models and is a safe starting point. Check your embedding model's documentation: some are trained and normalized for dot product or Euclidean (L2) distance specifically, and using the wrong metric quietly degrades result quality.
For most production RAG workloads, yes: pgvector with an HNSW index handles millions of vectors with strong recall while keeping everything in one database. Dedicated systems pull ahead at very large scale or when you need multi-tenant isolation and features pgvector doesn't offer.
A practical comparison of retrieval-augmented generation and fine-tuning for AI agents: cost, latency, data freshness, and when each approach actually makes sense.
Read the guideRelational, document, key-value, wide-column: what each model is actually good at, and why "both" is often the real answer.
Read the guide