Every retrieval-augmented generation prototype starts the same way: embed some documents, store the vectors somewhere, run a similarity search per query. The somewhere is the decision that follows teams into production. Vector search has split into two camps — extending the database you already run, with pgvector on PostgreSQL, versus adopting a dedicated engine such as Qdrant, Weaviate, Milvus, or a managed service — and the right answer depends on scale, freshness, and filtering far more than on marketing benchmarks.
The uncomfortable truth is that most workloads are smaller than the vendor decks assume, and most teams underestimate the operational cost of one more stateful system. Start from the workload, not the tool.
What pgvector Does Well
pgvector gained HNSW indexing in version 0.5, and with half-precision vectors and iterative index scans in later releases it comfortably serves millions of vectors with single-digit-millisecond queries on ordinary hardware. Its real advantages are not raw speed, though — they are properties dedicated engines struggle to match:
- ▸Transactional consistency: the embedding lives in the same database as the row it describes. Insert a document and its vector in one transaction; delete a user and their vectors with one cascade. No sync pipeline, no orphaned embeddings.
- ▸Real filtering: tenant, date, and status predicates are ordinary SQL against ordinary indexes, combined with vector similarity by one query planner. Dedicated engines have improved metadata filtering, but joins against your actual relational schema remain out of reach.
- ▸One system to operate: backups, high availability, monitoring, and access control — all the Postgres machinery you already run now covers vectors too, and your compliance story does not grow a new chapter.
The honest limits: HNSW index builds are memory-hungry and maintenance_work_mem matters, graph inserts are slower than B-tree writes, and a single Postgres node caps how many vectors fit in RAM before recall or latency degrade.
Where Dedicated Engines Earn Their Keep
Cross into the hundreds of millions of vectors, need horizontal sharding, or run high-throughput semantic search as the core product, and dedicated engines justify themselves. Qdrant and Milvus shard collections across nodes and offer scalar and product quantization to trade memory for recall on your own terms. Weaviate bundles hybrid search — BM25 fused with dense vectors — which matters because pure dense retrieval is weak on exact identifiers, part numbers, and rare proper names. Managed offerings remove the operational question entirely, at the price of egress costs, vendor coupling, and another data processor in your security review.
The decision is rarely about approximate-nearest-neighbor performance in isolation; HNSW is HNSW everywhere. It is about who does the sharding, filtering, and lifecycle work — you, Postgres, or a vendor.
Benchmarks That Predict Production
Public leaderboards will not predict your workload. Benchmark with your own embeddings, your own filters, and your own update rate:
1. Build the index with production parameters and record build time and peak memory. 2. Measure recall against exact brute-force results on a held-out query set — recall at 10 is the number that decides answer quality. 3. Sweep the search-effort parameter to map the recall-latency curve; production should sit where that curve flattens. 4. Run queries concurrently with a realistic insert and delete stream — freshness under write load is where systems diverge most. 5. Re-test with your strictest metadata filter applied; filtered ANN can collapse recall on some engines.
The Pipeline Around the Index Matters More
Most retrieval-quality failures are pipeline failures, not index failures. Version the embedding model in your schema — a model upgrade means re-embedding everything, and mixing vectors from different models in one index silently ruins similarity. Chunking strategy — size, overlap, structure awareness — moves recall more than any index parameter. Store the source text and metadata beside the vector so re-embedding is a batch job, not an archaeology project. And treat re-embedding migrations like schema migrations: dual-write to a new index, backfill, flip reads behind a flag, then drop the old index.
The pragmatic path for most teams: start on pgvector, instrument recall and latency from day one, and design the embedding pipeline so the storage engine is swappable. If growth pushes past what one Postgres node can serve, the migration is mechanical, because the pipeline was the real asset all along. That sequencing puts a working AI feature in front of customers in weeks, defers infrastructure spend until the workload proves it is needed, and keeps sensitive data inside the database your compliance posture already covers — three outcomes any CFO and any auditor will sign off on.