Vector Databases Explained: Embeddings, ANN, and When You Need One

Vector search powers semantic search, RAG (retrieval-augmented generation), recommendation systems and image similarity. Here is the field from a backend developer’s perspective.

What is an embedding?

An embedding is a fixed-size vector of floats that represents a piece of content (text, image, audio) in a way where semantic similarity → small distance.

"a cute puppy"   → [0.12, -0.34, 0.88, ..., 0.07]   (1536 dims)
"a small dog"    → [0.10, -0.32, 0.86, ..., 0.05]   (very close)
"a tax invoice"  → [0.61,  0.04, 0.21, ..., -0.41]  (far)

Models like OpenAI text-embedding-3-small, Cohere embed-v3 and open-source nomic-embed produce these.

Why a special database?

To find the nearest neighbors out of 100M vectors you cannot scan linearly. Vector databases use Approximate Nearest Neighbor (ANN) indexes:

• HNSW (Hierarchical Navigable Small World) — graph-based, excellent recall, fast.
• IVF (Inverted File) — partitions space; tunable speed/recall trade-off.
• PQ (Product Quantization) — compresses vectors; reduces memory by 10×.

The 2025 landscape

Tool	Best for
pgvector	You already use Postgres and have < 50M vectors
Qdrant	Open-source, single-binary, hybrid search built-in
Weaviate	Modules for OpenAI, Cohere, custom rerankers
Pinecone	Managed, hands-off, pay per pod
Milvus	Massive scale (billions), distributed
Elasticsearch	You already index full text and want hybrid

Hybrid search wins

Pure vector search misses exact terms (product codes, names). Combine:

1. BM25 keyword score (Elastic, Postgres FTS).
2. Vector similarity score.
3. Reciprocal Rank Fusion (RRF) to merge them.

This is now the default architecture for production semantic search.

Storage math

A 1536-dim float32 vector = 6 KB. 10M vectors = 60 GB. Use int8 quantization to cut by 4× with minor recall loss.

When you don’t need a vector DB

• Less than 100k items → in-memory FAISS or even brute-force cosine works.
• You need exact matches → use a real text index, not embeddings.
• Your “search” is structured (filters on price, category) → SQL is enough.

TL;DR

Start with pgvector if you already run Postgres. Move to a dedicated vector DB only when scale or features (hybrid, multi-tenant, replication) force you. The model and the chunking strategy matter more than the index for retrieval quality.