Overview

Vector databases store and search high-dimensional embeddings. They’re essential for semantic search, recommendation systems, and RAG applications.

How They Work

Text → Embedding Model → Vector [0.1, -0.3, 0.8, ...] → Store → ANN Search

Database Comparison

DatabaseHostingOpen SourceBest For
PineconeManagedNoProduction, scale
WeaviateBothYesHybrid search
ChromaSelf-hostYesLocal dev, RAG
MilvusBothYesLarge scale
QdrantBothYesFiltering
pgvectorSelf-hostYesPostgres users

Chroma (Local Development)

import chromadb
from chromadb.utils import embedding_functions

client = chromadb.Client()
ef = embedding_functions.SentenceTransformerEmbeddingFunction()

collection = client.create_collection("docs", embedding_function=ef)

collection.add(
    documents=["Machine learning is...", "Deep learning uses..."],
    ids=["doc1", "doc2"]
)

results = collection.query(query_texts=["What is ML?"], n_results=2)

Pinecone (Production)

from pinecone import Pinecone

pc = Pinecone(api_key="your-key")
index = pc.Index("my-index")

index.upsert(vectors=[
    {"id": "doc1", "values": [0.1, 0.2, ...], "metadata": {"source": "wiki"}}
])

results = index.query(vector=[0.1, 0.2, ...], top_k=5, include_metadata=True)

Qdrant (Filtering)

from qdrant_client import QdrantClient
from qdrant_client.models import Filter, FieldCondition, MatchValue

client = QdrantClient("localhost", port=6333)

# Search with filters
results = client.search(
    collection_name="docs",
    query_vector=[0.1, 0.2, ...],
    query_filter=Filter(
        must=[FieldCondition(key="category", match=MatchValue(value="tech"))]
    ),
    limit=10
)

Choosing a Database

  • Prototyping: Chroma (no setup)
  • Production SaaS: Pinecone (managed)
  • Self-hosted scale: Milvus or Qdrant
  • Existing Postgres: pgvector
  • Hybrid search: Weaviate

Key Resources