Approximate Nearest Neighbor search lets systems find the closest vectors in a large index without comparing every point. Indexes such as HNSW or IVF make semantic retrieval fast enough for interactive search and AI assistants at scale.
Whilst ANN delivers speed, in practice you trade a little recall for latency — and the wrong settings surface near-misses that look confident. For example, a support bot may retrieve a similar policy from the wrong year because the index was tuned for speed over precision. We often recommend fixing candidate counts and measuring recall@k on real queries before calling the index “production-ready.”
Exact nearest-neighbor search compares a query vector to every stored vector. That is accurate and too slow once corpora reach millions of embeddings. ANN indexes build a structure — commonly an HNSW graph of neighbor links, or IVF clusters that narrow the scan to a few partitions — so the engine only examines a subset of candidates. You choose how approximate: higher search effort (efSearch, nprobe, candidate count) improves recall and costs latency. Production work is parameterization and measurement, not the algorithm name on the slide. Rebuild or upsert strategy matters when documents change; a fast index on stale vectors still retrieves the wrong neighbor. Pair ANN with metadata filters when hard constraints (date, tenant, doc type) must bind; distance alone will not enforce them.
import hnswlib
import numpy as np
dim, n = 1536, 100_000
data = np.random.randn(n, dim).astype(np.float32)
data /= np.linalg.norm(data, axis=1, keepdims=True) # cosine via inner product
index = hnswlib.Index(space="cosine", dim=dim)
index.init_index(max_elements=n, ef_construction=200, M=16)
index.add_items(data, np.arange(n))
index.set_ef(64) # search ef: raise for recall, lower for latency
query = data[0]
labels, distances = index.knn_query(query, k=10)
# validate recall@k on a labeled set before shipping ef / M defaults