Embeddings and the vector space

How text becomes a point in a high-dimensional space — and why cosine similarity, not distance, is what makes retrieval work.

Why it matters

An embedding model maps a piece of text to a vector of a few hundred to a few thousand numbers (e.g. 1536 dimensions for a common OpenAI model). The model is trained so that texts with similar meaning end up as vectors that point in similar directions — 'the cat sat on the mat' and 'a feline rested on the rug' land close together even though they share almost no words.

This is why retrieval uses cosine similarity instead of raw distance: cosine similarity measures the angle between two vectors, ignoring their length. That matters because a long, repetitive chunk can produce a vector with a much larger magnitude than a short, precise chunk about the same topic — if you compared raw Euclidean distance, the long chunk could look 'far away' purely because of its length, not its meaning. Cosine similarity strips that out: two vectors pointing the same direction score 1.0 (identical meaning), perpendicular vectors score 0 (unrelated), and opposite vectors score -1 (contradictory).

The practical payoff: once every chunk and every query is a point in this shared space, retrieval becomes a geometric operation — 'find the k chunks whose vectors have the smallest angle to the query vector' — instead of a keyword match. That's also why embeddings fail on pure keyword or exact-number lookups: the model has smoothed away the specific digits or IDs in favor of general meaning, which is precisely where hybrid (keyword + vector) search earns its keep.

An embedding maps text to a point in a high-dimensional space (often 384–3072 dims) such that texts with similar meaning land near each other. Retrieval then becomes geometry: embed the query, find the nearest chunk vectors.

Why cosine, not Euclidean. Cosine similarity measures the angle between two vectors and ignores their length. Two passages about refunds should match even if one is a sentence and one is a paragraph (very different magnitudes, similar direction). That length-invariance is exactly what you want for semantic search.

cos(a, b) = (a · b) / (‖a‖ ‖b‖)
# 1.0 = identical direction, 0 = unrelated, -1 = opposite

Three facts that change how you build:

1. The model defines the geometry. Distances are only comparable within one embedding model. If you re-embed your corpus with a new model, every stored vector and threshold is invalidated — you must reindex.
2. Symmetry matters. Some models are trained for symmetric similarity (sentence ↔ sentence), others asymmetric (short query ↔ long document). Using the wrong one quietly tanks recall.
3. Embeddings encode the training distribution. Domain jargon (drug codes, legal citations) may collapse to nearly the same point. That's when you reach for fine-tuned or hybrid lexical+vector retrieval.

Debugging tip. When retrieval is bad, embed a known-good query and a known-good chunk and print the cosine score by hand. If the 'obvious' match scores low, the problem is the embedding model or the chunk text — not the database.

Embeddings are a city map for meaning. Every sentence becomes an address; cosine similarity is the compass bearing between two addresses. Change the map (the embedding model) and every address moves — so directions you memorised on the old map now point at the wrong building.

• Mixing embedding models in one index. Symptom: nonsense neighbours. Fix: one model per index; reindex on change.
• Comparing scores across models. A 0.8 from model A is not a 0.8 from model B. Fix: re-tune thresholds per model.
• Ignoring query/document asymmetry. Fix: pick a model trained for your retrieval shape (short query → long doc).
• Pure vector search on rare identifiers. Codes and SKUs embed poorly. Fix: hybrid with keyword/BM25.