Skip to content
Blog

Dense Vector Embeddings Explained for Backend Engineers

A practical explanation of dense vector embeddings: how they are generated, what their dimensions mean, and how backend systems store and query them.

Published on • November 22, 2026

AI Assistant

Introduction

If you build backend systems for AI applications, you will eventually encounter dense vector embeddings — arrays of floating-point numbers that represent the meaning of text, images, or audio. This article explains what they are, how they are produced, and how they flow through a production system.

What Is a Dense Vector?

A dense vector is a fixed-length array of floats where every dimension carries information. A 768-dimensional embedding is literally 768 numbers, typically between -1 and 1, with no zeros to skip.

This contrasts with sparse vectors (like TF-IDF or BM25 term vectors), where most dimensions are zero and only a few term weights are non-zero.

“Dense” means the meaning is distributed across all dimensions. You cannot point to dimension #412 and say “this is the word cat.” Meaning emerges from the pattern of the whole vector.

How Embeddings Are Produced

An embedding model (Sentence-BERT, OpenAI text-embedding-*, Voyage, Cohere, etc.) maps input to a point in a continuous vector space with one critical property:

Semantically similar inputs land close together.

  • “How do I reset my password?” and “I forgot my login credentials” → nearby points
  • “How do I reset my password?” and “Best restaurants in Bangkok” → distant points

The model is trained so that geometric distance (via cosine similarity or Euclidean distance) correlates with semantic similarity.

What Backend Engineers Need to Know

1. Dimensions are model-specific

A 1536-dimension vector from one model and a 1536-dimension vector from another model are not comparable. Dimensions from different models occupy different spaces. Never mix models against one index without re-embedding.

2. Storage is predictable

DimensionsFloat32 size1M vectors
3841.5 KB~1.5 GB
7683 KB~3 GB
15366 KB~6 GB

Add HNSW graph overhead (roughly 2× the raw vector size is common) and index metadata when sizing your cluster.

3. The write path

text → embedding model → vector + metadata → vector DB upsert

4. The read path

query text → same embedding model → query vector → top-k search → ranked chunks

Both sides must use the same model, same version, and same preprocessing.

Embeddings Are Not Magic

Embeddings encode similarity, not truth. They inherit the biases and blind spots of their training data. Production systems pair them with rerankers, metadata filters, and keyword search to stay accurate.

Conclusion

Dense embeddings are the interface between human meaning and machine computation. Once you understand them as fixed-size numeric records with strict compatibility rules, the rest of the vector database stack — indexing, metrics, recall — becomes much easier to reason about.

References