Embeddings
Lists of numbers that represent the meaning of a piece of text (or an image), produced by an embedding model, so that similar content ends up close together.
Why it matters. The basis of semantic search and retrieval, where you find the relevant documents before sending them to a model.
How it works
An embedding is a fixed-length vector of numbers that an embedding model produces for a piece of input, usually text. Inputs with similar meaning produce vectors that are close together, measured with cosine similarity or a similar distance.
Embedding models are separate from chat models and are usually priced per input token, typically well below chat-model rates. They differ in vector size, maximum input length and the languages and domains they handle well. Vectors from different embedding models are not compatible with each other.
Typical uses are semantic search, retrieval-augmented generation (RAG), clustering, deduplication and recommendations. Store vectors in a vector database or index, and record which model produced them.
Worked example
Illustrative: embedding 10,000 documents of 800 tokens each is 8M tokens; at $0.10 per 1M that is $0.80. Stored as 1,024-dimension vectors of 4-byte floats, they take 10,000 x 1,024 x 4 bytes, about 41 MB.
Common mistakes
- Switching embedding models without re-embedding the whole corpus, mixing incompatible vectors.
- Embedding long documents as a single vector instead of chunking them, which blurs their meaning.
- Sending the nearest matches to the model without a relevance threshold or reranking step.