Embeddings

How do embeddings work?

An embedding model converts an input into a vector with many numbers. IBM describes how the closeness of two vectors reflects how similar the objects are, measured with Euclidean distance, cosine similarity or the dot product. Early methods such as Word2Vec and GloVe produced one fixed vector per word. Newer models also embed whole sentences and passages, as Elastic’s overview of word, sentence and document embeddings shows.

Microsoft notes that language models assign an embedding to each token and can calculate one for a whole text from these token embeddings.

What are embeddings used for?

Typical uses are semantic search, recommendations, classification and anomaly detection. In AI search with retrieval-augmented generation (RAG), passages of documents and the user’s question are converted to embeddings and compared in a vector database. That finds passages that fit a question even when they use different words.

For website text this means a question and a page can match without sharing the same keyword. How individual vendors weigh embeddings in ranking or citations is not public.

Example (illustrative)

“How do I cancel my order?” and “Withdraw a purchase” share no important words, yet their embeddings sit close together because both describe the same action. “What’s the weather in Hamburg?” lies far away from both.

FAQ

Are embeddings the same as tokens? No. Tokens are the text pieces a model reads. Embeddings are the number vectors that tokens or whole texts are converted into.

What is the difference between vectors and embeddings? A vector is a list of numbers. An embedding is a vector produced by a model so that its position reflects the meaning of the original data.

Last updated: October 8, 2026