Deutsch
Retrieval & grounding

Vector database

A vector database stores embeddings, the number vectors that represent the meaning of text, images or audio, and retrieves the entries closest to a query. It ranks by similarity instead of by exact matches.

How does a vector database work?

The usual sequence has four steps. A model converts the data into vectors, and the database stores and indexes them. A user query is converted by the same model into a vector. The database then compares that vector with the stored ones using a similarity measure such as cosine similarity or Euclidean distance. It returns the nearest entries, ranked by similarity. Many systems store metadata next to each vector so results can also be filtered, for example by language or date.

Comparing a query with millions of vectors one by one would be slow, so these systems use approximate nearest neighbor (ANN) indexes. They trade a small loss in accuracy for speed. Traditional databases handle exact queries on structured rows and columns, while vector databases handle similarity queries on embeddings. Some established databases now also offer vector search as a feature.

Embeddings: meaning as closeness in spaceSimplified two-dimensional sketch. Dog, puppy and cat sit close together, tax return and tax assessment far away in a group of their own. A query “young dog” sits between dog and puppy, and a circle marks its nearest neighbors. Real embeddings often have 256, 512 or 1024 dimensions.dimension 1dimension 2large distancedogpuppycattax returntax assessment“young dog”embedding of a termquery as an embeddingnearest neighborsClose together meanssimilar meaning.Far apart meanslittle in common.Real embeddings oftenhave 256, 512 or1024 dimensions.Made-up example in 2 dimensions, positions not calculated.
Made-up example. According to Google, points that sit close together in embedding space are more closely related than distant ones. Real embeddings often have 256, 512 or 1024 dimensions. Sources: Google Machine Learning Crash Course: Embeddings; Google Machine Learning Crash Course: Embedding space and static embeddings.

Vector databases are a common building block of retrieval-augmented generation (RAG). A system splits documents into passages, creates embeddings and stores them. For a question, it retrieves the closest passages and passes them to a language model as source material. For website owners this suggests that individual sections of a page can be retrievable on their own (assessment).

Retrieval-augmented generation: retrieve first, then answerA question triggers a search in a search index or vector database. The best passages are added to the prompt together with the question. The language model writes the answer from them and points to the sources of the passages. The answer can only cite what was retrieved first.1 QUESTION2 RETRIEVAL3 CONTEXT4 ANSWERQuestionfrom a useror an appRetrievalsearch orvector search findsmatching passagesKnowledge sourcesearch index, web,vector databasePrompt with contextquestion + passages[1] passage · URL A[2] passage · URL B[3] passage · URL CLLMAnswertext withsource links[1][3]Citations point to retrieved passagesSchematic, based on Lewis et al. (2020) and Google’s grounding docs.
In retrieval-augmented generation, the system first retrieves matching passages and passes them to the language model as context. Google calls the link between model output and verifiable sources grounding. The answer can only cite what was retrieved first (an inference from the method, not a Google statement). Sources: Lewis et al. (2020): Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv 2005.11401); Google Cloud: Grounding overview (Vertex AI); Google Cloud: Grounding with Google Search (Vertex AI).

Example

A help center with 5,000 articles is split into passages and stored as vectors. The question “How do I change my shipping address?” also finds a section titled “Edit delivery details”, because both phrasings produce similar vectors. A plain keyword search could miss it.

FAQ

Is a vector database a SQL or NoSQL database? It is a separate category built for similarity search over vectors. Some SQL and NoSQL databases offer vector features, and dedicated products exist as well.

Does every AI application need one? No. Whether an existing database with vector support is enough depends on data volume and requirements.

Sources

  1. What is a vector database? - Google Cloudcloud.google.com
  2. What is a vector database? - lakeFS (secondary source)lakefs.io
  3. What is a vector embedding? - IBM Thinkibm.com
  4. What is vector search? - Elasticelastic.co

Last updated: