Deutsch
Retrieval & grounding

Chunking

Also known as: text chunking, semantic chunking

Chunking, in the context of AI search, means splitting long documents into smaller passages, called chunks, that a system stores, searches and passes to a language model one at a time. The learning technique of the same name from psychology is a different topic.

How does chunking work?

Applications using retrieval-augmented generation (RAG) rarely process a whole document at once. They split it into chunks, create an embedding for each and store them in a vector database. For a question, the system retrieves the best matching chunks and gives them to the model as source material.

According to reports from vendors such as Pinecone and MongoDB, these approaches are common:

  • Fixed size: splits by token count. Simple, but can cut through a sentence.
  • Sentence or paragraph based: respects natural breaks in the text.
  • Recursive: splits step by step at headings, paragraphs and sentences.
  • Document structure based: follows formats such as HTML or Markdown.
  • Semantic: uses embeddings to detect where the topic shifts. It needs more computation.

Chunks that are too small lose context, chunks that are too large add noise and cost. According to reports, overlap between neighboring chunks keeps information from being lost at the split points. Chunk size and overlap are settings of each system, and website owners cannot influence them.

Retrieval-augmented generation: retrieve first, then answerA question triggers a search in a search index or vector database. The best passages are added to the prompt together with the question. The language model writes the answer from them and points to the sources of the passages. The answer can only cite what was retrieved first.1 QUESTION2 RETRIEVAL3 CONTEXT4 ANSWERQuestionfrom a useror an appRetrievalsearch orvector search findsmatching passagesKnowledge sourcesearch index, web,vector databasePrompt with contextquestion + passages[1] passage · URL A[2] passage · URL B[3] passage · URL CLLMAnswertext withsource links[1][3]Citations point to retrieved passagesSchematic, based on Lewis et al. (2020) and Google’s grounding docs.
In retrieval-augmented generation, the system first retrieves matching passages and passes them to the language model as context. Google calls the link between model output and verifiable sources grounding. The answer can only cite what was retrieved first (an inference from the method, not a Google statement). Sources: Lewis et al. (2020): Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv 2005.11401); Google Cloud: Grounding overview (Vertex AI); Google Cloud: Grounding with Google Search (Vertex AI).

Semantic chunking

Semantic chunking splits a text where the topic changes instead of after a fixed length. The system compares embeddings of neighboring sentences and starts a new chunk where they differ noticeably. This takes more computation than fixed-size splitting. The example below shows the difference.

What does it mean for SEO?

Google’s Search Central documentation says there are no additional requirements to appear in AI Overviews or AI Mode. A page needs to be indexed and eligible to show with a snippet. Vendors of AI systems do not disclose how they split pages. It is plausible that pages with clear headings divide into useful sections more easily (assumption). There is no evidence of a ranking effect.

Example

A “Shipping costs” page covers domestic, EU and return shipping in one long block. A system with fixed chunk size may cut through the EU table. A semantic approach splits between the three topics, so the question “What does shipping to Austria cost?” retrieves exactly the EU section.

FAQ

What is chunking in simple terms? Cutting a long text into smaller pieces so that a system can find and use the relevant piece instead of the whole document.

Do I need to write differently for chunking? Google asks for no special optimization for its AI features. Clear headings and self-contained paragraphs help readers in any case.

Sources

  1. Chunking strategies - Pinecone Learning Center (secondary source)pinecone.io
  2. Chunking explained - MongoDB (secondary source)mongodb.com
  3. AI features and your website - Google Search Centraldevelopers.google.com

Last updated: