Chunking Strategies
Choose what to cut on, size chunks in the embedder's own tokens, keep headings, tables and code whole, search small but return big, and measure any change with recall@k on real questions.
Last updated
After this section you can
- Choose what to cut on from the document’s structure, and say what each kind of cut breaks on
- Fit heading path, overlap and text inside the embedder’s window, and return a parent when a child matches
- Compare two chunkers by recall@k on real questions and tell a gain from noise
Chunking: Cut on Structure, Size for the Embedder
A chunk is the smallest piece search can return. Cut where the document’s own structure ends an idea, fit each chunk to the embedder’s window, and measure every change on real questions.
The vector from Embeddings stands for whatever lands in one chunk, so the cut decides what search can find. Cut on the document’s own units, keep the heading path (the section titles above a chunk), overlap and text inside the embedder’s window, and choose between chunkers by how often the right chunk comes back (recall@k) on real questions, not by a library default.
Acme’s refund-policy.pdf, page 4; the size cut is a toy (a recursive splitter at 150 characters). Question: “How long do I have to ask for a refund on an annual plan?” The answer, 60 days, is in chunk 13. On a phone the figure pans sideways.