> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Effective Chunking Strategy for RAG Systems
> Evaluating chunking strategies for RAG systems and recommending semantic unit splitting with overlap to preserve context and maximize retrieval relevance.
In this lesson we evaluate common chunking strategies used in Retrieval-Augmented Generation (RAG) systems and recommend the most effective approach for preserving context while maximizing retrieval relevance. We'll compare popular methods, explain their trade-offs, and provide practical, model-aware recommendations you can apply to real-world pipelines.
## Common chunking strategies
* Splitting text by fixed character count
* Splitting text by fixed token count
* Splitting text by semantic units with overlap (recommended)
* Random chunking to increase diversity
## Recommended approach: semantic units with overlap
Splitting by semantic units (sentences, paragraphs, or logical sections) and adding controlled overlap preserves natural context and meaning. Chunks aligned with semantic boundaries are more coherent for both embedding models and LLM retrieval. Overlap helps prevent information loss when a concept crosses a chunk boundary and improves recall during retrieval.
## Why other strategies fall short
| Strategy | When it fails | Key issues |
| ------------------------- | -----------------------------------------------: | ------------------------------------------------------------------------------------------------ |
| Fixed-character splitting | Any natural language text | Cuts words/sentences arbitrarily; produces incoherent chunks that reduce embedding quality |
| Fixed-token splitting | Better than fixed-character, but still imperfect | Requires exact tokenizer match; can split sentences/logical units in unnatural places |
| Random chunking | For primary retrieval indices | Increases diversity but breaks coherence; lowers precision — can be useful only for augmentation |
### Additional notes on token-based splitting
* Token-aware splitting is preferable to raw character counts because embeddings and LLMs operate on tokens. However, tokenization must match the embedding/LLM tokenizer to avoid mismatches.
* Use the embedding model’s tokenizer (for example, `tiktoken`) to compute token lengths before embedding or truncation.
## Practical recommendations
* Chunk unit
* Prefer semantic boundaries: sentences, paragraphs, or logical sections.
* Preserve special content (code blocks, tables, lists) as single chunks when possible to keep structure intact.
* Chunk size
* Tune to your embedding model and LLM context window. A common target is a few hundred tokens per chunk (e.g., 200–500 tokens), adjusted for your model and document types.
* Overlap
* Use overlap to cover boundary-spanning concepts. Typical overlap options:
* Sentence-based: 1–3 sentences
* Proportional: 10–20% of the chunk
* Token-based: 50–100 tokens as a pragmatic starting point
* Token accounting
* Use the embedding model’s tokenizer to measure tokens. Example: [tiktoken on GitHub](https://github.com/openai/tiktoken).
* Metadata
* Store source identifiers, position offsets, and structural metadata so retrieved chunks can be traced and reassembled.
* Deduplication and normalization
* Remove near-duplicates to avoid redundant retrievals, but keep intended overlapping chunks since overlaps are deliberate for context continuity.
### Quick reference: chunking guidelines by document type
| Document type | Suggested chunk unit | Chunk size (tokens) | Typical overlap |
| ----------------------------- | ------------------------------------ | ---------------------------------: | ------------------------------------: |
| Short articles / blog posts | Paragraphs or sentence groups | 150–300 | 10–20% or 1–2 sentences |
| Long technical docs / manuals | Sections → paragraphs (hierarchical) | 300–600 per paragraph chunk | 10–20% or 50–100 tokens |
| Code-heavy documents | Keep code blocks whole | Varies; ensure block fits in chunk | Minimal; avoid splitting code blocks |
| Streaming/real-time | Sliding window with semantic fence | Small (100–300) | Sliding overlap (e.g., 50–100 tokens) |
## Example: sentence-based chunking with overlap (Python)
Below is a simple, production-friendly function that splits a list of sentences into overlapping chunks. The function uses sentence counts for chunk size and overlap, which is easy to reason about and map back to semantic boundaries. Before embedding, convert chunk text to tokens using your model tokenizer to ensure chunk token budgets are respected.
```python theme={null}
from typing import List
def chunk_sentences(sentences: List[str], chunk_size: int, overlap: int) -> List[str]:
"""
Split a list of sentences into overlapping chunks.
- chunk_size and overlap are in number of sentences.
- Returns a list of chunk strings where sentences are joined with a space.
"""
if chunk_size <= 0:
raise ValueError("chunk_size must be > 0")
if overlap < 0:
raise ValueError("overlap must be >= 0")
chunks: List[str] = []
i = 0
step = max(1, chunk_size - overlap)
while i < len(sentences):
chunk = sentences[i : i + chunk_size]
chunks.append(" ".join(chunk))
if i + chunk_size >= len(sentences):
break
i += step
return chunks
# Example usage:
sentences = [
"First sentence.", "Second sentence.", "Third sentence.",
"Fourth sentence.", "Fifth sentence."
]
chunks = chunk_sentences(sentences, chunk_size=2, overlap=1)
# chunks -> [
# "First sentence. Second sentence.",
# "Second sentence. Third sentence.",
# "Third sentence. Fourth sentence.",
# "Fourth sentence. Fifth sentence."
# ]
```
## When to consider alternatives
* Very short documents: Semantic chunking still applies; overlap may be unnecessary.
* Extremely long documents: Use hierarchical chunking—first split into sections, then paragraphs with overlap—to preserve high-level structure.
* Real-time or streaming ingestion: Use sliding windows or rolling buffers. Aim to respect semantic boundaries (e.g., sentence/paragraph fences) when possible to retain coherence.
## Validation and tuning
* Measure token counts with the embedding tokenizer to ensure chunks fit model limits.
* Sample retrieval results and run relevance/recall evaluations to confirm that chosen chunk size and overlap improve retrieval quality.
* Iterate: different corpora (legal text, scientific papers, code) will require different heuristics.
When implementing chunking, always validate the chunks by checking token counts with the embedding model’s tokenizer and by sampling search results to ensure retrieval quality improves with your chosen chunk size and overlap.
## References and further reading
* tiktoken tokenizer: [https://github.com/openai/tiktoken](https://github.com/openai/tiktoken)
* Retrieval-Augmented Generation concepts: search for RAG architectures and embedding-based retrieval papers and blog posts for deeper guidance.