Skip to main content
In this lesson we continue from the document loader step where we loaded handbook.pdf. The goal is to split the document into smaller, overlapping chunks (passages) so they can be embedded and stored in a vector store for retrieval-augmented generation (RAG). Proper chunking preserves context, improves retrieval relevance, and improves the quality of LLM responses when using external documents.

​
1. Load the PDF and inspect pages

Use LangChain’s PDF loader to read the document as page-level Document objects:
This gives you page-level Document objects with page_content and default metadata (e.g., source, page).

​
2. Create a text splitter to produce chunks

We commonly use RecursiveCharacterTextSplitter for robust splitting that respects sentence boundaries and reduces awkward cuts. Here we create a splitter configured with a chunk size of 200 characters and 50 characters of overlap between consecutive chunks:
Why use a splitter:
  • It transforms long pages into passage-sized Documents suitable for embeddings and vector stores.
  • It can preserve natural language boundaries (sentences, paragraphs) when configured properly.

​
3. Why chunk with overlap?

Overlap between chunks preserves context across chunk boundaries so that ideas cut near a boundary remain recoverable. Overlap reduces the chance of losing relevant context during retrieval and increases the probability that a retrieved chunk contains a complete thought. Benefits:
  • Better context for LLM prompts when a single chunk lacks all required information.
  • Smoother transitions across document segments for multi-sentence ideas.
  • Tolerates noisy splits and improves recall at retrieval time.
Tradeoffs:
  • Introduces redundancy (more tokens to embed/store).
  • Larger total storage and slightly higher retrieval cost.
Tip: If queries require longer context, increase chunk_size. For precise answers on short queries, reduce chunk_size but keep a modest chunk_overlap (e.g., 25–50 characters).

​
4. Inspect the resulting chunks and metadata

After splitting, inspect chunk count and a sample chunk to confirm expected behavior:
Each chunk is a Document-like object containing page_content and metadata (for example, source and page). Example representation:
Notes on metadata:
  • By default, PDF splitting includes source and page metadata. This is useful for citing sources in model responses.
  • When processing multiple documents, robust metadata (filename, URL, document ID) helps trace every chunk to its origin.

​
5. Choosing chunk size and overlap (practical guidance)

Choose chunk_size and chunk_overlap based on document characteristics, retrieval goals, and the LLM context window. The table below summarizes common scenarios: Factors to consider:
  • Document structure (long paragraphs vs short bullets).
  • Expected query type (fact lookup vs long-answer generation).
  • Vector store and embedding cost/performance.
  • LLM prompt budget (context window size).

​
6. Summary: chunking workflow

  1. Load the document into page-level Documents (e.g., with PyPDFLoader).
  2. Configure RecursiveCharacterTextSplitter with appropriate chunk_size and chunk_overlap.
  3. Call split_documents(pages) to produce a list of chunk Document objects.
  4. Each chunk contains page_content and metadata and can be embedded and stored in a vector store for semantic search and RAG.
After chunking, the typical next steps are:
  • Create embeddings for each chunk (e.g., OpenAI embeddings, other models).
  • Store embeddings in a vector store (FAISS, Pinecone, Milvus, etc.).
  • Implement semantic search and retrieve relevant chunks for LLM prompts.
Useful references: A solid understanding of chunking helps ensure your RAG pipeline retrieves coherent, contextually complete passages and produces higher-quality responses from LLMs.

Watch Video