Skip to main content
All right — imagine you just heard your new favorite video game is coming out, so you went out and bought the latest graphics card. Before you start taking things apart, you ask an AI assistant: “Will this new graphics card fit in my case?” That looks like a simple yes/no question, but the model may not know about this exact card (maybe it was released after the model’s training cutoff). The model understands the question, but it might lack the precise measurements needed to answer “yes” or “no.” Those measurements are in product documentation: card specifications list length, and case specifications list maximum GPU clearance. The wording in your question (“Will this fit?”) and the wording in the docs (“maximum GPU clearance,” “maximum length”) can differ, and the relevant information may be buried in long PDFs or support pages. Retrieval is the piece that finds that relevant documentation before a model composes the response. This lesson focuses on that retrieval side: how we convert documents and queries into embeddings, index them, and retrieve the most relevant passages using vector search. If you’ve worked with modern AI systems, you’ve probably heard “RAG.” RAG stands for Retrieval-Augmented Generation and describes two connected tasks:
  • Retrieval: the application finds relevant information (from documents, databases, or APIs) related to the user’s query.
  • Generation: a language model uses that retrieved information to compose an answer.
This lesson stays entirely on the retrieval component; a later lesson will show how retrieved passages are combined with a language model to generate final responses.
This lesson is about the retrieval component only — how we turn documents and queries into embeddings, index them, and retrieve the most relevant passages. Generation (using the retrieved passages with an LLM) is covered in a later lesson.
A few important notes before we dive in:
  • Not every RAG system requires vector search. Some systems use keyword search, SQL queries, or direct API lookups.
  • Vector search is popular because it finds semantic matches even when wording differs (e.g., “Will this fit?” vs. “maximum GPU length”).
  • You do not manually compute embedding numbers — an embedding model produces the vectors for you.
High-level retrieval flow
  1. Source documents
    • Gather the knowledge store: product manuals, spec sheets, internal docs, support articles, knowledge bases, and FAQs.
    • These documents are the corpus your retrieval system will search.
  2. Chunking
    • Split documents into smaller passages or “chunks”: paragraphs, table rows, short spec sections, or any logical unit that fits the retrieval strategy.
    • Keep metadata with each chunk (file name, section id, URL, timestamps) so the system can return the original source when a match is found.
  3. Embeddings
    • Convert each chunk to a numeric representation called an embedding (a vector) using an embedding model.
    • Embeddings are high-dimensional encodings that let us measure semantic similarity mathematically: related chunks (for example, “maximum GPU length: 350 mm” and “graphics card clearance”) will lie close to one another in embedding space.
  4. Vector database / index
    • Store chunk embeddings and their metadata in a vector database or vector index.
    • Vector DBs provide fast similarity search (nearest-neighbor or ANN algorithms) across stored embeddings.
  5. Query as embedding
    • Convert the user query into an embedding using the same embedding model (or a model that maps into the same vector space).
    • The vector DB finds stored embeddings nearest to the query embedding using a similarity metric such as cosine similarity or dot product.
  6. Retrieve passages
    • The search returns the original passages (chunks) and their metadata for the closest embeddings — e.g., a line in the case spec that states “supports GPUs up to 355 mm.”
    • These retrieved passages are the inputs used in the next phase (generation), where they are combined with the user query and application instructions for a language model to produce the final answer.
Quick reference table: components and purpose Why this helps
  • Semantic matching: Embeddings let you match semantically related text even when the exact words differ.
  • Source fidelity: The system returns the original human-readable passages (with metadata), so downstream components (or humans) can verify sources rather than relying solely on model-generated statements.
  • Flexible retrieval: Vector search works well for fuzzy or paraphrased queries where keyword search would fail.
A few clarifications and common questions
  • Embedding dimensionality: Visual diagrams often use 2D plots for intuition, but real embeddings live in high-dimensional spaces (hundreds to thousands of dimensions). The geometric intuition still applies: related texts are nearby.
  • What the vector DB returns: It returns the original chunk(s) and metadata — the vectors are used to rank and find matches, but the application presents the retrieved text.
  • Alternatives: Keyword search and structured queries are still valid and sometimes preferable (e.g., exact-match lookups). Vector search excels when you need semantic matching across different phrasings.
  • Performance considerations: Production systems commonly use ANN (approximate nearest neighbor) indexes for speed and to scale to millions of chunks.
Recap of the path
  • Documents → chunked passages → embeddings → stored as searchable points in a vector DB (with references to original text).
  • The user question becomes a query embedding occupying a point in the same vector space and retrieves nearby chunks.
  • This lesson ends here. The next lesson shows how to combine the retrieved passage(s) with the user query and application instructions, sending that combined input to a language model — turning retrieval into retrieval-augmented generation.
Links and references

Watch Video