Skip to main content
This lesson walks through a complete Retrieval-Augmented Generation (RAG) workflow—from a user question to a grounded answer. Understanding each step and how they connect helps you design, debug, and optimize reliable RAG systems. We’ll cover:
  • Ingestion: turning source documents into vectors and storing them.
  • Query processing: embedding the user query, retrieving relevant chunks, and augmenting the prompt.
  • Generation: sending the augmented prompt to a foundation model to produce a grounded answer.
Example scenario: an organization has product documents for a device called the iUniverse Pro. The documents contain facts like battery life (18 hours), warranty (2 years), and Bluetooth support (5.3). When a user asks “How long does the iUniverse Pro battery last?” the RAG pipeline should return a grounded answer such as “18 hours.”
An infographic titled "Workflow: Example RAG Process Flow" showing a three-step process with numbered circular nodes connected to blue rounded content boxes. The boxes include a user question ("How long does the iUniverse Pro battery last?") and document details listing battery life (18 hours), warranty (2 years) and Bluetooth support (5.3).

​
Core concepts (quick)

  • Embeddings: numeric vectors that capture semantic meaning of text.
  • Vector store: an index/database that stores vectors + original text + metadata to enable fast semantic search.
  • Grounding: supplying retrieved source text to a foundation model so its answer is traceable to document facts.
  • Consistency: use the same embedding model (or compatible models) for ingestion and query-time embeddings.

​
Ingestion: documents → chunks → vectors → vector store

Ingestion prepares your proprietary content for retrieval:
  1. Chunking: split long documents into smaller, meaningful chunks (paragraphs or logical sections).
  2. Embedding: convert each chunk into a numeric embedding using an embedding model.
  3. Storage: save the embedding together with the original chunk text and metadata in a vector store (examples: S3 Vectors, Postgres with pgvector, OpenSearch).
Store both vector and original text so the vector enables semantic match and the text provides grounded context for the LLM. Example stored record (JSON):
Table — Common vector store options Choose an embedding model and remain consistent for ingestion and query so similarity comparisons are meaningful.

​
Query: user asks a question

When an app receives a user question, the RAG system must embed the question to perform semantic search against the stored vectors. Example user question:
  • “How long does the iUniverse Pro battery last?”
A workflow diagram titled "Workflow: Query Step 1 – User Sends Prompt" shows the question "How long does the iUniverse Pro battery last?" in a speech bubble alongside a small robot illustration. Below are five numbered steps describing the retrieval/embedding flow: chunking and generating embeddings, storing them, embedding the user query, retrieving similar chunks, and sending the augmented query to an LLM.

​
Embed the user query

Convert the user text into an embedding using the same model used during ingestion. For example, using an Amazon Titan embedding model:
Note: embedding the query is a lightweight operation and does not contact the generative foundation model (LLM) yet — it only prepares for semantic retrieval.

​
Similarity search in the vector store

Run a nearest-neighbor (similarity) search using the query embedding. The vector store returns the most semantically similar chunks. Importantly, what you receive back is the chunk text and metadata — not the vector — because the text will be used for prompt augmentation and grounding.
The image is a presentation slide titled "Workflow: Query Step 3 — Similarity Search in Vector Store" showing a diagram of a RAG/vector-search pipeline. It highlights chunking and embedding, finding the most similar chunk, and returning that chunk, with five numbered boxes summarizing the full embedding-and-retrieval steps.
At this point the RAG pipeline has:
  • The original user question.
  • One or more retrieved document chunks (plain text) that are semantically relevant to the question.

​
Prompt augmentation and generation

Combine the retrieved context with the user question to create an augmented prompt for the foundation model. This structure gives the model explicit facts it can cite. Augmented prompt structure (recommended order):
  • System/instruction layer (optional): desired response format, safety filters, style constraints.
  • Retrieved context: one or more relevant chunks from the vector store, with citations/metadata.
  • User question: the original prompt to answer.
An infographic titled "Workflow: Query Step 4 – Prompt Augmentation" showing the question "How long does the iUniverse Pro battery last?" being combined with product chunks and sent into a RAG system. Below are five numbered steps: generate embeddings, store them, embed the user query, retrieve similar chunks, and send the augmented query to an LLM.
Send the augmented prompt to your chosen foundation model (for example, an Amazon Bedrock model such as a Meta Llama variant). The model reads the provided context and generates a response that is grounded in the supplied document text — reducing hallucinations and improving traceability.
A presentation slide titled "Workflow: Query Step 5 – Foundation Model Generate Answer" showing three dark rounded panels numbered 01–03 with icons (book/magnifier, battery, checkmark) and short explanatory text about reading context, the iUniverse Pro's 18‑hour battery, and the answer being grounded. The slide is branded with a © Copyright KodeKloud note.
Because the model has the retrieved chunks in the prompt, the response can be phrased confidently and traced back to the provided documents — this is what we mean by grounding.

​
Putting the full pipeline together

High-level flow:
  • App receives a user prompt → passes it to the RAG pipeline.
  • (Preceding ingestion step): documents were chunked, embedded, and stored in a vector store.
  • Query-time operations: embed the user query → similarity search → retrieve text chunks → augment the prompt → send to the foundation model → generate a grounded answer.
A slide-sized workflow diagram titled "Workflow: App Actions vs Bedrock Actions" showing a RAG pipeline with boxes for App, RAG System, embedding model, Vector Store, Ingestion, and a chosen model, connected by arrows for prompts, vectors, and grounded responses. The diagram visualizes how documents are ingested, chunked into embeddings, searched, and used to produce grounded answers.
Quick checklist for production RAG systems
  • Use consistent embedding models (or compatible dimensionality/semantics) across ingestion and query.
  • Store text + metadata with vectors for traceability.
  • Limit retrieved context size to fit model input tokens; prefer high-relevance chunks.
  • Add system prompts to constrain format and enforce safety.
  • Track provenance (source file, chunk id, timestamp) for every returned chunk.
Key takeaway: RAG has two distinct phases — ingestion (embed and store document chunks) and query (embed query, retrieve similar chunks, augment the prompt, and generate a grounded answer). Ensure you use a consistent embedding approach between ingestion and query for effective retrieval and traceable responses.
This RAG explanation applies directly to Amazon Bedrock and Bedrock Knowledge Bases and can be adapted to other embedding models, vector stores, and foundation models.

Watch Video