Skip to main content
Previously we stopped after retrieval located the relevant specifications. At that point we had the graphics card length and the case’s maximum supported GPU length (pulled from the manufacturer’s documentation). However, the language model has not yet seen those measurements — the vector database helps find text, but the model only answers from text it is given. The application must take what retrieval found, combine it with the user’s question, and present that assembled context to the LLM. This article explains that second half of the RAG flow. RAG (retrieval-augmented generation) = Retrieval (R) + Augmentation + Generation (G)
  • Retrieval locates passages likely to answer the question.
  • The application augments the model’s input with those retrieved passages.
  • The model generates the final answer based on that augmented context.
  • The model’s trained weights remain unchanged — we’re only giving it relevant context for this request.
What the application has at this point
  • The original user question (shortened): “Will it fit?” — referencing the graphics card and case.
  • Retrieved passages from product documentation, e.g.:
    • Graphics card length: 336 mm
    • Case maximum GPU length: 360 mm
These retrieved passages are human-readable text extracted from source documents. The vector index helped find them, but the model receives the text itself — not the vectors.
Use retrieval to include only concise, relevant passages in the model context. This reduces token usage, avoids distracting information, and helps the LLM focus on the facts that matter.
Assembling the model input The application combines a small set of distinct pieces into a single context that the model will read. A typical layout (and why each piece matters): These roles can be represented as separate messages (system/user/tool) or assembled into a single combined prompt depending on the API. The LLM ultimately receives them as one temporary context. Why retrieval matters here
  • Instead of attaching an entire manual, retrieval selects only the passages likely to answer the question.
  • This conserves tokens, reduces noise, and focuses the model’s reasoning on the relevant facts.
  • Different retrieval results on subsequent requests enable the same model to answer questions about content it did not contain during training.
Sending the assembled input to the model Once the application builds the context (instructions + user question + retrieved specs), it sends that input to the LLM. The model reads:
  1. the instructions (what to do),
  2. the user’s question (why we’re doing it), and
  3. the retrieved passages (what facts to use).
    The model then generates a response based solely on that temporary context.
Important notes on correctness and behavior
  • The model itself is unchanged; retrieved specifications are transient context for a single request.
  • Each request can retrieve different documents and build a new input for the same model.
  • The accuracy of the answer depends on what was retrieved and how clearly the application assembled the input. Missing, incorrect, or ambiguous specs can lead to incorrect answers.
Always validate retrieved source passages for accuracy and recency. The LLM will confidently use whatever text you provide, even if the passage is incomplete or outdated.
Example: fit check Given the retrieved specs:
  • Graphics card length: 336 mm
  • Case maximum GPU length: 360 mm
The application sends those passages along with instructions and the user’s question. The model can perform the comparison and respond: This completes the end-to-end RAG workflow:
  1. Retrieval finds useful specs.
  2. The application assembles instructions, the user’s question, and the retrieved passages.
  3. The LLM receives that combined context and generates the answer.
Each request can supply fresh, relevant source information so the RAG system can answer questions about facts the model did not contain in its training data.
A presenter stands beside a glowing lightboard covered in neon-colored handwritten diagrams. The board shows boxes labeled "Will it fit?", "Model Input" (with instructions, question, specs), "LLM", and a calculation "336 < 360" with a "YES" result.

Watch Video