Skip to main content
In this lesson you’ll learn how to use foundation models (via Amazon Bedrock or similar services) to answer questions about your own data using natural language. We’ll cover:
  • The problem: finding precise answers across many documents.
  • The solution: context‑augmented question answering and retrieval‑augmented generation (RAG).
  • Prompt design patterns and examples for reliable, grounded answers.
  • Best practices and expected outcomes.

​
Problem statement

Organizations accumulate large volumes of content—documentation, policies, knowledge bases, and reports. The core challenge is locating the single correct answer quickly when relevant facts are spread across multiple sources. A practical solution is to use Amazon Bedrock with a chosen foundation model to answer questions that depend on your internal content. When you ask a model a question it will typically respond in one of two ways:
  • From pretrained knowledge (general world knowledge).
  • From context you supply (documents or snippets attached to the prompt).
If you don’t supply relevant context, the model may hallucinate or return outdated/generalized answers because it lacks access to your internal policies. If you include the relevant documents as context, the model can produce accurate, grounded responses. Consider this example: “How many days of PTO does a level 4 employee get?” Without your HR policy text, the model will likely guess. With the HR policy included as prompt context, the model can answer precisely.
An infographic titled "Solution: Question Answering" showing a prompt and an HR policy document as model inputs feeding into a "Bedrock Foundation Model." The model output box states: "A level 4 gets 15 days PTO per calendar year."

​
Open vs. context‑based question answering

  • Open Q&A: The model answers from its pretrained knowledge (what it learned during training). Useful for general questions, but unreliable for internal facts.
  • Context‑based Q&A: You provide documents or snippets and instruct the model to answer using only that content. This grounds the response in your sources and greatly reduces hallucination.
Context‑based Q&A is a fundamental building block for document search systems and knowledge assistants. It is commonly combined with retrieval‑augmented generation (RAG), where a semantic search step fetches the most relevant documents before the model generates an answer.
A side-by-side diagram comparing Open Question Answering (LLMs answering from training data/general knowledge) with Context-Based Question Answering (LLMs that use retrieved documents and a user question to produce answers grounded in source content). A banner below highlights Retrieval-Augmented Generation (RAG) and components like document search, knowledge assistants, and RAG systems.

​
Prompt pattern and structure

Use a simple, repeatable pattern when asking a model to answer from a provided document:
Key elements to include:
  • A clearly labeled question.
  • Explicit instruction to use the provided input (grounding).
  • A strict output format (for example: “one short sentence”).
  • A scope limitation so the model does not introduce external knowledge.
An infographic titled "Workflow: Prompt Design for Q&A" showing a laptop screen with an example prompt on it. To the left are four numbered tips: Be specific; Provide context if needed; Specify answer format; Limit response scope.

​
Prompt examples: poor vs. better

Poor prompts often mix multiple questions, leave the desired format unspecified, and do not provide grounding:
  • Poor: “Tell me about AWS security and encryption, how it works, and what services use that.”
    • Problems: multiple questions in one prompt, unclear scope, no output format.
Focused prompts work much better:
  • Better: “Explain how KMS is used to encrypt data in Amazon S3 in three bullets.”
    • Benefits: single focused question, explicit format, scoped task.
You can use a short checklist when converting a loose question into a production prompt:
An infographic titled "Workflow: Avoiding Common Mistakes" that compares a "Poor Prompt" and a "Better Prompt" side‑by‑side, using AWS encryption examples. The left side lists issues like multiple questions and no context, while the right shows a focused single question asking about AWS KMS and S3.

​
Best practices for reliable Q&A

  • Ask one question at a time; decompose complex requests into sub‑questions.
  • Define scope clearly (timeframe, role, product, or document sections).
  • Specify the desired output format (one sentence, numbered steps, 3 bullets).
  • Provide grounding documents or passages. If the model must rely solely on those documents, state that explicitly.
  • Add an explicit fallback: instruct the model to answer “I don’t know” or “Not found in the provided documents” when the information is not present.
  • Monitor and log model responses, confidence signals, and the source document used for auditing.
An infographic titled "Workflow: Avoiding Common Mistakes" that lists best practices for prompting. It shows four colored tip boxes — one question at a time, define the scope, specify response format, and provide context (grounding).
Include an explicit fallback instruction in your prompt such as: If the answer is not present in the provided documents, reply “Not found in the provided documents.” This simple rule significantly reduces hallucinations and makes behavior predictable for downstream systems.
Do not assume model outputs are authoritative without source attribution. Always surface the document or snippet used for the answer and validate high‑impact responses with a human reviewer.

​
Quick prompt checklist

  • Single, clearly labeled question.
  • Explicit grounding instruction.
  • Strict output format.
  • Explicit fallback for missing information.
  • (Optional) Provide citation format if you want the model to reference document IDs or line numbers.

​
Expected outcomes

Implementing context‑based Q&A will:
  • Reduce time to answer by surfacing precise facts from your documents.
  • Improve discoverability of internal knowledge.
  • Minimize hallucination when prompts are grounded and include a fallback.
  • Enable knowledge assistants and RAG systems to provide reliable, auditable responses.

​
Key takeaway

Clear, focused, single‑question prompts that are explicitly grounded in the right source documents produce the most accurate answers.
A presentation slide titled "Key Takeaway" with a blue badge labeled "01" and the text "Clear, focused questions produce more accurate answers." The left side is a dark blue panel and the rest of the slide is light gray.
This concludes the lesson on implementing question‑and‑answer prompts. Related topics include retrieval pipelines and sentiment analysis using foundation models in Amazon Bedrock. Links and references

Watch Video