Skip to main content
In this lesson we’ll learn how to understand and optimize costs for applications built on Amazon Bedrock. Organizations often ask: why is my Bedrock usage costing more than expected? As usage scales, costs can grow quickly and sometimes unpredictably. This lesson explains common cost drivers, how to gain visibility using monitoring and cost tools, a structured troubleshooting workflow, and practical optimizations you can apply. What you’ll learn
  • Common cost drivers for GenAI applications on Bedrock.
  • How to combine monitoring (CloudWatch) and cost tools (Cost Explorer, Budgets) to find root causes.
  • A step-by-step troubleshooting workflow.
  • Practical optimizations and expected impact.
Core problem Teams frequently develop against Bedrock and enjoy the model outputs during testing. When the app reaches production scale, costs can spike unexpectedly. Understanding the specific drivers and where to look for evidence is the first step to reducing spend.
The slide titled "Problem: GenAI costs can grow quickly and unpredictably" shows four circular icons with captions explaining cost drivers: large prompts increase token usage, more powerful models cost more per request, high request volume multiplies cost, and no visibility into where cost is coming from.
Key cost drivers
  • Large input prompts increase token usage and directly raise costs.
  • More powerful (larger) models cost more per request or per token.
  • High request volume multiplies per-request charges.
  • Lack of instrumentation and visibility makes it difficult to identify where spend is coming from.
Quick diagnostic questions
  • Are unusually large prompts coming from a particular part of the app (e.g., ingestion pipelines, interactive UI, background jobs)?
  • Which models are being targeted for those prompts?
  • Are large prompts being routed to multi-billion-parameter models unnecessarily?
If large prompts are sent to a high-capacity model, per-request cost can be significant. Shrinking prompts or routing tasks to smaller, task-appropriate models often yields immediate savings. Gain visibility: monitoring + cost tools Combine runtime telemetry with billing tools to map technical signals to dollars.
  • Emit metrics from your application that capture model usage: token counts (input/output), model name/version, and invocation counts, plus latency and error rates.
  • Visualize these metrics in Amazon CloudWatch dashboards and create alarms for anomalies or thresholds.
  • Centralize application logs and model-invocation details in CloudWatch Logs to analyze request patterns and trace sources of large prompts or volume spikes.
  • Use AWS Cost Explorer and AWS Budgets to view and alert on spend by service, account, tags, and usage type.
Best practice: correlate CloudWatch metrics/logs with Cost Explorer data to attribute spend to specific services, teams, or features. This helps prioritize optimization work. How to instrument and visualize
  • Metrics: emit tokens_in, tokens_out, model_invocations, and latency_ms with relevant dimensions (model_name, service, environment, feature_flag).
  • Dashboards: create views that show token usage by model, invocation rate, and cost by service over time.
  • Alarms: add alerts for sudden token spikes, unexpected model usage, or cost thresholds.
A presentation slide titled "Solution: Use AWS monitoring and cost tools together" showing four panels: CloudWatch Metrics, CloudWatch Logs, Cost Explorer, and "Combine the insights." Below are green boxes listing benefits like monitoring usage/latency, understanding request patterns, and tracking AWS spend over time.
Structured approach to cost management Follow a repeatable workflow when investigating cost surprises:
  1. Identify likely cost drivers (large prompts, model choice, request volume, lack of caching).
  2. Look for supporting evidence in CloudWatch Metrics (token counts, invocation spikes, latency).
  3. Analyze CloudWatch Logs to find where requests originate and to inspect payload sizes.
  4. Consult Cost Explorer and Budgets to map usage to dollars and to detect daily/weekly trends.
  5. Implement optimizations and re-measure to validate impact.
A presentation slide titled "Workflow: Structured Approach to Cost Management" showing a five-step process with icons. The steps are: 01 Identify Cost Drivers, 02 CloudWatch Metrics, 03 Analyze Logs, 04 Cost Explorer, and 05 Optimize.
Practical optimization techniques
  • Model selection
    • Use smaller, more efficient models for routine tasks (formatting, simple summarization, classification).
    • Reserve larger models for tasks that truly require higher reasoning or creativity.
    • Route requests dynamically: choose model by intent or task complexity.
  • Prompt optimization
    • Make prompts concise to reduce input tokens.
    • Provide precise instructions and enforce output schemas (e.g., JSON-only responses) to control verbosity.
    • Set a maximum output token limit where appropriate.
  • Call reduction
    • Avoid duplicate or unnecessary repeated calls; batch where possible.
    • Debounce UI-driven calls to reduce accidental bursts.
  • Caching and alternative architectures
    • Cache identical or similar responses at the application level to avoid repeated model invocations.
    • Use RAG (retrieval-augmented generation) with a vector DB: store large documents in the vector store and include only relevant chunks via similarity search to keep prompts small.
Bedrock does not deduplicate or cache responses for you. If your application has repeated identical queries, consider a caching layer (Redis/Memcached via Amazon ElastiCache or another store) to store and serve repeat responses within an acceptable TTL.
Caching options and RAG workflow examples
  • Application caching (Redis / Memcached)
    • Workflow: check cache → serve if valid → otherwise call Bedrock → store result in cache.
    • Useful for idempotent queries and expensive repeated computations.
  • Retrieval-augmented generation (RAG) with a vector database
    • Store documents or long contexts as embeddings.
    • For each query, perform a similarity search to fetch the most relevant chunks and send only those chunks as prompt context.
    • This reduces prompt size and keeps relevant information available to the model without sending entire documents.
AWS cost visibility tools The primary AWS tools for cost analysis are Cost Explorer, AWS Budgets, and CloudWatch. Use them together:
  • CloudWatch: technical telemetry (tokens, invocations, latency).
  • Cost Explorer: spend visualizations and grouping by service, region, tag, and usage type.
  • AWS Budgets: create alerts for specific spend thresholds or forecasted overages.
A presentation slide titled "Workflow: AWS Tools for Cost Visibility" showing three circular icons labeled "AWS Cost Explorer," "AWS Budgets," and "CloudWatch." Each icon has a simple white line graphic inside a teal-rimmed circle on a dark blue background.
Using Cost Explorer effectively
  • Location: AWS Console → Billing and Cost Management → Cost Explorer.
  • Capabilities:
    • Visualize spend as stacked bar charts, line graphs, or scatter plots.
    • Set date ranges and granularity (hourly, daily, monthly).
    • Group and filter by service, Region, linked accounts, tags, and usage type.
Example: a daily view can reveal which service contributes to spikes and whether Bedrock or related services (e.g., compute, networking) are responsible for the increase.
Screenshot of the AWS Cost Explorer showing a stacked bar chart of daily service costs for March, with a total cost of 27.87 and average daily cost 2.32. The left pane shows billing navigation and the right pane displays report parameters.
Cost Explorer tips
  • Adjust granularity to reveal hourly bursts versus long-term trends.
  • Use grouping and tagging to attribute cost to teams, features, or environments.
  • Exclude credits or specific accounts if you need to model post-credit spend.
  • Leverage forecasting and AWS Cost Anomaly Detection to catch unexpected changes earlier.
Summary — put it all together
  • Instrument your app to emit token counts, model invocations, and latency to CloudWatch.
  • Use CloudWatch Logs to trace request patterns and identify sources of large prompts or high-volume usage.
  • Use AWS Cost Explorer and AWS Budgets to translate those technical signals into dollars and monitor spend over time.
  • Apply targeted optimizations—model selection, prompt tuning, caching, RAG—and measure the impact iteratively.
Next steps Future lessons will provide concrete CloudWatch dashboard examples, sample alarms to surface token and model usage, and cost-reduction case studies that show measured savings from model routing, prompt engineering, and caching.

Watch Video