Using Amazon CloudWatch With Bedrock Part 3

CloudWatch Logs includes a built-in query engine called Logs Insights (also known as Log Analytics) that lets you interrogate and extract structured information from raw log events using an SQL-like query language. Logs Insights makes it easy to locate timestamps, event types, model identifiers, latency metrics, and other fields across large volumes of Bedrock invocation logs—much faster and more precise than manually scanning raw log lines.

Logs Insights uses an SQL-like syntax but is not full SQL. To write effective queries, know your log schema (the names and types of fields your application and Bedrock emit).

Common Logs Insights queries for Bedrock model invocations

Below are practical queries and patterns to analyze Bedrock invocation logs (for example, the log group that receives Bedrock model invocation events).

Basic extraction of timestamp, event type, and model ID (sorted by timestamp desc, limited to 10,000 results):

fields @timestamp, event, modelId
| sort @timestamp desc
| limit 10000

When you run a query like the one above, Logs Insights shows a time-series visualization of event counts for the selected time range (for example, last hour). That visualization helps you spot traffic spikes or quiet periods. Under the chart you’ll find the matching log events; expand a log event to inspect the extracted fields.

Filter for failed Bedrock requests (show timestamp, event, modelId, durationMs, requestId; most recent 20):

fields @timestamp, event, modelId, durationMs, requestId
| filter event = "bedrock_request_failed"
| sort @timestamp desc
| limit 20

Find slow successful responses (duration greater than 3000 ms):

fields @timestamp, modelId, durationMs, outputLength
| filter event = "bedrock_request_succeeded" and durationMs > 3000
| sort durationMs desc
| limit 20

Average latency by model (helpful to compare model performance):

# AVERAGE LATENCY BY MODEL
stats avg(durationMs) as avgLatency by modelId
| sort avgLatency desc

Approximate token / context window usage (prompt length + output length), sorted descending:

# TOKEN USAGE APPROXIMATION
fields @timestamp, modelId, promptLength, outputLength
| sort (promptLength + outputLength) desc
| limit 20

These queries help answer operational questions such as:

Which models are slowest on average?
Which requests are failing, and when?
Are any requests approaching a model’s context window limit?

Quick reference: event names and their purpose

Event name	Purpose	Example Logs Insights filter
`bedrock_request_started`	Marks the start of a model invocation (useful for calculating end-to-end latency)	`filter event = "bedrock_request_started"`
`bedrock_request_succeeded`	Successful model invocation (contains duration and output length)	`filter event = "bedrock_request_succeeded"`
`bedrock_request_failed`	Failed invocation and error context	`filter event = "bedrock_request_failed"`

Application-level logging (Lambda)

Bedrock model invocation logs capture model-level input/output, but your application should emit structured logs that provide context around those calls—what the app was doing, which user or API request triggered the model call, and correlation identifiers. For AWS Lambda, anything written to stdout is automatically forwarded to the function’s CloudWatch Logs group. Using a structured logger (JSON) makes logs easier to parse and correlate with model invocation logs.

A slide titled "Workflow: App Logging in Lambda" showing three colorful arrows that illustrate app logging flow from the application and the Lambda Python runtime into a central stdout path. The diagram notes that stdout is automatically sent to CloudWatch logs and suggests enhancing output formatting with a logging module.

Below is a complete Python Lambda handler that:

Calls Bedrock via the runtime client,
Emits structured JSON events to stdout so they appear in the function’s CloudWatch Logs group, and
Includes the Lambda context.aws_request_id so you can correlate application logs with Bedrock model logs.

import json
import time
import logging
import boto3

logger = logging.getLogger()
logger.setLevel(logging.INFO)

# Instantiate the Bedrock runtime client (region as appropriate)
bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")

def lambda_handler(event, context):
    prompt = event.get("prompt", "Summarize the benefits of Amazon Bedrock.")
    model_id = "amazon.nova-lite-v1:0"

    start = time.time()

    # Application-level start event (will go to the Lambda function's log group)
    logger.info(json.dumps({
        "event": "bedrock_request_started",
        "modelId": model_id,
        "promptLength": len(prompt),
        "requestId": context.aws_request_id
    }))

    try:
        # Call the Bedrock runtime using the generic invoke_model API.
        # The exact request/response shape can vary by model, so we send a simple JSON body and parse common response structures.
        response = bedrock.invoke_model(
            modelId=model_id,
            contentType="application/json",
            accept="application/json",
            body=json.dumps({"input": prompt}).encode("utf-8")
        )

        # Read and parse the response body (StreamingBody)
        resp_bytes = response["body"].read()
        resp_text = resp_bytes.decode("utf-8", errors="replace")
        try:
            resp_json = json.loads(resp_text)
        except Exception:
            resp_json = None

        # Try to extract output text from common response shapes.
        output_text = None
        if isinstance(resp_json, dict):
            # Shape: {"output": {"message": {"content": [{"text": "..."}, ...]}}}
            out = resp_json.get("output")
            if isinstance(out, dict):
                msg = out.get("message") or {}
                content = msg.get("content") if isinstance(msg, dict) else None
                if isinstance(content, list):
                    for c in content:
                        if isinstance(c, dict) and "text" in c:
                            output_text = c["text"]
                            break

            # Shape: {"outputs": [{"content": [{"text": "..."}]}]}
            if output_text is None and "outputs" in resp_json and isinstance(resp_json["outputs"], list):
                for o in resp_json["outputs"]:
                    content = o.get("content", [])
                    if isinstance(content, list):
                        for c in content:
                            if isinstance(c, dict) and "text" in c:
                                output_text = c["text"]
                                break
                    if output_text:
                        break

            # Fallback: look for any top-level string value
            if output_text is None:
                for v in resp_json.values():
                    if isinstance(v, str):
                        output_text = v
                        break

        # Final fallback to raw text
        if output_text is None:
            output_text = resp_text

        duration_ms = round((time.time() - start) * 1000, 2)

        # Success event with additional metrics
        logger.info(json.dumps({
            "event": "bedrock_request_succeeded",
            "modelId": model_id,
            "durationMs": duration_ms,
            "outputLength": len(output_text) if isinstance(output_text, str) else 0,
            "requestId": context.aws_request_id
        }))

        return {
            "statusCode": 200,
            "body": json.dumps({
                "response": output_text
            })
        }

    except Exception as e:
        duration_ms = round((time.time() - start) * 1000, 2)

        # Exception event — logger.exception will include the stack trace
        logger.exception(json.dumps({
            "event": "bedrock_request_failed",
            "modelId": model_id,
            "durationMs": duration_ms,
            "requestId": context.aws_request_id,
            "error": str(e)
        }))

        return {
            "statusCode": 500,
            "body": json.dumps({"error": "Invocation failed"})
        }

Do not log sensitive data (PII, credentials, full user messages, or API keys). Use redaction and minimize sensitive fields in logs. Structured logs still carry risk if they include raw user content.

Notes and best practices

Use structured logs (JSON) for application events—this makes Logs Insights filters simple and reliable.
Include a correlation identifier such as context.aws_request_id so you can map a Lambda invocation’s application logs to Bedrock model invocation logs.
Use distinct, consistent event names such as bedrock_request_started, bedrock_request_succeeded, and bedrock_request_failed. The example Log Insights queries above filter on those event names.
Emit additional contextual fields (user id, API path, feature flag state, deployment stage) when relevant—this metadata speeds troubleshooting and attribution.
For most Lambda use cases, writing to stdout is sufficient; CloudWatch Logs will automatically capture those entries. If you need programmatic control, you can instantiate a CloudWatch Logs client with boto3 to create log groups, push events, or manage retention.
When parsing model responses, be prepared for different response shapes depending on the model—implement safe fallbacks and defensive parsing as shown in the example.

Links and references

By combining Bedrock model-level logs with concise, structured application logs, you can craft powerful Logs Insights queries that surface failures, latency hotspots, and usage patterns across users and applications—enabling faster and more accurate troubleshooting.

Introduction to Gen AI and LLMs

Introduction to Amazon Bedrock

Getting Started With Amazon Bedrock

Foundation Models in Amazon Bedrock

Basic Concepts of Prompt Engineering

Introduction to Text Generation With Amazon Bedrock

How Foundation Models Process Input

Extract Insights Using Prompt Engineering

Working With Different Task Types

Integrating Amazon Bedrock With Other AWS Services

Build a Marketing Email Generation Application

Introduction to Retrieval Augmentation With Bedrock Knowledge Bases

Best Practices and Optimization

Governance, Safety in Responsible AI

Security

Monitoring and Logging

Taking action with Bedrock Agents

Advanced Topics (Optional)

Final Project

Wrap Up

Using Amazon CloudWatch With Bedrock Part 3

Common Logs Insights queries for Bedrock model invocations

Quick reference: event names and their purpose

Application-level logging (Lambda)

Notes and best practices

Links and references

Watch Video

​Common Logs Insights queries for Bedrock model invocations

​Quick reference: event names and their purpose

​Application-level logging (Lambda)

​Notes and best practices

​Links and references

Watch Video

Common Logs Insights queries for Bedrock model invocations

Quick reference: event names and their purpose

Application-level logging (Lambda)

Notes and best practices

Links and references