Using Amazon CloudWatch With Bedrock Part 4

If your Bedrock-powered application runs on an Amazon Elastic Compute Cloud (EC2) instance (a virtual machine) rather than AWS Lambda, your logging strategy changes slightly. From a VM you can either ship local logs using an agent or have the application call CloudWatch Logs directly using the AWS SDK (for example, Boto3). Writing logs directly to CloudWatch centralizes observability and avoids relying solely on the instance filesystem. Key operational considerations when writing logs from a VM:

CloudWatch Logs expects event timestamps in milliseconds since the epoch.
When appending to an existing log stream, CloudWatch requires the correct sequenceToken (upload sequence token). If the stream is new, create the log group and log stream before calling put_log_events.
Your application needs IAM permissions like logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents, and optionally logs:DescribeLogStreams.
On EC2, attach an IAM role (instance profile) to the instance so your code can use temporary credentials automatically. For hosts outside AWS, provide programmatic credentials (access key/secret key) with minimal necessary scope.

The image is a simple infographic titled "Workflow: Logging in VM" showing two semicircles: the left (blue) says "Create SDK client to CloudWatch Logs" and the right (green) says "Use put log events method to write directly to a custom CloudWatch log group," with cloud/log icons.

Example: writing logs from EC2 to CloudWatch using Boto3

This sample shows the common sequence:
1. Ensure the log group and log stream exist.
2. Query the upload sequence token (if any).
3. Call put_log_events.
4. Handle InvalidSequenceTokenException/DataAlreadyAcceptedException by refetching the token and retrying once.

import json
import time
import boto3
import botocore

logs = boto3.client("logs", region_name="us-east-1")

LOG_GROUP = "bedrock-app-logs"
LOG_STREAM = "api-server-1"

def ensure_log_group_and_stream():
    # Create log group if it doesn't exist
    try:
        logs.create_log_group(logGroupName=LOG_GROUP)
    except botocore.exceptions.ClientError as e:
        if e.response["Error"]["Code"] not in ("ResourceAlreadyExistsException", "ResourceAlreadyExists"):
            raise

    # Create log stream if it doesn't exist
    try:
        logs.create_log_stream(logGroupName=LOG_GROUP, logStreamName=LOG_STREAM)
    except botocore.exceptions.ClientError as e:
        if e.response["Error"]["Code"] not in ("ResourceAlreadyExistsException", "ResourceAlreadyExists"):
            raise

def _get_sequence_token():
    resp = logs.describe_log_streams(
        logGroupName=LOG_GROUP,
        logStreamNamePrefix=LOG_STREAM,
        limit=1
    )
    streams = resp.get("logStreams", [])
    if not streams:
        return None
    return streams[0].get("uploadSequenceToken")

def write_log(message: dict) -> None:
    timestamp_ms = int(time.time() * 1000)
    event = {
        "timestamp": timestamp_ms,
        "message": json.dumps(message)
    }

    sequence_token = _get_sequence_token()
    kwargs = {
        "logGroupName": LOG_GROUP,
        "logStreamName": LOG_STREAM,
        "logEvents": [event]
    }
    if sequence_token:
        kwargs["sequenceToken"] = sequence_token

    try:
        logs.put_log_events(**kwargs)
    except botocore.exceptions.ClientError as e:
        # Handle invalid sequence token by fetching latest token and retrying once
        code = e.response["Error"]["Code"]
        if code in ("InvalidSequenceTokenException", "DataAlreadyAcceptedException"):
            sequence_token = _get_sequence_token()
            if sequence_token:
                kwargs["sequenceToken"] = sequence_token
            else:
                kwargs.pop("sequenceToken", None)
            logs.put_log_events(**kwargs)
        else:
            raise

if __name__ == "__main__":
    ensure_log_group_and_stream()

    write_log({
        "event": "bedrock_request_started",
        "modelId": "amazon.nova-lite-v1:0",
        "promptVersion": "v3",
        "userId": "user-123"
    })

Grant the required IAM permissions to the identity used by your code: typically logs:CreateLogGroup, logs:CreateLogStream, logs:DescribeLogStreams, and logs:PutLogEvents. On EC2, prefer an instance profile (IAM role); if running outside AWS, use short-lived credentials and scope them narrowly.

Comparing logging approaches for Bedrock applications

Choose based on where compute runs, privacy/governance needs, and credential provisioning.

Scenario	How logs are captured	Requirements	Recommended when
AWS Lambda-hosted code	Anything written to stdout/stderr is automatically captured and sent to CloudWatch Logs	Lambda execution role (created by default) with logging permissions	Quick setup, no SDK changes; serverless functions
Application calling CloudWatch Logs directly	App uses the SDK (for example, Boto3) to call `put_log_events`	Credentials that permit CloudWatch Logs (`logs:*` as needed); on EC2 use instance role	Full control over log structure, centralized logging from VMs or external hosts
Bedrock model invocation logging	Bedrock service logs model inputs/outputs at service level	Bedrock permissions to configure invocation logging	When you want service-level tracing without app changes; consider privacy implications

Links and references

An infographic titled "Workflow: Logging Dependencies" shows a target with an arrow on the left and a vertical sequence of colorful service icons and labels on the right. The labels (Bedrock Model, Direct CloudWatch, Lambda App Logging) list each approach's code dependencies and required IAM/permissions.

Summary and best practices

For AWS Lambda: write to stdout/stderr and ensure the execution role has logging permissions; CloudWatch capture is automatic.
For EC2 / VMs: either run a log shipping agent or have the app call CloudWatch Logs APIs directly; use instance roles to avoid long-lived credentials.
For external hosts: use programmatic credentials scoped to only the required CloudWatch permissions and rotate them regularly.
For Bedrock-level tracing: enable model invocation logging in the Bedrock service when you need centralized model I/O capture — but review privacy and governance requirements before enabling.

Introduction to Gen AI and LLMs

Introduction to Amazon Bedrock

Getting Started With Amazon Bedrock

Foundation Models in Amazon Bedrock

Basic Concepts of Prompt Engineering

Introduction to Text Generation With Amazon Bedrock

How Foundation Models Process Input

Extract Insights Using Prompt Engineering

Working With Different Task Types

Integrating Amazon Bedrock With Other AWS Services

Build a Marketing Email Generation Application

Introduction to Retrieval Augmentation With Bedrock Knowledge Bases

Best Practices and Optimization

Governance, Safety in Responsible AI

Security

Monitoring and Logging

Taking action with Bedrock Agents

Advanced Topics (Optional)

Final Project

Wrap Up

Using Amazon CloudWatch With Bedrock Part 4

Watch Video