Handling Errors Troubleshooting and Edge Cases Part 2

​

Retry loop with exponential backoff (boto3 example)

• Creating a boto3 Bedrock Runtime client.
• Iterating attempts from 1..N for human-friendly logging.
• Retrying only on specific error codes (e.g., ThrottlingException).
• Breaking early on success and logging failures when unretryable or final attempt fails.

import time
import boto3
from botocore.exceptions import ClientError

bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")

max_retries = 3

for attempt in range(1, max_retries + 1):
    try:
        response = bedrock.invoke_model(
            modelId="amazon.nova-lite-v1:0",
            body='{"inputText":"Summarize this review"}',
            contentType="application/json",
            accept="application/json"
        )
        print("Request succeeded")
        break

    except ClientError as e:
        error_code = e.response["Error"]["Code"]

        # If throttled and we still have attempts left, back off and retry
        if error_code == "ThrottlingException" and attempt < max_retries:
            wait_time = 2 ** attempt  # exponential backoff: 2, 4, 8...
            print(f"Throttled. Retrying in {wait_time} seconds... (attempt {attempt}/{max_retries})")
            time.sleep(wait_time)
        else:
            print(f"Request failed: {e.response['Error'].get('Message', str(e))}")
            break

• Use range(1, max_retries + 1) so logs show attempts starting at 1.
• 2 ** attempt implements exponential backoff. Consider adding jitter (randomized sleep) to avoid synchronized retries across clients.
• Retry selectively — limit retries to transient/throttling errors. For other errors, log and stop.
• Record metrics (counts of throttles, retries, latency) so you can monitor trends and set alarms.

​

Validate model output (guard against malformed responses)

import json

response_body = '{"summary":"Great product and fast delivery."}'

try:
    data = json.loads(response_body)
    if "summary" not in data:
        raise ValueError("Missing expected `summary` field")
    print(data["summary"])
except json.JSONDecodeError:
    print("Model response was not valid JSON")
except ValueError as e:
    print(f"Validation error: {e}")

• Catch json.JSONDecodeError for invalid JSON.
• Use explicit validation: check required fields and types; raise ValueError or a custom exception to unify handling.
• Decide a policy per validation failure: retry the model call, sanitize the response, ask the model to reformat, or return a safe fallback to the user.
• Log schema failures with examples (scrub any PII) to improve prompts or post-processing.

​

Retrieval-Augmented Generation (RAG) — when no context is found

• Proceed and tell the model explicitly that no supporting context was found.
• Refuse to answer and prompt the user to rephrase or provide more detail.
• Return a safe fallback message that avoids speculation.

retrieved_chunks = []

if not retrieved_chunks:
    print("No relevant knowledge base results found.")
    final_prompt = "Answer the user's question, but say clearly that no supporting context was retrieved."
else:
    context = "\n".join(retrieved_chunks)
    final_prompt = f"Use this context:\n{context}\n\nAnswer the question."

​

Handling token limits and context constraints

• Summarize or truncate long inputs before sending.
• Limit retrieved chunks in RAG (e.g., top 3 instead of top 10).
• Set an explicit max tokens (or equivalent) to cap the output size.
• Preprocess or compress retrieved documents to extract the most relevant spans.
• Break tasks into multiple calls or multi-step workflows when appropriate.

• Instruct the model with explicit output constraints, for example: “Summarize this document in one paragraph of no more than 200 words.”
• Use server-side limits in your API call to enforce a maximum number of output tokens.

​

Handling safety responses and guardrails

• Return a short, clear message such as: “This prompt cannot be processed because it violates our safety policy.”
• Avoid exposing internal guardrail names, policy internals, or implementation details.
• Degrade the UI gracefully (do not crash the app).
• Log safety events (without sensitive user content) so you can audit trends and tune safeguards.

​

Quick checklist before deploying to production

• Implement selective retries with exponential backoff and jitter.
• Validate model outputs and fail early when schema expectations are unmet.
• Define clear RAG fallback behavior for empty retrievals.
• Enforce token limits via input compression and API-level output caps.
• Display user-friendly messages for safety blocks and log events securely.
• Instrument metrics for throttling, retries, validation failures, and safety interceptions.

​

Links and references

• Amazon Bedrock documentation
• boto3 — AWS SDK for Python
• botocore exceptions reference
• Best practices for retriable API requests (exponential backoff and jitter) — see many cloud SDK guides for reference.