Inference Profiles

In this article we explain inference profiles in Amazon Bedrock: what they are, why some model IDs include prefixes like us., how to use them in code, and the trade-offs to consider. You’ll learn how inference profiles affect routing and capacity so you can reliably invoke Bedrock models from your application.

Problem

When selecting a model from the Bedrock model catalog, many developers copy the catalog’s programmatic model ID into their CLI or application code. Sometimes that exact identifier—copied directly from the catalog—fails when you try to invoke the model. This is confusing because the catalog ID seems correct, yet some invocations succeed and others fail. The mismatch is frequently caused by inference profiles: Bedrock’s runtime routing configuration that may require a system-defined prefix on the model ID.

What are inference profiles?

Inference profiles are platform-provided deployment/routing configurations for certain Bedrock foundation models. An inference profile lets Bedrock route your request across a specified geography (for example, within U.S. regions) rather than binding it to a single regional endpoint. This increases the chance Bedrock finds available capacity for your request. Key characteristics:

Platform-level routing mechanism provided by AWS/Bedrock.
Do not change the model architecture or user-facing APIs.
Often system-defined (for example, us); users generally cannot create these profiles.
Adding a profile prefix to the model ID (for example us.) tells Bedrock to route the request within that profile scope.

Table: quick reference

Topic	Summary	Example
Purpose	Route requests across a geographic profile to find capacity	`us.meta-llama-3-1-8b-instruct-v1`
Scope	System-defined routing (not user-created)	`us.` indicates U.S. inference profile
Effect on model	No change to model behavior or architecture	Same prompts and parameters apply

How it looks in code

You typically only change the modelId string to include the inference profile prefix; your application logic remains the same. Below is a concise Python example using the Bedrock runtime client to illustrate the difference. The key change is adding the us. prefix to the modelId to select the system-defined U.S. inference profile.

# python
import boto3
import json

# Create a Bedrock runtime client (region can remain your default)
client = boto3.client("bedrock-runtime")

# Model ID without inference profile (may fail for some models)
model_id_plain = "meta-llama-3-1-8b-instruct-v1"

# Model ID with system inference profile (routes within US regions)
model_id_with_profile = "us.meta-llama-3-1-8b-instruct-v1"

# Simple payload: ask for 2 short bullet points
payload = {
    "input": "Explain Bedrock in two short bullet points.",
    # Runtime-specific parameters such as max tokens may be placed in the request body
    # or passed using provider-specific fields. This is a simplified example.
    "parameters": {"max_tokens": 200}
}

response = client.invoke_model(
    modelId=model_id_with_profile,
    contentType="application/json",
    accept="application/json",
    body=json.dumps(payload).encode("utf-8")
)

# The response body is returned as bytes; decode and print
body = response["body"].read().decode("utf-8")
print(body)

Notes on the code:

This example demonstrates how the modelId is formed. SDKs and API versions may differ in exact parameter names and locations for runtime options.
For production use, consult the AWS Bedrock developer guide and the specific SDK docs for your language:
- AWS Bedrock Developer Guide: https://docs.aws.amazon.com/bedrock/latest/devguide/

Some Bedrock catalog entries require a system-defined inference profile prefix (for example, us.). If an invocation fails using the catalog ID, retry with the relevant inference profile prefix before changing other parts of your code.

Why use inference profiles?

Benefits:

Reliability: Fewer invocation failures because Bedrock can route to any region inside the profile’s scope that has capacity.
Simplicity: Your application does not need to implement region-level fallback logic; Bedrock handles routing.
No model modification: The model’s behavior is unchanged—only where Bedrock looks for capacity is expanded.

Caveats:

Not every model uses or requires inference profiles. They are platform-provided and sometimes mandatory for specific foundation models.
Broader routing may increase latency if Bedrock routes the request to a geographically distant region.
The model catalog may not consistently indicate when a profile prefix is required; trial-and-error (or documentation) may be needed.

Table: Benefits vs. Caveats

Benefit	Caveat
Reduces invocation failures due to capacity constraints	Possible increased latency if routed outside your primary region
Simplifies client logic by delegating routing to Bedrock	Not all models support inference profiles
No changes to model usage or prompts	Catalog might not explicitly state which models require prefixes

Summary

Some Bedrock models require a system-defined inference profile prefix (for example us.). Adding that prefix to the model ID instructs Bedrock to route the request within the profile’s geographic scope so the runtime can locate available capacity. This reduces invocation errors and makes model access more consistent across regions, at the possible cost of higher latency if routing outside your primary region. Further reading and references:

AWS Bedrock Developer Guide: https://docs.aws.amazon.com/bedrock/latest/devguide/
Bedrock API and SDK documentation (refer to the SDK for your language)

Introduction to Gen AI and LLMs

Introduction to Amazon Bedrock

Getting Started With Amazon Bedrock

Foundation Models in Amazon Bedrock

Basic Concepts of Prompt Engineering

Introduction to Text Generation With Amazon Bedrock

How Foundation Models Process Input

Extract Insights Using Prompt Engineering

Working With Different Task Types

Integrating Amazon Bedrock With Other AWS Services

Build a Marketing Email Generation Application

Introduction to Retrieval Augmentation With Bedrock Knowledge Bases

Best Practices and Optimization

Governance, Safety in Responsible AI

Security

Monitoring and Logging

Taking action with Bedrock Agents

Advanced Topics (Optional)

Final Project

Wrap Up

Inference Profiles

Problem

What are inference profiles?

How it looks in code

Why use inference profiles?

Summary

Watch Video

​Problem

​What are inference profiles?

​How it looks in code

​Why use inference profiles?

​Summary

Watch Video

Problem

What are inference profiles?

How it looks in code

Why use inference profiles?

Summary