Demo Deploy a Hosted Model Using SageMaker Endpoints

In this lesson you’ll learn how to host a trained model in Amazon SageMaker by creating and using SageMaker endpoints. This is a demonstration-oriented walkthrough using SageMaker Studio (Jupyter notebooks) and the AWS Management Console. We’ll show both low-level (Boto3) and high-level (SageMaker Python SDK) approaches so you can compare patterns for model registration, endpoint creation, and invoking inference. What we’ll cover

Create an endpoint configuration and endpoint via the Boto3 SDK.
Create a model and endpoint using the SageMaker Python SDK (single-flow approach).
Compare invocation methods: Boto3 invoke_endpoint vs SageMaker SDK Predictor.
Test and clean up endpoints to avoid incurring charges.

High-level demo steps (what we’ll run from the notebook):

A dark-blue presentation slide titled "Demo Steps" listing three numbered steps. The steps instruct confirming a model package is available, running a Jupyter notebook with boto3 to create and test SageMaker endpoint/config, and running a notebook with the SageMaker SDK to create and test the endpoint using the Predictor class.

We’ll use the SageMaker SDK Predictor for the SDK example and Boto3’s invoke_endpoint for the Boto3 example so you can see the differences in interaction style. Getting started in SageMaker Studio All commands in this demo are executed from a JupyterLab notebook inside SageMaker Studio. Here is the JupyterLab Launcher in SageMaker Studio where the notebooks live:

A screenshot of a JupyterLab / SageMaker notebook interface with a file browser on the left listing a "sagemaker-demystified" project and several notebooks and files. The main pane shows a Launcher with tiles for Python 3, Glue PySpark, SparkMagic kernels, consoles, and other items like Terminal, Text File, and Markdown File.

Open the notebook named “Inference Demo” and run the steps below.

Initialize the notebook: import libraries, create a SageMaker session, obtain the execution role, and set an S3 bucket/prefix for model artifacts

# Step 1: Import necessary libraries
import boto3
import sagemaker
from sagemaker import get_execution_role
from sagemaker.session import Session
from sagemaker import Model

# Define your SageMaker session and role
sagemaker_session = Session()
role = get_execution_role()

# Define S3 bucket and prefix (avoid a leading slash in the prefix)
s3_bucket = sagemaker_session.default_bucket()
s3_prefix = 'model'

print("Execution role:", role)
print("S3 bucket:", s3_bucket)
print("S3 prefix:", s3_prefix)

When you run it you may see SDK informational logs followed by the printed role and bucket values. Example:

sagemaker.config INFO - Not applying SDK defaults from location: /etc/xdg/sagemaker/config.yaml
sagemaker.config INFO - Not applying SDK defaults from location: /home/sagemaker-user/.config/sagemaker/config.yaml
Execution role: arn:aws:iam::485186561655:role/service-role/AmazonSageMaker-ExecutionRole-20241216T134619
S3 bucket: sagemaker-eu-central-1-485186561655
S3 prefix: model

Upload the model artifact to S3 (if applicable) and determine the container image URI for hosting

If your model was produced by a training job, you can skip the upload and reference the training job’s output S3 URI directly.
For models that were saved locally in the notebook environment, upload to S3 and then choose the appropriate container image (this demo uses Linear Learner).

# Upload the model artifact to S3
local_file_path = 'model.tar.gz'  # local artifact in the notebook filesystem
s3_uri = sagemaker_session.upload_data(local_file_path, bucket=s3_bucket, key_prefix=s3_prefix)
print(f"File uploaded to: {s3_uri}")

model_artifact = f's3://{s3_bucket}/{s3_prefix}/model.tar.gz'
print("Model artifact S3 path:", model_artifact)

# Determine what AWS region we are in
region = sagemaker_session.boto_region_name
print("Region:", region)

# Retrieve the Linear Learner image URI for the region
container_image_uri = sagemaker.image_uris.retrieve(framework="linear-learner", region=region)
print("SageMaker Linear Learner Image URI:", container_image_uri)

# Assign a meaningful name to the model
model_name = 'house-prices-inference-demo'
print("Model name:", model_name)

Example output:

File uploaded to: s3://sagemaker-eu-central-1-485186561655/model/model.tar.gz
Model artifact S3 path: s3://sagemaker-eu-central-1-485186561655/model/model.tar.gz
Region: eu-central-1
SageMaker Linear Learner Image URI: 664544806723.dkr.ecr.eu-central-1.amazonaws.com/linear-learner:1
Model name: house-prices-inference-demo

If your model artifact is already available from a training job’s output, you can skip the upload step and reference the existing training output S3 path.

Create a SageMaker model object (register the model in SageMaker)

The SageMaker SDK Model constructs a model resource in SageMaker that maps the container image to the model artifact and IAM role. Only when you call model.create() is the model created in SageMaker.

# Create the SageMaker model using Model class from the SageMaker SDK
model = Model(
    image_uri=container_image_uri,  # container image URI for hosting
    model_data=model_artifact,      # S3 path to the model artifact
    role=role,
    name=model_name
)

# Register the model in SageMaker (creates a model package)
model.create()
print(f"Model created in SageMaker with name: {model_name}")

After this step, confirm in the AWS Management Console under Inference → Models that the model package house-prices-inference-demo exists with the matching model artifact and container image. Example resource values you may see (for reference):

arn:aws:sagemaker:eu-central-1:485186561655:model/house-prices-inference-demo
arn:aws:iam::485186561655:role/service-role/AmazonSageMaker-ExecutionRole-20241216T134619
s3://sagemaker-eu-central-1-485186561655/model/model.tar.gz
664544806723.dkr.ecr.eu-central-1.amazonaws.com/linear-learner:1

Create an endpoint configuration using Boto3

When using Boto3 you create an Endpoint Configuration first. The config defines production variants—mapping models to instance types, counts, and traffic weights. Below we create a single variant consuming all traffic.

# Create Endpoint Config using Boto3
sagemaker_client = boto3.client('sagemaker', region_name=region)

endpoint_config_name = "my-endpoint-config-using-boto3"

create_endpoint_config_response = sagemaker_client.create_endpoint_config(
    EndpointConfigName=endpoint_config_name,
    ProductionVariants=[{
        'InstanceType': 'ml.t2.medium',
        'InitialVariantWeight': 1.0,
        'InitialInstanceCount': 1,
        'ModelName': model_name,
        'VariantName': 'AllTraffic'
    }]
)

print(f"Endpoint configuration created: {create_endpoint_config_response['EndpointConfigArn']}")

Validate the endpoint configuration in the Console at Inference → Endpoint configurations.

Create the SageMaker endpoint that references the endpoint configuration

# Create the Endpoint using the endpoint configuration created above
endpoint_name = "my-endpoint-using-boto3"

create_endpoint_response = sagemaker_client.create_endpoint(
    EndpointName=endpoint_name,
    EndpointConfigName=endpoint_config_name
)

print(f"Endpoint creation initiated: {create_endpoint_response['EndpointArn']}")

Once started, the endpoint will appear in the Console (Inference → Endpoints) with status Creating. Provisioning typically takes several minutes. The console provides an endpoint summary, the ARN, and operational metrics while the endpoint is being created:

A screenshot of the Amazon SageMaker console showing an endpoint named "my-endpoint-using-boto3" with status "Creating," plus its ARN/URL and endpoint summary. The page also shows monitoring tabs and operational metrics panels (CPU/Memory).

Two important notes:

Boto3 requires a two-step process: create the endpoint configuration, then create the endpoint that references it.
The SageMaker Python SDK provides higher-level constructs to create model and endpoint resources in fewer lines of code and supports using Predictor for inference.

SageMaker Studio and the AWS Management Console show related but slightly different views. Studio lists endpoints under Deployments → Endpoints, but it does not display endpoint configuration objects separately. To inspect endpoint configurations directly, use the Management Console (Inference → Endpoint configurations) or enumerate them via SDK calls.

Optional: Poll for endpoint creation completion (Boto3) Use this snippet to wait for the endpoint to transition to InService or to report failures. This is useful for automation in notebooks or CI pipelines.

import time

def wait_for_endpoint(endpoint_name, timeout=900, poll_interval=15):
    start = time.time()
    while time.time() - start < timeout:
        resp = sagemaker_client.describe_endpoint(EndpointName=endpoint_name)
        status = resp['EndpointStatus']
        print(f"Endpoint status: {status}")
        if status == 'InService':
            print("Endpoint is InService")
            return True
        if status in ('Failed', 'RollingBack'):
            raise RuntimeError(f"Endpoint reached status {status}: {resp}")
        time.sleep(poll_interval)
    raise TimeoutError("Timeout waiting for endpoint to be InService")

wait_for_endpoint(endpoint_name)

Testing and invocation

For the Boto3-based endpoint, call the runtime invoke_endpoint API to send binary/payload requests and receive the response.
For the SDK-based endpoint (later in the notebook), use the Predictor class which wraps serialization/deserialization and runtime invocation in a convenient interface.

Cleanup to avoid charges When finished, delete endpoint resources:

# Delete endpoint (Boto3)
sagemaker_client.delete_endpoint(EndpointName=endpoint_name)

# Delete endpoint configuration
sagemaker_client.delete_endpoint_config(EndpointConfigName=endpoint_config_name)

# Delete model (SageMaker SDK or Boto3)
sagemaker_client.delete_model(ModelName=model_name)

Resource and command summary

Resource	Use / Purpose	Example or SDK
Model	Registers the model artifact + container in SageMaker	`Model(...).create()` (SageMaker SDK)
Endpoint configuration	Defines production variants and instance types	`sagemaker_client.create_endpoint_config(...)` (Boto3)
Endpoint	The network endpoint to receive inference requests	`sagemaker_client.create_endpoint(...)` (Boto3)
Invoke endpoint	Submit inference payload	`runtime_client.invoke_endpoint(...)` (Boto3) or `Predictor.predict(...)` (SageMaker SDK)

Links and references

SageMaker SDK documentation: https://sagemaker.readthedocs.io/
Boto3 SageMaker client: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html
SageMaker Predictor: https://sagemaker.readthedocs.io/en/stable/predictors.html
Amazon S3 basics: https://learn.kodekloud.com/user/courses/amazon-simple-storage-service-amazon-s3

Next steps in the notebook

Run the SDK-based example that builds the Model and endpoint using higher-level APIs and then uses Predictor to make requests.
Compare the two invocation styles (Boto3 vs Predictor) for serialization, error handling, and convenience.
Delete the endpoint, endpoint configuration, and model resources to avoid ongoing charges.

Machine Learning Prerequisites

SageMaker Introduction

SageMaker User Interface

Persona SageMaker Activities - Data Engineer

Persona SageMaker Activities - Data Scientist

Persona SageMaker Activities - MLOps Engineer

Wrap Up

Course Introduction

Demo Deploy a Hosted Model Using SageMaker Endpoints

Watch Video