Demo End to End CICD ML Pipelines from SageMaker Project Templates
Demo of end-to-end SageMaker project templates provisioning CI/CD for ML workflows that build, register, approve and deploy models
This lesson walks through building a complete CI/CD workflow that integrates SageMaker Pipelines with AWS developer tools. Using a SageMaker Project template, we provision the infrastructure, trigger model builds, register models in the Model Registry, and deploy them across staging and production with a manual approval gate.Core steps:
Review built‑in SageMaker Project templates.
Use a template that provisions a Git‑compatible repository (AWS CodeCommit in this demo).
Inspect the CloudFormation template that the SageMaker Project uses and follow the resulting CI/CD flow (CodeCommit → CodePipeline → CodeBuild → SageMaker Pipelines → Model Registry → deployment).
Start in SageMaker Studio and confirm a clean environment (no model packages, no endpoints).
At the start of the lesson there are no model packages or endpoints in the account.
From SageMaker Studio navigate to Deployments → Projects and choose a project template. Templates are blueprints that create all required CI/CD and ML infra (for example: CodeCommit repos, CodeBuild projects, CodePipeline pipelines, S3 artifacts buckets, SageMaker Pipelines, IAM roles) using a CloudFormation stack.
Effective September 9, 2024, SageMaker project templates that create AWS CodeCommit repositories are deprecated. This demo uses a CodeCommit template to show the in-console experience, but for production consider third‑party Git providers (e.g., GitHub) with a CodeStar connection or equivalent.
Choose a template that creates both:
a model‑build pipeline (trains and registers models), and
a model‑deploy pipeline (staging → production with a manual approval gate).
Templates that integrate with third‑party Git providers will prompt for repo URLs and branches instead of creating CodeCommit repos.
Supply a project name, description, and tags that fit your org conventions.
When the project is created, SageMaker launches a CloudFormation stack that provisions the resources described by the template. Open the CloudFormation console to monitor stack events and inspect created resources (S3 buckets, CodeCommit repos, CodeBuild projects, CodePipeline pipelines, SageMaker Pipelines, IAM roles, etc.).
You can inspect the CloudFormation template (a declarative YAML manifest) used by the project. The template declares parameters for the SageMaker project and resources such as an artifacts S3 bucket, EventBridge rules, CodeCommit repositories, CodeBuild projects, and CodePipeline pipelines.Example excerpt from the template:
Copy
Description: > Toolchain template which provides the resources needed to represent infrastructure as code. This template specifically creates a CI/CD pipeline to build a model using a SageMaker Pipeline and deploy the resulting trained ML Model from Model Registry to two stages in CD -- staging and production.Parameters: SageMakerProjectName: Type: String Description: Name of the project MinLength: 1 MaxLength: 32 AllowedPattern: '^[a-zA-Z](-*[a-zA-Z0-9])*' SageMakerProjectId: Type: String Description: Service generated Id of the project.Resources: MLOpsArtifactsBucket: Type: AWS::S3::Bucket DeletionPolicy: Retain Properties: BucketName: Fn::Sub: sagemaker-project-${SageMakerProjectId} ModelBuildCodeCommitRepository: Type: AWS::CodeCommit::Repository Properties: RepositoryName: Fn::Sub: sagemaker-${SageMakerProjectName}-${SageMakerProjectId}-modelbuild RepositoryDescription: Fn::Sub: 'SageMaker Model building workflow infra-as-code for Project ${SageMakerProjectName}' Code: S3: Bucket: sagemaker-servicecatalog-seedcode-eu-central-1 Key: toolchain/model-building-workflow-v1.0.zip BranchName: main SageMakerModelPipelineBuildProject: Type: AWS::CodeBuild::Project Properties: Name: Fn::Sub: sagemaker-${SageMakerProjectName}-${SageMakerProjectId}-modelbuild Description: 'Builds the model building workflow code repository, creates the SageMaker Pipeline and executes it'
When the stack completes, the new SageMaker Project appears in Studio.
The CloudFormation stack created two source repositories (model‑build and model‑deploy). Open CodeCommit to review the seed code: buildspecs, helper scripts, CloudFormation templates, and the sample SageMaker pipeline code that the build runs.
The project provisions two CodePipeline pipelines:
Model build pipeline — triggers on commits to the model‑build repo, runs CodeBuild to create and execute a SageMaker Pipeline that preprocesses, trains, evaluates, and registers a model.
Model deploy pipeline — triggers on commits to the model‑deploy repo or on model approval events; packages CloudFormation templates and deploys staging and production endpoints. A manual approval action gates production.
Open CodePipeline to view pipeline stages and their status.
When CodeCommit is seeded by the template, CodePipeline detects commits and starts the model build pipeline automatically.
CodeBuild: How the Pipeline Invokes a SageMaker Pipeline
During the build stage, CodePipeline invokes a CodeBuild project. CodeBuild executes the repository’s buildspec.yml which installs dependencies and runs a helper that programmatically creates and executes the SageMaker Pipeline.Representative buildspec from the model‑build repository:
SageMaker Pipelines: Processing → Training → Evaluation → Model Registry
Open SageMaker Pipelines in the console to inspect the generated pipeline and its execution. A typical pipeline graph includes nodes for Preprocess, Train, Evaluate, a model‑quality check (e.g., CheckMSE), and RegisterModel.
Click into Processing and Training jobs from the execution view to inspect logs, instance types, role ARNs, and S3 artifact locations. Example pipeline graph and evaluation details:
When complete, Processing jobs appear in the Processing Jobs list.
If evaluation conditions meet the template thresholds (for example, MSE below the configured limit), the pipeline registers a model package in the SageMaker Model Registry. By default (in this template), new model packages enter the registry in a “Pending manual approval” state.
At this point the model build pipeline has executed end‑to‑end: Source → CodeBuild → run‑pipeline (SageMaker Pipeline) → Processing/Training/Eval → Model Registry.
Approve the Model; Trigger the Model Deploy Pipeline
Approving the model package in the Model Registry will trigger the deploy pipeline if an EventBridge rule or pipeline trigger is configured. Approve the model in the SageMaker console by changing the model package status from pending to approved and adding an optional comment.
Approving the model triggers the model‑deploy CodePipeline which runs a CodeBuild project. That build prepares CloudFormation templates for staging and production, packages artifacts, and emits deployable artifacts consumed by CloudFormation in later pipeline stages.Typical tasks in the model‑deploy buildspec:
Run a helper (e.g., build.py) to generate endpoint configuration CloudFormation templates for staging and prod.
Package templates with aws cloudformation package and upload artifacts to the project S3 bucket.
Emit packaged template files as pipeline artifacts.
When the deploy stage runs, CodePipeline starts CloudFormation stacks that create the SageMaker Model, EndpointConfiguration, and Endpoint resources. The pipeline deploys staging first, executes tests (often a CodeBuild smoke test), and then pauses for a manual approval before creating the production stack.You can approve the pipeline deployment from the CodePipeline console (Review & Approve).
After approval, CodePipeline proceeds to create the production CloudFormation stack and production endpoint. Monitor CloudFormation events and the SageMaker Endpoints page while the endpoint warms up (this can take several minutes).
Both pipelines use CodeBuild. In the CodeBuild console you can open each build project to examine its environment, source settings, and buildspec.
Buildspecs are the authoritative instructions for CodeBuild: they install dependencies, run helper scripts or run-pipeline, and package or deploy CloudFormation templates.
If you prefer to use GitHub, GitLab, Bitbucket, or GitHub Enterprise, choose a template that supports third‑party Git. You must first create a CodeStar connection between your AWS account and the Git provider (Developer Tools → Settings → Connections). The project creation dialog will then request repository URLs and branch names instead of creating CodeCommit repos.
Manage or create CodeStar connections in the AWS console to link your Git provider; the connection name is referenced in the SageMaker Project dialog.
Reviewed SageMaker Project templates and selected one that provisions both model build and model deploy CI/CD pipelines.
Created a SageMaker Project, which launched a CloudFormation stack to provision S3, source repos, CodeBuild projects, CodePipeline pipelines, SageMaker Pipelines, and IAM roles.
Followed the CI/CD flow: CodeCommit → CodePipeline → CodeBuild → run‑pipeline (SageMaker Pipeline) → Processing/Training/Evaluation → Model Registry.
Approved a registered model to trigger the deploy pipeline; observed staging deployment, automated tests, and a manual approval gate before production.
This project template provides a fully integrated CI/CD workflow for ML on AWS. For production systems:
Prefer third‑party Git hosts and CodeStar connections,
Define branching and approval policies,
Consider cross‑account deployments and more robust testing (canaries, blue/green),
Add monitoring and Model Monitor checks for production model behavior.
