Skip to main content
In this lesson, we review Infrastructure as Code (IaC) options for provisioning and operating machine learning (ML) environments on AWS. Defining infrastructure as code enables repeatability, automation, and scalability—critical for consistent training, testing, and production ML workflows. IaC helps automate complex training environments, standardize inference deployments, and improve observability and compliance across the ML lifecycle.
The image presents information about Infrastructure as Code (IaC) options for machine learning workloads, highlighting improvements in repeatability, scalability, and automation.
IaC centralizes tasks such as provisioning training compute, configuring data stores and networking, and deploying inference endpoints and monitoring pipelines. This centralization reduces manual errors and accelerates delivery of ML systems into production.
The image is a slide titled "ML Mission Control: IaC Options for ML Workloads" highlighting how Infrastructure as Code (IaC) integrates with machine learning lifecycles, specifically in provisioning training environments and deploying inference endpoints.
Common IaC tools you’ll encounter when building ML systems on AWS:
  • AWS CloudFormation — declarative templates (YAML/JSON) to provision resources.
  • AWS Cloud Development Kit (CDK) — code-first approach using languages like Python or TypeScript; synthesizes to CloudFormation.
  • AWS Command Line Interface (CLI) — direct API-level control for scripts and ad-hoc automation.
The image displays options for Infrastructure as Code (IaC) tools provided by AWS for machine learning workloads, including CloudFormation, CDK, and CLI.
Why use IaC for ML? The primary benefits include:
  • Consistency: identical environments across development, staging, and production.
  • Automation: orchestrated SageMaker jobs, pipelines, and endpoints.
  • Scalability: templates/constructs that adapt to load and resource needs.
  • Reproducibility: recreate experiments and pipelines for debugging and compliance.
The image outlines four reasons for using Infrastructure as Code (IAC) for machine learning workloads: Consistency, Automation, Scalability, and Reproducibility. Each reason includes a brief description and an icon.
Below is a concise comparison to help you choose the right IaC approach for ML workloads. Below we review each option in more detail and show how it fits typical ML workflows on AWS.

​
AWS CloudFormation

AWS CloudFormation uses declarative templates (YAML/JSON) to define and provision AWS resources. It’s a good fit when you need repeatable, auditable ML infrastructure such as SageMaker endpoints, training pipelines, and accompanying networking and IAM. Typical workflow:
  • Store CloudFormation templates in a deployment repository.
  • Build a CI/CD pipeline (e.g., CodeCommit, CodeBuild, CodePipeline) to automate stack deployments.
  • CloudFormation reads templates and provisions resources (S3 buckets for artifacts, IAM roles, VPCs/security groups, SageMaker model and endpoint configuration, etc.).
CloudFormation pipelines can automate model promotion across environments (for example, dev → staging → prod) and enable controlled updates with ChangeSets.
The image presents key features of AWS CloudFormation for ML, including parameters and mappings, nested stacks/modules, change sets, and drift detection and rollbacks, with automation through CodeCommit/CodePipeline or CDK.
Key CloudFormation features that help ML workflows:
  • Parameters and mappings for reusable, environment-specific templates.
  • Nested stacks or modular templates to decompose complex architectures.
  • ChangeSets to preview and approve resource changes.
  • Drift detection to find manual configuration changes outside IaC.
  • Automated rollbacks for failed deployments.
Best practices for CloudFormation with ML:
  • Enforce least-privilege IAM policies for SageMaker and execution roles.
  • Store model artifacts and large binaries in Amazon S3 instead of embedding them in templates.
  • Avoid launching long-running training jobs directly within stack deployments—trigger training from CI/CD pipelines to avoid stack timeouts and complicated rollbacks.
  • Use KMS encryption for sensitive data and VPC endpoints for private access to S3 and ECR.
The image lists best practices for AWS CloudFormation in machine learning, including using least-privilege IAM, storing model artifacts in S3, avoiding long training in stacks, and enabling KMS encryption.

​
AWS CDK

AWS CDK provides a code-first developer experience: you define infrastructure using languages like Python or TypeScript, then the CDK synthesizes your code into CloudFormation templates and deploys them. Typical flow:
  • Write CDK constructs in your chosen language.
  • Synthesize to CloudFormation with cdk synth.
  • Deploy with cdk deploy which applies the generated CloudFormation templates to create SageMaker models, endpoints, VPCs, and other resources.
The image is a flowchart titled "AWS CDK for ML Workloads," showing a process from developer to SageMaker model involving AWS CDK and CloudFormation execution.
Common CDK use cases for ML:
  • Define SageMaker Pipelines, training jobs, and inference endpoints directly in Python/TypeScript.
  • Integrate infrastructure definitions into CI/CD pipelines for fully automated deployments.
  • Create reusable, higher-level constructs to standardize architecture and reduce duplication across teams.
The image describes use cases for AWS CDK in machine learning workloads, highlighting the definition of SageMaker pipelines, CI/CD integration, and reuse of constructs for repeatable architectures.
Advantages of AWS CDK for ML:
  • Abstraction and reuse: encapsulate patterns (for example, a standard inference endpoint) into reusable constructs.
  • Developer-friendly workflows: leverage standard languages, unit tests, and IDE tooling.
  • Produces CloudFormation templates under the hood, so you retain CloudFormation features such as change sets and rollbacks.
The image highlights the advantages of AWS CDK for ML workloads: abstraction and reuse, software-friendly workflows, and CloudFormation generation.
Operational and exam tips for CDK and ML:
  • Treat CDK code like application code: use version control, code reviews, and unit tests.
  • Never hard-code secrets or large model weights in CDK code; reference S3, AWS Secrets Manager, or SSM Parameter Store instead.
  • Remember CDK synthesizes to CloudFormation—understand both layers when troubleshooting deployments.
The image provides exam tips for using AWS CDK with machine learning workloads, emphasizing code-based infrastructure, repeatable code over templates, using S3 and Secrets Manager over hardcoding, and synthesizing to CloudFormation.

​
AWS CLI

The AWS CLI maps directly to AWS APIs and is ideal for rapid prototyping, ad-hoc tasks, and simple automation scripts. Unlike CDK or CloudFormation, the CLI has no intermediate template layer.
The image is a flowchart illustrating the use of AWS CLI for infrastructure as code (IAC), showing how developers interact with AWS services via CLI to initiate API calls for creating a training job, a SageMaker model, and a SageMaker endpoint.
Best uses for the CLI:
  • Quickly prototype SageMaker training jobs and endpoints.
  • Automate small ML workflows with shell or Python scripts.
  • Validate CloudFormation/CDK resources by calling APIs directly before formalizing into templates or constructs.
The image outlines use cases for AWS CLI in Infrastructure as Code (IAC), highlighting rapid prototyping with SageMaker, automating ML workflows with shell scripts, and testing CloudFormation/CDK stacks.
Strengths and limitations of the CLI:
  • Strengths: fast, flexible, and scriptable—great for one-off tasks and experiments.
  • Limitations: lacks drift detection, modularity, and automated rollbacks—so it’s less suitable for large, production-grade infrastructure compared to CloudFormation/CDK.
The image is a slide discussing AWS CLI for IAC, highlighting its strengths, such as flexibility and scriptability, and its limitations, like its unsuitability for large production ML infrastructure.
Example: simplified AWS CLI command to create a SageMaker training job (illustrative):
(Adapt the JSON, ARNs, and S3 URIs to your environment.)

​
Security, Cost, and Operations

Security: apply multiple layers of protection when defining infrastructure as code:
  • IAM: enforce least privilege for training jobs, pipelines, and deployment roles.
  • Encryption: enable KMS encryption for data at rest and TLS for data in transit.
  • Network: restrict public access and use VPC endpoints for S3, ECR, and other services.
  • Secrets: use AWS Secrets Manager or SSM Parameter Store—never hard-code credentials.
  • Audit: enable CloudTrail and AWS Config to detect and alert on unexpected changes.
The image is a diagram titled "Security in IAC for ML" that outlines five key security components: Network, Secrets, Audit, IAM, and Encryption. Each component includes a brief description of its function in enhancing security.
Cost optimization and resilience strategies:
  • Right-size compute for each environment and use Spot Instances for training where appropriate.
  • Enforce resource tagging in IaC to enable cost allocation and reporting.
  • Disaster recovery: IaC enables automated rebuilds in another region—version control your recovery playbooks and data replication policies.
  • CI/CD integration: automate infrastructure deployments and model promotions with pipelines; leverage ChangeSets for safe updates and rollbacks.
The image outlines "Cost Optimization With IaC," featuring three sections: cost optimization, disaster recovery, and CI/CD versioning, each with associated strategies and actions.
When building production ML pipelines, combine IaC with CI/CD and observability: treat infrastructure code as part of the application lifecycle, run automated tests, and monitor deployments to detect drift or regressions early.

​
Common Anti-Patterns to Avoid

Avoid these pitfalls when applying IaC to ML workloads:
  • Hard-coding credentials or model artifacts directly into code or templates.
  • Using wildcard IAM policies (for example, *) instead of least-privilege policies.
  • Making manual, one-off configuration changes that cause drift from declared state.
  • Over-provisioning resources that inflate costs.
  • Ignoring ChangeSets and rollbacks—these help ensure safe production updates.
The image lists four anti-patterns to avoid: hardcoding credentials, wildcard IAM policies, manual configuration, and over-provisioning resources.
Never embed secrets or model weights in IaC artifacts. Use managed secret stores and S3 references. Hard-coding sensitive data is the most common cause of costly security incidents.

​
Key Takeaways

  • IaC is essential for consistent, automated, and scalable ML deployments on AWS.
  • Choose the right tool for the job: CloudFormation for declarative stacks, CDK for a code-first developer experience, and AWS CLI for quick prototypes and ad-hoc automation.
  • Security practices: enforce least-privilege IAM, enable encryption, isolate resources in VPCs, and use managed secret stores.
  • Operational best practices: right-size and tag resources to control cost; plan for disaster recovery; integrate IaC into CI/CD pipelines for traceability and safe rollbacks.
  • Avoid anti-patterns like hard-coding secrets, wildcard permissions, and manual configuration drift.
The image is a summary slide about the importance of Infrastructure as Code (IaC) for AWS Machine Learning, covering tools, security, operations, and avoiding anti-patterns.
Use these resources to deepen your knowledge and implement secure, repeatable IaC practices for your ML workloads on AWS.

Watch Video