Skip to main content
Secure network design is essential when you run machine learning workloads in the cloud. This guide explains how to apply layered network access controls to protect ML data, models, and compute—focusing on AWS primitives and recommended configurations for SageMaker, S3, and EC2.
This lesson explains how VPCs, security groups, VPC endpoints, IAM roles/policies, and AWS Config combine to create defense-in-depth for ML resources such as SageMaker, S3, and EC2.
The image illustrates network access controls for ML resources using AWS VPC and AWS Shield, displayed with respective icons.

​
Why network access controls matter

Start with the core ML services:
  • SageMaker — training and hosting managed by AWS.
  • S3 — storage for datasets, model artifacts, and checkpoints.
  • EC2 — custom compute for training, preprocessing, or serving.
Applying network controls (VPCs, security groups, VPC endpoints) prevents unauthorized access and reduces the attack surface. Complement these controls with AWS Shield for DDoS protection at the edge and IAM for fine-grained authorization.
The image is an illustration explaining the use of network access controls for machine learning (ML) resources, featuring icons representing ML tools like SageMaker, S3, and EC2, followed by network controls such as VPC, SGS, and Endpoints, leading to secure and isolated access.

​
Core AWS network primitives for ML

Use the following AWS building blocks to secure ML workloads. They work together to isolate traffic, enforce least privilege, and keep data on the AWS network.
The image lists AWS tools for network access controls, including VPC, Security Group, and VPC Endpoint.

​
Design pattern: place ML resources inside a VPC

Best practice: run SageMaker training/hosting, EC2 instances, and any custom endpoints inside a VPC with private subnets that do not route directly to an Internet Gateway. This approach keeps both management and data-plane traffic within your controlled network boundaries and reduces exposure to the public internet.
  • Use private subnets for compute and model hosting.
  • Use NAT/egress proxies only when necessary and lock them down.
  • Centralize shared services (artifact stores, logging, model registries) in controlled VPCs or via VPC peering/Transit Gateway.
The image is a diagram illustrating the configuration of a VPC for machine learning resources, showing the connection between SageMaker, a VPC (Private Network), and isolated subnets with no internet.

​
Security groups: allow only the minimum

Security groups are your first line of defense for instance-level traffic:
  • Permit inbound traffic only from trusted IP ranges or other security groups (use security group IDs for cross-instance communication).
  • Limit access to required ports (e.g., 443 for HTTPS).
  • Avoid 0.0.0.0/0 or overly broad CIDR blocks on sensitive ports.
If you need subnet-level, stateless controls use Network ACLs, but favor security groups for stateful, instance-level protection and flexibility.
The image is a diagram showing the configuration of security groups for machine learning resources, allowing only trusted IPs/ports and blocking all other traffic.

​
VPC endpoints: keep traffic on the AWS network

Routing service traffic over the public internet increases exposure. Use VPC endpoints to keep communication internal to AWS:
  • Gateway endpoints — for S3 and DynamoDB. Add a route in your route table so resources in private subnets reach S3 without leaving the AWS network.
  • Interface endpoints (AWS PrivateLink) — create private ENIs to access many AWS service APIs and supported partner services.
Using endpoints ensures model artifacts and data transferred between SageMaker and S3 (or other AWS services) do not traverse the public internet.
The image is a diagram illustrating the configuration of VPC endpoints for machine learning resources, showing a flow from SageMaker to an S3 bucket via a VPC Endpoint.

​
Integrating IAM with network controls

Network controls and IAM should be used together for defense-in-depth:
  • Create IAM roles with least privilege for SageMaker training jobs, inference, and any EC2 instances.
  • Use resource-based policies (S3 bucket policies, VPC endpoint policies) to restrict which principals and networks can access artifacts.
  • Combine IAM restrictions with VPC placement and security group rules so compromised credentials alone are not sufficient to exfiltrate sensitive data.
The image illustrates integrating IAM with network controls, showing a flow from IAM Role to VPC + Security Groups, and finally to accessing ML resources securely.

​
Monitoring, compliance, and drift detection

Continuously validate and audit network configurations:
  • Use AWS Config to codify desired states for VPCs, subnets, security groups, and endpoints.
  • Monitor changes and get automated compliance reports and alerts for drift from the baseline.
  • Integrate AWS Config with AWS CloudWatch Events, Lambda, or Systems Manager Automation for automated remediation.
The image illustrates a process flow for "Security and Compliance With Network Controls," showing steps involving AWS Config, monitoring VPC and Security Groups, and generating a Compliance Report.
Anti-patterns to avoid: wide-open security groups (0.0.0.0/0 on sensitive ports), sending ML traffic over the public internet instead of using VPC endpoints, and assigning overly broad IAM admin permissions where scoped roles would suffice.

​
Anti-patterns (concise)

  • Allowing 0.0.0.0/0 on sensitive ports.
  • Not using VPC endpoints for S3/model traffic.
  • Granting broad administrative IAM permissions to service roles.

​
Summary: key action items

  1. Use VPCs and private subnets to isolate ML workloads from the public internet.
  2. Apply least-privilege security groups—allow only required sources and ports.
  3. Add VPC endpoints (gateway or interface) so traffic to AWS services stays private.
  4. Combine scoped IAM policies and roles with network controls for defense-in-depth.
  5. Continuously monitor configurations with AWS Config and remediate drift.
The image is a slide titled "Summary and Action Items" with five steps: using VPC for isolation, restricting SGs, adding VPC endpoints, combining IAM with network controls, and monitoring with AWS.

Watch Video