Skip to main content
In this lesson we cover AWS DataSync — a fully managed service that automates, accelerates, and secures data transfers between on‑premises storage (or other clouds) and AWS storage services. DataSync is optimized for large‑scale migrations, recurring syncs, and replication workflows, and includes features for verification, scheduling, incremental updates, and monitoring.
A diagram titled "What Is DataSync?" showing AWS DataSync as a central cloud icon linking an on‑premise storage box on the left to an Amazon Web Services box on the right with dotted, bidirectional arrows. The graphic illustrates data transfer between on‑premises infrastructure and AWS.

​
Core concepts

Use these core DataSync building blocks to model any transfer workflow:
  • Agent
    The DataSync agent is deployed as a virtual appliance in your environment (for example, VMware ESXi, Linux KVM, or Microsoft Hyper‑V). It mounts or connects to on‑premises storage and proxies data to AWS on behalf of the DataSync service.
  • Location
    A location represents a source or destination endpoint. Common examples include on‑premises NFS or SMB shares and AWS targets like Amazon S3, Amazon EFS, or Amazon FSx.
  • Task
    A task ties two locations together and defines transfer behavior: what metadata to preserve, whether to copy metadata only or content, how to handle deletions, and how to map POSIX permissions or Windows ACLs.
  • Task execution
    When you start a task (manually, via schedule, or via API), DataSync runs an execution. Each execution goes through lifecycle states you can monitor and troubleshoot.
A presentation slide titled "DataSync" showing four rounded panels labeled 01 Agent, 02 Location, 03 Task, and 04 Task Execution, each with a colorful circular icon. The Agent panel also lists supported platforms like VMware ESXi, Linux KVM, and Microsoft Hyper-V.

​
Task and execution states

Monitor both task-level and execution-level states to understand readiness and progress.
A presentation slide titled "Task and Execution Status" with a "Task Execution" banner and colorful circular icons labeled Queued, Launching, Preparing, Transferring, Verifying, Success, and Error.
An individual DataSync agent can run only one task execution at a time. If multiple tasks target the same agent, executions are queued and run sequentially.

​
Discovery and planning

Agents can run discovery (inventory) jobs to collect metadata and usage patterns from your on‑premises storage. The agent sends this metadata to the DataSync service; you can use it to:
  • Estimate data size, file counts, and transfer duration.
  • Choose an appropriate AWS storage target (EFS, S3, FSx) and lifecycle plan.
  • Plan bandwidth and scheduling for phased migrations.
Discovery improves migration planning by providing visibility into hot vs. cold data and access frequency.

​
Key features

DataSync provides built‑in capabilities to simplify large‑scale data movement:
  • High‑speed transfers optimized for bulk data migration.
  • Simple management via AWS Console, CLI, and APIs for creating tasks and automations.
  • Optional data integrity verification during or after transfers.
  • Scheduling and automation for recurring syncs.
  • Support for on‑premises protocols (NFS, SMB) and native AWS targets (S3, EFS, FSx).
  • Incremental transfers: after the initial copy, DataSync transfers only changed data.
  • Encryption in transit (TLS) and at rest (server‑side encryption with AWS KMS support).
  • Monitoring and logging through Amazon CloudWatch and AWS CloudTrail.
Best practices: run agents close to the source data to reduce latency, enable verification for critical migrations, and use incremental syncs for ongoing replication to save bandwidth.
A presentation slide titled "DataSync – Features" showing seven rounded feature cards with orange icons. Each card lists capabilities like High‑Speed Data Transfer, Simple Management, Data Integrity & Verification, Automation & Scheduling, Support for Multiple Protocols, Incremental Transfers, and Security & Encryption.

​
Common use cases

  • On‑premises → AWS migrations
    Migrate file and object datasets from on‑premises storage into AWS services such as Amazon EFS, Amazon S3, or Amazon FSx.
  • Region‑to‑region transfers
    Copy data across AWS Regions for disaster recovery, replication, or geographic distribution.
A diagram showing two AWS Regions containing AWS Storage Services (Amazon EFS, S3, and various Amazon FSx file systems) with DataSync Discovery appliances between them. The appliances are connected over TLS to enable discovery and transfer across the regions.
  • Cloud‑to‑AWS migrations
    Move data from other cloud providers (for example, Azure or Google Cloud) into AWS storage using DataSync agents that access external cloud storage.

​
Data protection and lifecycle

DataSync can ingest datasets into AWS for backup, replication, or archive workflows. Common patterns:
  • Replicate active datasets into AWS for disaster recovery.
  • Move cold data into lower‑cost storage classes (for example, Amazon S3 Glacier) for long‑term archiving.
  • Combine DataSync with lifecycle policies in S3/EFS to automate tiering and reduce on‑premises costs.
A network diagram showing how an on-premises or other-cloud storage connects via a DataSync agent (over TLS) to AWS DataSync Discovery and then to various AWS storage services (Amazon EFS, Amazon S3, and multiple Amazon FSx file systems). The graphic illustrates data transfer from an external cloud provider into an AWS region.

​
Integrations and monitoring

DataSync integrates with the major AWS storage services and AWS monitoring/auditing services: Use CloudWatch metrics and DataSync task logs to validate throughput, error rates, and completion status. Combine these signals with lifecycle rules in S3 or EFS to implement retention and archival policies.

Watch Video