Skip to main content
Welcome. This lesson covers source citation and data lineage — core concepts for governance, reproducibility, and compliance in Amazon SageMaker-based ML workflows. These topics are exam-relevant and are fundamental for building trustworthy generative AI systems.
Understanding data lineage, model versioning, and source citation is essential for reproducibility, compliance, and secure production ML.
Why source citation and data lineage matter
  • You must know where data originated, how it was processed, and where resulting artifacts are stored.
  • Data lineage records the origin of data and every transformation applied during its lifecycle.
  • Note: “feature” refers to a data attribute used by a model, not a software feature.
A slide titled "Data Lineage in AI – Importance" showing a linear flow of three connected boxes: Data Source, Data Processing, and Data Storage, each with a corresponding icon. The diagram illustrates the pipeline from data origin through processing to storage.
Core benefits enabled by lineage and citation
  • Data integrity: demonstrate that data has not been tampered with.
  • Compliance and auditability: provide an auditable trail for regulators and internal auditors.
  • Model reproducibility: re-create models, experiments, and production runs with confidence.
Think of lineage and source citation as version control plus an audit trail for data, models, and runtime environments.
A dark presentation slide titled "Data Lineage in AI – Importance." It shows three numbered panels: 01 Data Integrity (shield/lock icon), 02 Compliance (checklist icon), and 03 Model Reproducibility (gear icon).
What to track to ensure reproducibility To reliably reproduce a model or investigate issues, track and version the following artifacts:
  • Model binaries and weights
  • Datasets (raw and processed)
  • Hyperparameters, training run metadata, and evaluation metrics
  • Container images and runtime environment definitions
  • Source code, preprocessing scripts, and transformation logic
The image is a slide titled "Machine Learning — Need for Tracking Artifacts" showing a semicircular diagram of items to track (Source Code, Datasets, Container Images, Model Versions) with a central icon labeled "Artifacts to Track." It uses simple line icons and a dark blue background.
Version control practices (recommended)
  • Code: Store training and inference code in a Git repo (e.g., GitHub, AWS CodeCommit). Tag releases and keep commit hashes with experiment metadata.
  • Datasets: Use storage that supports immutability and versioning (e.g., Amazon S3 with versioning or a dataset registry). Include metadata, tags, and unique prefixes to identify dataset versions.
  • Containers: Publish runtime images to a registry (e.g., Amazon ECR). Use immutable references (digests) or incremented version tags rather than overwriting tags in place.
A slide titled "Version Control for Code and Datasets" showing two panels: "Code Repositories" with GitHub and AWS CodeCommit, and "Dataset Storage" with Amazon S3. It highlights versioned scripts for training/inference and unique dataset prefixes for identification and reproducibility.
Treat container images as immutable artifacts: reference images by digest or incremented version, and avoid in-place tag mutations so exact runtime environments remain reproducible.
Container-image tracking example Each container image should be uniquely identifiable (tag and digest). Publish new builds with new tags/digests so you can always retrieve the exact image used for a training job or inference endpoint.
A slide titled "Tracking Container Images With Amazon Elastic Container Registry (ECR)" showing a diagram of an ECR registry containing two container images (Tag: Training_v1 and Tag: Inference_v1) and a note that exact container images make it easy to reproduce model environments.
SageMaker subservices that enable source citation and lineage SageMaker provides several complementary capabilities that together support governance, traceability, and reproducibility:
  • Model Registry: centralizes trained models, groups them into model packages or model groups, and manages model versions (v1, v2, …). Stores associated metadata such as hyperparameters and metrics.
  • Model Cards: human-readable snapshots describing training details, intended use, limitations, data sources, and evaluation metrics for transparency and compliance.
  • Lineage Tracking: records relationships and the workflow graph connecting datasets, training jobs, container images, and outputs so you can query and reconstruct workflows.
  • Feature Store: a centralized feature repository that supports feature discovery, consistency between training and inference, point-in-time queries, and feature lineage (provenance of how features were derived).
A slide titled "SageMaker Model Registry for Model Versioning" showing diagrammed model groups and versions that include metadata (metrics and hyperparameters). The right side lists registry features like model organization, metadata, and version control.
Lineage enables relationship queries With lineage data you can answer questions such as:
  • Which models were trained using a specific dataset?
  • Which datasets were used with a particular container image?
  • What are the dependencies of a model (libraries, feature transforms, preprocessing steps)?
Lineage makes it possible to reproduce a model’s production path and to investigate issues or regulatory inquiries.
A presentation slide titled "SageMaker Lineage Tracking for Data Lineage and Workflow Documentation." Below the title are three numbered panels: 01 "Establishes governance," 02 "Enables traceability," and 03 "Maintains historical records," each with a simple icon.
Feature Store and feature lineage SageMaker Feature Store provides:
  • Reusable, discoverable feature sets for training and inference.
  • Consistent feature values across training runs and production inference.
  • Feature lineage: provenance for raw input data, references to transformation code, and metadata that help enforce integrity and compliance.
Feature Store also supports cataloging and point-in-time queries, which help ensure that the features used for training match those served in production.
A slide titled "SageMaker Feature Store — Feature Lineage" showing three panels with icons and captions: "Tracks lineage and raw data processing," "Captures execution code and data sources," and "Ensures data integrity and compliance."
A presentation slide titled "SageMaker Feature Store — Data Cataloging" showing a screenshot of the Feature Store UI on the left. On the right are three numbered points describing benefits: storing/cataloging ML features, metadata-driven tracking and reuse, and ensuring consistency and traceability.
Quick reference table — SageMaker capabilities Further reading and references Summary SageMaker’s Model Registry, Model Cards, Lineage Tracking, and Feature Store together provide the building blocks for source citation and data lineage. They enable reproducibility, governance, traceability, and compliance for production ML and generative AI. Know the role of each subservice — these concepts are exam-relevant. Thanks for reading.

Watch Video