> ## Documentation Index > Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt > Use this file to discover all available pages before exploring further. # Training and Fine tuning Process for Foundation Models > Overview of adapting foundation models through pre-training, fine-tuning, continuous pre-training, PEFT and RLHF plus practical tooling and best practices for data, evaluation, and deployment In this lesson we deepen the practical understanding of how foundation models are adapted from general-purpose capabilities to task- or domain-specific behavior. You already know the base vocabulary — pre-training and fine-tuning — and here we clarify those processes, introduce continuous pre-training, and summarize production-ready techniques and tools (with AWS examples such as [SageMaker](https://learn.kodekloud.com/user/courses/aws-sagemaker) and [Bedrock](https://learn.kodekloud.com/user/courses/introduction-to-amazon-bedrock)). This article presents a concise technical overview of customizing foundation models—focusing on practical patterns, trade-offs, and tooling rather than detailed mathematics. Use it as a checklist when designing model adaptation pipelines. Quick taxonomy * Pre-training: Train a large model on massive, mostly unstructured data to learn general representations. * Fine-tuning: Adapt a pre-trained model to a specific task or domain using labeled or instruction-style data. * Continuous pre-training: Periodically retrain (or adapt) a foundation model on new data to keep it current and reduce model drift. A dark presentation slide titled "Introduction to Foundation Models" showing three numbered colored bars labeled: 01 Pre-training, 02 Fine-tuning, and 03 Continuous Pre-training. Each item is displayed as a horizontal progress-style bar on the slide. What is pre-training? * Pre-training is the compute- and data-intensive phase where the model learns broad, general-purpose knowledge from very large corpora (text, images, audio, etc.). * Typical self-supervised objectives include next-token prediction and masked-token prediction for language models, contrastive objectives for vision, and multimodal alignment losses for cross-modal models. * Outcome: rich, transferable representations that serve as the backbone for many downstream tasks. High token counts and long training runs are common. What is fine-tuning? * Fine-tuning adapts a pre-trained model to a narrower task or vertical. It usually uses supervised labeled data, but instruction-based examples and preference data are also common. * Fine-tuning is substantially cheaper than full pre-training and can yield large performance gains on target tasks when hyperparameters and data are well-chosen. * Common use cases: classification, summarization, domain-adapted language, code generation, and retrieval-augmented generation. A diagram titled "Transition to Fine-Tuning" showing documents, images, and audio feeding into a Foundation Model that provides general knowledge, which is then fine-tuned into a task-specific model. The flow is visualized with arrows from data sources → foundation model → general knowledge → fine-tuned task-specific model. Fine-tuning characteristics * Use-case specificity: Fine-tuning focuses model capacity on a defined task (e.g., translation, summarization, classification), usually via labeled examples. * Reduced compute: Less expensive than pre-training, but requires careful tuning of learning rate, batch size, scheduling, and early stopping. * Instruction-based fine-tuning: Training with instruction–response pairs helps models follow natural-language directives across multiple tasks. A slide titled "What is Fine-Tuning?" with two dark panels. The left panel shows gear icons and notes fine-tuning uses labeled datasets in supervised learning, and the right panel shows a rising bar chart with text saying it improves model performance for specific tasks. Pre-training vs. fine-tuning — quick comparison * Pre-training: broad, general-purpose learning from unstructured or weakly labeled data; large compute and long runs. * Fine-tuning: targeted adaptation using supervised or instruction-style examples; shorter runs and smaller compute footprint. An infographic slide titled "Difference Between Pre-Training and Fine-Tuning" showing two side-by-side panels: a dark blue panel for Pre-training ("General-purpose learning from unstructured data") and a light blue panel for Fine-tuning ("Task-specific learning with labeled examples"). A rounded caption bar below notes that each process contributes to making the model versatile and accurate. Comparison table — when to use each | Stage | Typical objective | Compute & data | Best for | | - | - | - | - | | Pre-training | Learn general representations from massive corpora | Very high compute, huge token counts | Building a new foundation model or large cross-domain capabilities | | Fine-tuning | Specialize model for a task or domain | Moderate compute, focused dataset | Task-specific performance (classification, summarization, domain language) | | Continuous pre-training | Keep base model up-to-date on new data | Varies (scheduled retraining) | Data freshness, adapting to evolving domains | Instruction-based fine-tuning * Instruction tuning trains the model on many input–output pairs that explicitly include a human-style instruction, improving the model’s ability to follow directives (summarize, translate, generate code, answer questions). * This method often yields more useful and controllable outputs for end-users than task-only supervised fine-tuning. A slide titled "Instruction-Based Fine-Tuning" with three colored circular icons. They represent Summarization, Translation, and Code generation. Catastrophic forgetting * Over-specializing a model to a single task can overwrite the general knowledge learned during pre-training; this is called catastrophic forgetting. * The trade-off: higher performance on the target task vs. losing broad capabilities that may be needed later. A dark-themed slide titled "Catastrophic Forgetting in Fine-Tuning" showing a flow from "Initial Knowledge" → "Modified Weights" → "Lost Knowledge" to illustrate how fine-tuning on a single task can overwrite previously learned tasks. A caption at the bottom notes it's important to balance fine-tuning across tasks. Catastrophic forgetting can degrade multi-task performance if not managed. Mitigation strategies include multi-task training, regularization (e.g., weight decay, elastic weight consolidation), and parameter-efficient fine-tuning so that base weights remain largely intact. Parameter-Efficient Fine-Tuning (PEFT) * PEFT updates a small subset of parameters (or adds small trainable modules) instead of re-training all weights, dramatically reducing compute, storage, and deployment costs. * Popular PEFT families: * Low-Rank Adaptation (LoRA): Keep backbone weights frozen and add low-rank trainable matrices (A, B) that inject a low-rank delta into targeted layers—efficient for many downstream tasks. * Adapters / Prefix Tuning / Prompt Tuning: Insert small trainable modules or learnable prompts into the model architecture or token stream. * Representation Fine-Tuning (ReFT): Modify internal activations/representational patterns rather than bulk parameter matrices; can help on structured reasoning tasks. A presentation slide titled "PEFT Techniques: LoRA and ReFT" showing two boxed panels: the left explains LoRA (Low-Rank Adaptation) and freezing original weights while adding trainable low-rank matrices, and the right describes ReFT (Representation Fine-Tuning) as modifying model representations rather than weights. Notes on LoRA vs ReFT * LoRA: excellent for low-cost fine-tuning across many tasks with limited compute; adapter weights are small and easy to store/deploy. * ReFT and adapter-style methods: may produce better results for tasks requiring structured outputs or complex internal transformations; empirical performance depends on the problem and dataset. Multitask fine-tuning * Train on multiple tasks in parallel (each with labeled examples) to maintain generality and reduce catastrophic forgetting. * Benefits: shared representations across tasks, better transfer, and a single adaptable model for related subtasks. An infographic titled "Multitask Fine-Tuning" shows three tasks feeding into simultaneous training that produces a fine-tuned model. The diagram notes benefits: it avoids catastrophic forgetting and produces versatile models for various tasks. Domain-specific fine-tuning * Domain fine-tuning specializes models to verticals (finance, healthcare, legal). This focuses on terminology, style, constraints, and common subtasks so the model understands domain-specific nuance and compliance requirements. * Domain adaptation often includes both supervised signals and curated unlabeled corpora for targeted continuous pre-training. A slide titled "Domain-Specific Fine-Tuning" showing a flow diagram: a foundation model icon on the left, a central box labeled "Domain-Specific Datasets," and an adapted domain-specific model icon on the right. Arrows indicate domain-specific fine-tuning into the datasets and a training process producing the adapted model. Practical tools and workflows (AWS examples) Use the table below to map common tooling to the tasks in a model customization pipeline. | Tool | Primary purpose | Notes / links | | - | - | - | | SageMaker JumpStart | Example notebooks & fine-tuning recipes | Pre-built examples for common models — [JumpStart](https://learn.kodekloud.com/user/courses/aws-sagemaker) | | Bedrock | Host foundation models & customization | Managed model hosting and customization workflows — [Bedrock](https://learn.kodekloud.com/user/courses/introduction-to-amazon-bedrock) | | SageMaker Studio | Interactive experimentation & notebooks | Fast iteration and data munging — [Studio](https://learn.kodekloud.com/user/courses/aws-sagemaker) | | Data Wrangler / Athena / Glue / EMR | ETL and large-scale data prep | Use for collecting, cleaning, and transforming training corpora | | SageMaker Feature Store | Centralized feature storage | Shareable features for training and inference — [Feature Store](https://learn.kodekloud.com/user/courses/aws-sagemaker) | | SageMaker Clarify | Bias detection & explainability | Detect dataset/model bias and generate reports — [Clarify](https://learn.kodekloud.com/user/courses/aws-sagemaker) | | SageMaker Ground Truth | Managed data labeling | Labeling workflows for supervised datasets — [Ground Truth](https://learn.kodekloud.com/user/courses/aws-sagemaker) | Slide titled "SageMaker Clarify for Bias Detection and Governance" showing the Amazon SageMaker Clarify logo and two bullet points. It notes the tool detects dataset biases for fair model performance and ensures no group is disproportionately represented. Reinforcement Learning from Human Feedback (RLHF) * RLHF shapes model outputs through human preference data: raters compare outputs, a reward model is trained, and reinforcement learning updates the model to maximize the learned reward. * Commonly used to align models with human preferences, safety constraints, and style guidelines (e.g., helpfulness and harmlessness). Data preparation and lifecycle High-quality data is the foundation of reliable fine-tuning and continuous pre-training. Key practices: * Clean, deduplicate, and validate data sources. * Ensure correct and consistent labeling for supervised tasks. * Design prompt templates and instruction examples carefully for instruction tuning. * Maintain dataset splits: training, validation (for hyperparameter tuning), and a held-out test set for final evaluation. * Track loss curves, validation metrics, and use early stopping to avoid overfitting. * For continuous pre-training, implement a curated pipeline to vet and version incoming data. Best practices for evaluation * Choose metrics appropriate to the task: accuracy, F1, BLEU, ROUGE, perplexity, exact match, or custom business KPIs. * Monitor validation loss and task metrics during training. * Evaluate final models on a never-seen test set and perform human evaluation when automated metrics are insufficient. * Use fairness and bias detection tools (e.g., SageMaker Clarify) and incorporate safety checks. Operational considerations * Version control: track model checkpoints, PEFT adapters, and dataset versions. * Efficient storage: store PEFT adapters independently of backbone weights so many variants don’t duplicate the full model. * Deployment and monitoring: log model outputs, detect drift, and schedule retraining or continuous pre-training according to data freshness needs and business SLAs. Closing summary * Pre-training creates general-purpose capabilities; fine-tuning (including PEFT) efficiently tailors those capabilities to tasks and domains. * Continuous pre-training helps maintain model relevance but requires strict data curation and governance. * Choose tuning approaches (full fine-tune, PEFT, multi-task, RLHF) based on compute budget, desired model behavior, and acceptable risk; always start with data quality and evaluation criteria. When planning customizations, start with data curation and clear evaluation metrics. Align your approach (full fine-tune vs. PEFT vs. RLHF) to your compute constraints, deployment strategy, and the complexity of the behavior you need the model to produce. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.