# KodeKloud: Beta Courses NVIDIA Generative AI LLMs Associate Certification

## NVIDIA Generative AI LLMs Associate Certification

### Introduction 

- [Course Introduction](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Introduction/Course-Introduction/page.md): Introductory course for NVIDIA Generative AI LLMs Associate certification covering LLM fundamentals, retrieval augmented generation, vector search, prompt engineering, deployment, trustworthy AI, and hands on labs.

### Core Machine Learning and AI Knowledge

- [RAG Architecture Components](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/RAG-Architecture-Components/page.md): Describes RAG architecture components, including retrieval, chunking, vector databases and generation, and explains why gradient descent continual learning is not part of inference.
- [Python Package for Text Vectorization](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/Python-Package-for-Text-Vectorization/page.md): Guide comparing Python libraries for creating dense text embeddings, recommending spaCy with vector models and transformer alternatives, plus usage examples and similarity search tips.
- [LLM Evaluation Metric for Factual Accuracy](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/LLM-Evaluation-Metric-for-Factual-Accuracy/page.md): Explains using truthfulness scores to evaluate LLM factual accuracy and compares them to perplexity, BLEU, and speed
- [Selecting Embedding Models for RAG](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/Selecting-Embedding-Models-for-RAG/page.md): Selecting embedding models for RAG prioritizing semantic similarity to improve retrieval, with notes on tradeoffs and evaluation
- [Primary Advantage of RAG vs Standalone LLM](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/Primary-Advantage-of-RAG-vs-Standalone-LLM/page.md): Explains that Retrieval-Augmented Generation augments LLMs with external, updatable information to reduce hallucinations and keep responses current compared to standalone models.
- [Purpose of Few Shot Examples in Prompting](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/Purpose-of-Few-Shot-Examples-in-Prompting/page.md): Explains that few-shot examples in prompts steer LLMs’ output format, tone, and behavior via in-context learning without changing model weights or reducing inference cost.
- [Ensuring Chatbot Uses Specific Documentation](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/Ensuring-Chatbot-Uses-Specific-Documentation/page.md): Explains using retrieval-augmented generation to ground chatbots in company technical documentation and outlines RAG implementation, benefits, and comparisons.
- [Preparing Datasets for RAG Systems](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/Preparing-Datasets-for-RAG-Systems/page.md): Explains why chunking documents into appropriate segments is vital for RAG systems, affecting embeddings, retrieval relevance, and LLM answers, with best practices and practical tips
- [Understanding Attention Mechanism in Transformers](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/Understanding-Attention-Mechanism-in-Transformers/page.md): Explains attention in transformers, showing how queries keys and values compute learned weights so models focus on relevant tokens for each prediction.
- [Prompt Engineering for Text Summarization](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/Prompt-Engineering-for-Text-Summarization/page.md): Advice on few-shot prompt engineering using multiple diverse examples and clear instructions to produce consistent, concise text summaries, with best practices, examples, and implementation tips.
- [Balancing Metrics for LLM Production Deployment](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/Balancing-Metrics-for-LLM-Production-Deployment/page.md): Balancing model accuracy and computational cost when deploying LLMs to meet performance, latency, memory, and cost constraints while preserving response quality
- [Selecting Vector Databases for RAG](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/Selecting-Vector-Databases-for-RAG/page.md): Guidance on selecting vector databases for RAG, prioritizing effective approximate nearest neighbor search, index types, similarity metrics, scalability, filtering, and operational features
- [Defining Foundation Models LLMs](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/Defining-Foundation-Models-LLMs/page.md): Description of foundation models and LLMs pre-trained on broad data and adapted to diverse tasks through fine-tuning, prompting, and retrieval.
- [Advantage of LoRA vs Full Fine Tuning](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/Advantage-of-LoRA-vs-Full-Fine-Tuning/page.md): Explains how LoRA reduces memory and compute for fine-tuning large language models by training small low-rank adapter matrices instead of updating full model weights.
- [Python Package for Bag of Words Classification](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Core-Machine-Learning-and-AI-Knowledge/Python-Package-for-Bag-of-Words-Classification/page.md): Recommends scikit-learn for traditional bag-of-words text classification and contrasts it with deep learning frameworks like PyTorch and TensorFlow

### Data Analysis

- [Model Selection for Imbalanced Data Accuracy vs F1](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Data-Analysis/Model-Selection-for-Imbalanced-Data-Accuracy-vs-F1/page.md): Guidance on choosing LLM classifiers for imbalanced datasets, preferring F1 over accuracy and recommending per-class metrics, PR-AUC, and techniques to improve minority class performance
- [Visualizing Transformer Attention Weights](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Data-Analysis/Visualizing-Transformer-Attention-Weights/page.md): Guide to visualizing transformer attention weights using heat maps with explanations, plotting tips, and example code
- [Visualizing Demographic Bias in Training Data](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Data-Analysis/Visualizing-Demographic-Bias-in-Training-Data/page.md): Analyzing demographic bias in LLM training data and recommending stacked bar charts and best practices to identify underrepresented groups and visualize categorical distributions
- [Evaluating Statistical Significance of Fine Tuning](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Data-Analysis/Evaluating-Statistical-Significance-of-Fine-Tuning/page.md): Assessing LLM fine-tuning significance using hypothesis testing p-values, appropriate statistical tests, and complementary measures like effect sizes and confidence intervals.
- [Visualizing Learned Concepts in LLM Representations](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Data-Analysis/Visualizing-Learned-Concepts-in-LLM-Representations/page.md): Guidance on using UMAP and t-SNE to visualize semantic clusters in LLM embedding spaces.
- [Visualizing Correlations Between LLM Evaluation Metrics](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Data-Analysis/Visualizing-Correlations-Between-LLM-Evaluation-Metrics/page.md): Explains using correlation matrix heat maps to reveal pairwise relationships and redundancy among LLM evaluation metrics, with comparisons to other visualization techniques and interpretation tips
- [Visualizing Impact of Context Length on LLM Accuracy](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Data-Analysis/Visualizing-Impact-of-Context-Length-on-LLM-Accuracy/page.md): Analyzing how context length affects LLM accuracy and recommending scatterplots with trendlines and smoothing to reveal relationships, outliers, and diminishing returns

### Experimentation

- [Evaluating LLM Quality Without Single Correct Answer](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Experimentation/Evaluating-LLM-Quality-Without-Single-Correct-Answer/page.md): Evaluating open-ended language model quality when no single correct answer exists, advocating RLHF with human preference-based reward modeling, workflow, comparisons, and practical trade-offs.
- [Ensuring Validity in AB Testing LLM Prompts](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Experimentation/Ensuring-Validity-in-AB-Testing-LLM-Prompts/page.md): Guidelines for valid A/B testing of LLM prompts emphasizing using the same model, controlling settings, randomization, consistent evaluation, power analysis, logging, and replication
- [Rigorous Experimental Design for Zero Shot LLM Testing](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Experimentation/Rigorous-Experimental-Design-for-Zero-Shot-LLM-Testing/page.md): Guidelines for rigorous zero-shot evaluation of LLMs using strictly unseen test sets, predefined metrics, standardized preprocessing, controlled prompts, and reproducibility practices
- [Reliable Methodology for Evaluating LLM Hallucinations](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Experimentation/Reliable-Methodology-for-Evaluating-LLM-Hallucinations/page.md): Describes a hybrid methodology combining human expert annotation and automated knowledge-base checks to reliably evaluate LLM hallucinations, with recommended metrics and experimental practices
- [Determining Optimal LLM Context Window Size](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Experimentation/Determining-Optimal-LLM-Context-Window-Size/page.md): Guidelines for experimentally determining the optimal LLM context window by systematically testing varied window sizes and measuring model and system metrics to balance performance, latency, memory, and cost.
- [Evaluating LLM Bias Across Demographic Groups](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Experimentation/Evaluating-LLM-Bias-Across-Demographic-Groups/page.md): Guidelines for detecting and measuring LLM performance disparities across demographic groups via stratified evaluations, group-level metrics, statistical testing, and privacy-aware practices.
- [Reliable Human Evaluation Protocol for LLM Outputs](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Experimentation/Reliable-Human-Evaluation-Protocol-for-LLM-Outputs/page.md): Guidelines for reliable human evaluation of LLM outputs using diverse evaluator panels, clear rubrics, annotator training, multiple ratings, and inter-rater reliability metrics.
- [Robust Experimental Control for Measuring Fine Tuning Impact](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Experimentation/Robust-Experimental-Control-for-Measuring-Fine-Tuning-Impact/page.md): Evaluating fine-tuning effects on LLMs by comparing models to their pre-fine-tuning baselines, using held-out test sets and rigorous controls to isolate causal performance changes.
- [Isolating Impact of Training Dataset Size on LLM Performance](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Experimentation/Isolating-Impact-of-Training-Dataset-Size-on-LLM-Performance/page.md): Examines how training dataset size affects LLM performance by comparing identical architectures trained on progressively larger data subsets while controlling confounding variables.
- [Reliable Baseline for Evaluating LLM Task Performance](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Experimentation/Reliable-Baseline-for-Evaluating-LLM-Task-Performance/page.md): Guidance on selecting reliable baselines for LLM task evaluation, emphasizing human performance, SOTA comparisons, controlled model variants, and best practices for fair measurement
- [Methodologically Sound Ablation Study Approach for LLMs](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Experimentation/Methodologically-Sound-Ablation-Study-Approach-for-LLMs/page.md): Guidance on conducting controlled ablation studies for LLMs, recommending systematic one-at-a-time component removal, best practices, pitfalls, and statistical rigor for reproducible insights.
- [Rigorous Statistical Comparison of Two LLM Models](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Experimentation/Rigorous-Statistical-Comparison-of-Two-LLM-Models/page.md): Guidelines for rigorously comparing two LLMs using paired per-example statistical tests, recommended tests, reporting practices, and a practical checklist to ensure valid significance and effect size estimation.

### Software Development

- [Monitoring High Throughput LLM Production Performance](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Software-Development/Monitoring-High-Throughput-LLM-Production-Performance/page.md): Explains monitoring high-throughput LLM production performance, recommending tokens per second as the primary metric and offering measurement, optimization, and diagnostic tips
- [Python Package for RAG Vector Database Implementation](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Software-Development/Python-Package-for-RAG-Vector-Database-Implementation/page.md): Comparing FAISS and Pinecone as Python vector database options for storing and searching embeddings in Retrieval Augmented Generation systems, with examples and alternative tools.
- [TensorRT Technique for Improving LLM Inference Latency](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Software-Development/TensorRT-Technique-for-Improving-LLM-Inference-Latency/page.md): Describes using TensorRT INT8 quantization to reduce LLM inference latency with minimal accuracy loss, covering calibration, mixed precision fallbacks, workflow steps, and comparisons to other optimizations
- [Software Architecture for Variable LLM Workloads](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Software-Development/Software-Architecture-for-Variable-LLM-Workloads/page.md): Advocates microservices with horizontal scaling for scalable, cost‑effective LLM deployments handling bursty workloads using autoscaling, GPU pooling, batching, and observability
- [Ensuring Reliable Performance in Production LLM Deployments](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Software-Development/Ensuring-Reliable-Performance-in-Production-LLM-Deployments/page.md): Guidance on achieving reliable, highly available production LLM deployments through comprehensive monitoring, alerting, observability, and incident response practices including metrics, tracing, logging, and canary testing.
- [RAG Component for Determining Document Relevance](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Software-Development/RAG-Component-for-Determining-Document-Relevance/page.md): Explains RAG pipeline components and emphasizes the re-ranker as the component that determines retrieved documents' relevance to a user query.
- [Reducing LLM Memory Footprint During Deployment](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Software-Development/Reducing-LLM-Memory-Footprint-During-Deployment/page.md): Overview of reducing LLM memory during deployment using knowledge distillation, quantization, and pruning while preserving reasonable performance
- [Effective Chunking Strategy for RAG Systems](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Software-Development/Effective-Chunking-Strategy-for-RAG-Systems/page.md): Evaluating chunking strategies for RAG systems and recommending semantic unit splitting with overlap to preserve context and maximize retrieval relevance.
- [Python Library for Semantic Similarity Calculation LLM Eval](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Software-Development/Python-Library-for-Semantic-Similarity-Calculation-LLM-Eval/page.md): Recommending Sentence Transformers for computing semantic similarity between LLM outputs and references, with example code, interpretation, and comparisons to NumPy, pandas, and Matplotlib.
- [Software Design Principle for Evolving LLM Applications](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Software-Development/Software-Design-Principle-for-Evolving-LLM-Applications/page.md): Advocating dependency injection to decouple and enable swapping LLM models, providers, and adapters for testability, configurability, and extensibility in evolving LLM applications.
- [Managing GPU Memory Constraints for LLMs in Python](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Software-Development/Managing-GPU-Memory-Constraints-for-LLMs-in-Python/page.md): Strategies to manage GPU memory for large language models in Python, including model parallelism, offloading, reduced precision, quantization, and gradient checkpointing.
- [Failure Handling Pattern for High Availability LLM Deployment Python](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Software-Development/Failure-Handling-Pattern-for-High-Availability-LLM-Deployment-Python/page.md): Explains using the circuit breaker pattern in Python to protect LLM deployments, preventing cascading failures, enabling fallbacks, retries, and recovery with observability and configuration guidance.

### Trustworthy AI

- [Minimizing Bias in LLM Fine Tuning](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Trustworthy-AI/Minimizing-Bias-in-LLM-Fine-Tuning/page.md): Best practices for detecting and reducing bias during LLM fine-tuning, emphasizing demographic performance evaluation, data balancing, model and post-processing mitigations, and iterative monitoring.
- [RAGs Role in LLM Trustworthiness](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Trustworthy-AI/RAGs-Role-in-LLM-Trustworthiness/page.md): Explains how Retrieval Augmented Generation (RAG) grounds LLM responses in external sources to improve factual accuracy, traceability, and reduce hallucinations while requiring curated, verified knowledge bases
- [Demonstrating Transparency in LLM Deployment](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Trustworthy-AI/Demonstrating-Transparency-in-LLM-Deployment/page.md): Discussion of transparency in LLM deployment emphasizing clear documentation of capabilities, limitations, evaluation, biases, provenance, and policies to support trust, accountability, and responsible use
- [Data Privacy Best Practices for LLMs](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Trustworthy-AI/Data-Privacy-Best-Practices-for-LLMs/page.md): Guidance on data privacy best practices for LLMs, focusing on data minimization, purpose limitation, retention policies, security controls, and user transparency
- [Protecting Against Prompt Injection Attacks](https://notes.kodekloud.com/docs/NVIDIA-Generative-AI-LLMs-Associate-Certification/Trustworthy-AI/Protecting-Against-Prompt-Injection-Attacks/page.md): Advice for defending LLM applications against prompt injection by validating and sanitizing user inputs and applying layered mitigations