# KodeKloud: Beta Courses Vector Database for GenAI

## Vector Database for GenAI

### Introduction to Vector Databases and Generative AI

- [Course Introduction](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Introduction-to-Vector-Databases-and-Generative-AI/Course-Introduction/page.md): A practical course on vector databases for GenAI covering embeddings, similarity metrics, ANN indexing, scalable architectures, retrieval augmented generation, and hands-on labs.
- [Vector Databases The Definition](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Introduction-to-Vector-Databases-and-Generative-AI/Vector-Databases-The-Definition/page.md): Introduction to vector databases explaining embeddings, storage and indexing for similarity search, typical operations, ANN algorithms, pros and cons, and common AI use cases like semantic search and RAG
- [Demo Ask Warren Anything You Want](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Introduction-to-Vector-Databases-and-Generative-AI/Demo-Ask-Warren-Anything-You-Want/page.md): A demo showing how a vector database and embeddings enable semantic search over Warren Buffett shareholder letters without using large language models
- [Vectors Made Simple](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Introduction-to-Vector-Databases-and-Generative-AI/Vectors-Made-Simple/page.md): Explains vectors and embeddings, how embedding maps data into high dimensional numeric space, and how vector search enables semantic similarity for search, recommendations, and retrieval
- [Demo Visualization of Vectors in vectorDB](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Introduction-to-Vector-Databases-and-Generative-AI/Demo-Visualization-of-Vectors-in-vectorDB/page.md): Demo showing PDF text converted to sentence-transformer embeddings stored in ChromaDB and visualized in 3D via PCA and Plotly, with chunking and metadata previews.
- [Role of Vector Databases in GenAI](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Introduction-to-Vector-Databases-and-Generative-AI/Role-of-Vector-Databases-in-GenAI/page.md): Explains how vector databases support Retrieval-Augmented Generation by storing embeddings for fast semantic search, enabling LLMs to access up-to-date contextual knowledge in production.
- [Airline Chatbot Vector Database](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Introduction-to-Vector-Databases-and-Generative-AI/Airline-Chatbot-Vector-Database/page.md): An airline chatbot architecture using embeddings and a vector database for RAG to retrieve policies and invoke backend APIs for bookings and cancellations
- [Vector Databases vs Relational and NoSQL Databases](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Introduction-to-Vector-Databases-and-Generative-AI/Vector-Databases-vs-Relational-and-NoSQL-Databases/page.md): Comparison of vector, relational, and NoSQL databases, highlighting data models, query patterns, use cases, scalability, and integration for AI tasks like semantic search and RAG
- [Demo Setting up a Vector Database](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Introduction-to-Vector-Databases-and-Generative-AI/Demo-Setting-up-a-Vector-Database/page.md): Guide to running Qdrant locally with Docker and Jupyter, using Python client to create collections, insert embeddings, and perform basic vector similarity queries.

### Vector Database Foundations

- [What Does a Vector Database Store](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Foundations/What-Does-a-Vector-Database-Store/page.md): Explains vector databases storing embeddings, IDs, metadata and indexes to enable semantic search, retrieval for ML, RAG, recommendations, and similarity-based applications.
- [Vector Embeddings Numerical Representations of Data](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Foundations/Vector-Embeddings-Numerical-Representations-of-Data/page.md): Explains vector embeddings and how text becomes numerical vectors, encoding semantics for similarity, enabling semantic search, clustering, and retrieval in vector databases
- [Querying Normal Database](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Foundations/Querying-Normal-Database/page.md): Compares querying relational databases with semantic vector search, showing exact structured SQL queries versus similarity-based embedding searches, plus examples, security tips, and next learning steps.
- [Querying Vector Database](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Foundations/Querying-Vector-Database/page.md): Explains how vector databases use embeddings and nearest neighbor search to enable semantic search, contrasting with traditional exact-match databases and covering practical considerations
- [Demo Difference in querying a normal DB vs vectorDB](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Foundations/Demo-Difference-in-querying-a-normal-DB-vs-vectorDB/page.md): Demo comparing PostgreSQL relational queries and LanceDB vector semantic search using a Node.js app to illustrate differences and hybrid architecture
- [Vector Quering Methods](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Foundations/Vector-Quering-Methods/page.md): Overview of seven vector database query methods and trade-offs, explaining semantic, KNN, hybrid, range, multi-vector, ANN, and two-stage retrieval with re-ranking.
- [When to Use Which Query Method Part 1](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Foundations/When-to-Use-Which-Query-Method-Part-1/page.md): Guidance on when to use semantic search, KNN, and hybrid vector query methods, with recaps, usage signals, and practical examples.
- [When to Use Which Query Method Part 2](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Foundations/When-to-Use-Which-Query-Method-Part-2/page.md): Guidance on choosing vector similarity query methods — range thresholds, multi-vector, ANN, and two-stage retrieve-and-rerank — with trade-offs, use cases, and tuning tips.

### From Data to Vectors: The Embedding Layer

- [From Data to Vectors The Embedding Layer](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/From-Data-to-Vectors-The-Embedding-Layer/From-Data-to-Vectors-The-Embedding-Layer/page.md): Explains how embedding layers convert text images and audio into numeric vectors, covering tokenization encoders normalization similarity metrics model selection evaluation and operational best practices.
- [Embedding Models](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/From-Data-to-Vectors-The-Embedding-Layer/Embedding-Models/page.md): Explains embedding models and retrieval-augmented workflows for converting text and other modalities into vectors, indexing in vector databases, and retrieving context for semantic search and RAG.
- [Text Embedding Models](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/From-Data-to-Vectors-The-Embedding-Layer/Text-Embedding-Models/page.md): Overview of text embedding models, comparing BERT, SBERT, and OpenAI text-embedding-3, their uses and practical guidance for semantic search and RAG
- [Demo Text Embedding with sentence transformer](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/From-Data-to-Vectors-The-Embedding-Layer/Demo-Text-Embedding-with-sentence-transformer/page.md): Hands-on tutorial converting a Markdown policy into SentenceTransformer embeddings, storing vectors in LanceDB, and performing semantic search with code examples and chunking strategies.
- [Image Embedding Models](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/From-Data-to-Vectors-The-Embedding-Layer/Image-Embedding-Models/page.md): Overview of image embedding models CLIP ResNet and ViT comparing their architectures strengths embedding sizes and use cases for visual search retrieval and downstream vision tasks
- [Demo Image Embedding](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/From-Data-to-Vectors-The-Embedding-Layer/Demo-Image-Embedding/page.md): Tutorial demonstrating embedding images with CLIP, storing vectors in LanceDB, and performing text-to-image searches in a Jupyter Notebook
- [Audio Embedding Models](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/From-Data-to-Vectors-The-Embedding-Layer/Audio-Embedding-Models/page.md): Overview of audio embedding models that convert audio into fixed-size vectors for search, classification, and retrieval, explaining methods, use cases, and notable models like Wav2Vec, OpenL3, and CLAP
- [Video Embedding Models](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/From-Data-to-Vectors-The-Embedding-Layer/Video-Embedding-Models/page.md): Overview of video embedding models that encode spatial and temporal video content into fixed-length vectors, covering architectures, challenges, applications, and practical considerations for retrieval and action recognition.
- [Choosing and Optimizing Embedding Models](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/From-Data-to-Vectors-The-Embedding-Layer/Choosing-and-Optimizing-Embedding-Models/page.md): Guide to selecting and optimizing embedding models for production, covering model choice, evaluation, fine-tuning, compression, indexing, and monitoring for retrieval and RAG systems

### Vector Similarity Explained

- [Three Ways to Measure Similarity](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Similarity-Explained/Three-Ways-to-Measure-Similarity/page.md): Describes cosine similarity, Euclidean distance, and dot product and their applications for semantic search, recommendations, and retrieval
- [Cosine Similarity](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Similarity-Explained/Cosine-Similarity/page.md): Explains cosine similarity as an angle-based vector comparison that ignores magnitude, used for measuring semantic similarity in text embeddings and semantic search.
- [Euclidean Distance](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Similarity-Explained/Euclidean-Distance/page.md): Explains Euclidean distance, compares it to cosine similarity, discusses use cases, examples, and caveats for high dimensional data like embeddings
- [Dot Product Similarity](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Similarity-Explained/Dot-Product-Similarity/page.md): Explains dot product similarity, how it combines vector magnitude and alignment, contrasts with cosine and Euclidean measures, and discusses applications like recommendation systems and maximum inner product search.
- [Demo Vector Search Metrics in 2D](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Similarity-Explained/Demo-Vector-Search-Metrics-in-2D/page.md): Visual explanation of cosine similarity, Euclidean distance, and dot product showing how each metric affects vector search rankings in 2D
- [Comparing Cosine Dot Product and Euclidean Similarity](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Similarity-Explained/Comparing-Cosine-Dot-Product-and-Euclidean-Similarity/page.md): Comparison of cosine similarity, Euclidean distance, and dot product with guidance on when to use each and practical tips for embeddings, clustering, and recommendations.
- [Fruit Emedding Similarity Metrics in Action](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Similarity-Explained/Fruit-Emedding-Similarity-Metrics-in-Action/page.md): Demonstrates a fruit recommendation example showing how different embedding similarity metrics like Euclidean, cosine, and dot product affect nearest neighbor rankings
- [How Similarity Results Can Differ](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Similarity-Explained/How-Similarity-Results-Can-Differ/page.md): Shows how cosine similarity, Euclidean distance, and dot product produce different similarity rankings for identical fruit embeddings.
- [Demo Setting up Vectors for Fruits](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Similarity-Explained/Demo-Setting-up-Vectors-for-Fruits/page.md): Demo demonstrating how to create, store, and query handcrafted 3D fruit vectors in a local LanceDB instance for similarity search using a mango query vector.
- [Demo Query a New Fruit Across 3 Searches](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Similarity-Explained/Demo-Query-a-New-Fruit-Across-3-Searches/page.md): Demonstrates querying a mango vector against fruit embeddings using three similarity metrics and visualizations

### Building Vector Storage on AWS S3

- [S3 Vector Buckets](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Building-Vector-Storage-on-AWS-S3/S3-Vector-Buckets/page.md): Describes Amazon S3 Vector Buckets for storing and querying vector embeddings natively in S3 to enable serverless similarity search, lower costs, and simplify ML workflows.
- [Demo Creating an S3 Vector Bucket](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Building-Vector-Storage-on-AWS-S3/Demo-Creating-an-S3-Vector-Bucket/page.md): Guide to creating and configuring AWS S3 vector buckets and vector indexes through the console, covering naming, dimensions, similarity metrics, encryption, and programmatic ingestion.
- [Demo Creating an IAM policy for S3 Vector Buckets](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Building-Vector-Storage-on-AWS-S3/Demo-Creating-an-IAM-policy-for-S3-Vector-Buckets/page.md): Guides creating and attaching a customer managed IAM policy granting least privilege S3 actions for programmatic access to vector buckets, including tagging, permissions, KMS considerations, and credential handling
- [Key Features of S3 Vector Buckets](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Building-Vector-Storage-on-AWS-S3/Key-Features-of-S3-Vector-Buckets/page.md): Overview of AWS S3 Vector Buckets' features, scale limits, similarity search, metadata filtering, serverless billing, integrations, and security for production vector search workflows.
- [Demo Accessing the S3 Vector Buckets](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Building-Vector-Storage-on-AWS-S3/Demo-Accessing-the-S3-Vector-Buckets/page.md): Guide to accessing and managing AWS S3 Vector Buckets with Python Boto3, including creating clients, listing vector buckets and indexes and common s3vectors operations
- [Demo Embedding and Accessing Vectors from S3 Vector Buckets](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Building-Vector-Storage-on-AWS-S3/Demo-Embedding-and-Accessing-Vectors-from-S3-Vector-Buckets/page.md): Demonstrates storing sentence embeddings in an S3 vector bucket, uploading vectors with metadata, validating dimensions, and performing similarity search using SentenceTransformer and AWS S3 Vectors API
- [S3 Vector Buckets vs Standard S3 Buckets](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Building-Vector-Storage-on-AWS-S3/S3-Vector-Buckets-vs-Standard-S3-Buckets/page.md): Comparing S3 Vector Buckets and standard S3 buckets for AI versus general object storage, explaining differences in data models, APIs, querying, and ideal use cases.
- [S3 Vector Buckets vs Vector Databases](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Building-Vector-Storage-on-AWS-S3/S3-Vector-Buckets-vs-Vector-Databases/page.md): Comparison of Amazon S3 vector buckets and vector databases, recommending a tiered architecture combining S3 for archival scale and vector DBs for low-latency, high-throughput serving.

### Vector Database Landscape

- [Vector Database Landscape](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Landscape/Vector-Database-Landscape/page.md): Survey of vector database options, categories, representative projects, trade-offs, and key decision factors for selecting open-source, cloud-managed, or commercial vector search solutions
- [Pinecone Vector Database](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Landscape/Pinecone-Vector-Database/page.md): Overview of Pinecone, a serverless fully managed vector database for high-performance similarity search, RAG workflows, metadata-filtered retrieval, and scalable production AI applications.
- [Weaviate Vector Database](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Landscape/Weaviate-Vector-Database/page.md): Overview of Weaviate, an open source vector database offering hybrid semantic and keyword search, multimodal support, modular embedding integrations, and self hosted or managed deployment options
- [Milvus Vector Database](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Landscape/Milvus-Vector-Database/page.md): Introduction to Milvus, a scalable distributed open source vector database for billion scale AI search, offering cloud native deployment, multi index support, GPU acceleration, and high availability.
- [Choosing Your Vector Database](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Landscape/Choosing-Your-Vector-Database/page.md): Practical guide to choosing a vector database by deployment, scale, feature needs and operational tolerance with vendor recommendations
- [Vector Database Feature Comparison](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Landscape/Vector-Database-Feature-Comparison/page.md): Comparison guide to choosing vector databases for AI workloads based on deployment model, core strengths, scalability, and primary usage patterns
- [VectorDB Benchmark](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Landscape/VectorDB-Benchmark/page.md): Open-source VDBench benchmarking for vector databases, comparing cost-normalized latency and throughput, streaming ingestion impact, recall versus performance, and filtered-search to guide production choices

### Vector Database Internals

- [Vector Database Internals](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Internals/Vector-Database-Internals/page.md): Explains vector database internals including indexing, HNSW, quantization, sharding, filtering, and maintenance, focusing on trade offs between latency, accuracy, memory, and operational scaling.
- [Indexing in Vector Database](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Internals/Indexing-in-Vector-Database/page.md): Explains why brute force vector search fails at scale and how ANN indexing techniques like graph, hashing, trees, and quantization enable efficient, scalable nearest neighbor search with trade offs
- [Demo Understanding Indexing in Vector Database](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Internals/Demo-Understanding-Indexing-in-Vector-Database/page.md): Hands-on demo showing how a simple three-bucket index in a vector database reduces distance computations versus brute-force linear search.
- [HNSW Multi Layered Graph Structure](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Internals/HNSW-Multi-Layered-Graph-Structure/page.md): Explains HNSW multi-layered graph index for fast approximate nearest neighbor search in vector databases, covering intuition, search workflow, parameters, and practical tuning.
- [HNSW Construction and Adoption](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Internals/HNSW-Construction-and-Adoption/page.md): Guide to constructing and tuning HNSW graphs for vector search, detailing insertion, layer assignment, neighbor selection, and key parameters M efConstruction and ef for performance trade-offs
- [Quantization](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Internals/Quantization/page.md): Overview of quantization techniques for compressing embeddings in vector databases, comparing product, scalar, and binary methods, their tradeoffs, and guidance for balancing compression versus recall
- [Real World Example of Quantization](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Internals/Real-World-Example-of-Quantization/page.md): Demonstrates how vector embedding quantization reduces storage and improves large-scale e-commerce search performance while balancing accuracy, indexing, and retrieval trade-offs.
- [Demo Understand Binary Quantization](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Internals/Demo-Understand-Binary-Quantization/page.md): Explains binary quantization for embeddings, demonstrating bitwise conversion, tradeoffs between recall and storage, and when to use it for high dimensional vectors.
- [Index Maintenance](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Internals/Index-Maintenance/page.md): Discusses challenges of maintaining nearest-neighbor vector indexes amid data mutations and strategies like buffering, compaction, and periodic rebuilding to preserve performance and accuracy.
- [Strategy 1 Write First Index Later](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Internals/Strategy-1-Write-First-Index-Later/page.md): Explains the write-first, index-later pattern where writes are durably logged and acknowledged immediately while indexing happens asynchronously to reduce latency and ensure durability with eventual consistency.
- [Strategy 2 Merge Small Pieces](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Internals/Strategy-2-Merge-Small-Pieces/page.md): Background merging compacts many small write fragments into larger segments to reduce query overhead, balancing write performance and query efficiency through tunable compaction policies.
- [Strategy 3 Rebuild Sometimes](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Internals/Strategy-3-Rebuild-Sometimes/page.md): Periodic full index rebuilds to remove fragmentation, tombstones, and restore efficient vector search performance via validation, atomic swaps, and scheduled off peak reindexing.
- [Applying Index Maintenance Customer Care Chat Data](https://notes.kodekloud.com/docs/Vector-Database-for-GenAI/Vector-Database-Internals/Applying-Index-Maintenance-Customer-Care-Chat-Data/page.md): Describes batching index maintenance for large-scale customer care chat systems, using append-only logs, batched embeddings, and bulk vector upserts to improve performance, cost, and reliability.