> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Course: llm-d: A Deep Dive

> Guide to building scalable, observable Kubernetes architectures and operational practices for serving large language models, covering failure modes, routing, placement, rollout strategies, and hands-on demos.

Notes for the **llm-d: A Deep Dive** course: 43 lessons across 11 sections.

## From ChatGPT to a Model Server

* [Introduction](/docs/llm-d-A-Deep-Dive/From-ChatGPT-to-a-Model-Server/Introduction/page.md)
* [What This Course Covers](/docs/llm-d-A-Deep-Dive/From-ChatGPT-to-a-Model-Server/What-This-Course-Covers/page.md)
* [What a Model Really Is](/docs/llm-d-A-Deep-Dive/From-ChatGPT-to-a-Model-Server/What-a-Model-Really-Is/page.md)
* [The Hardware CPU GPU and VRAM](/docs/llm-d-A-Deep-Dive/From-ChatGPT-to-a-Model-Server/The-Hardware-CPU-GPU-and-VRAM/page.md)
* [From a Script to a Service](/docs/llm-d-A-Deep-Dive/From-ChatGPT-to-a-Model-Server/From-a-Script-to-a-Service/page.md)
* [A Closer Look at the Model Server](/docs/llm-d-A-Deep-Dive/From-ChatGPT-to-a-Model-Server/A-Closer-Look-at-the-Model-Server/page.md)

## Inside a Single Request

* [How a Model Writes Text](/docs/llm-d-A-Deep-Dive/Inside-a-Single-Request/How-a-Model-Writes-Text/page.md)
* [The Pause and the Stream](/docs/llm-d-A-Deep-Dive/Inside-a-Single-Request/The-Pause-and-the-Stream/page.md)
* [The Speed Limit](/docs/llm-d-A-Deep-Dive/Inside-a-Single-Request/The-Speed-Limit/page.md)
* [The Models Scratchpad](/docs/llm-d-A-Deep-Dive/Inside-a-Single-Request/The-Models-Scratchpad/page.md)

## From One Server to a Fleet

* [One GPU Many Users](/docs/llm-d-A-Deep-Dive/From-One-Server-to-a-Fleet/One-GPU-Many-Users/page.md)
* [Serving the Giants](/docs/llm-d-A-Deep-Dive/From-One-Server-to-a-Fleet/Serving-the-Giants/page.md)
* [One Machine or Many](/docs/llm-d-A-Deep-Dive/From-One-Server-to-a-Fleet/One-Machine-or-Many/page.md)
* [Why Load Balancing Breaks](/docs/llm-d-A-Deep-Dive/From-One-Server-to-a-Fleet/Why-Load-Balancing-Breaks/page.md)

## Meet llm-d

* [Meet llm d](/docs/llm-d-A-Deep-Dive/Meet-llm-d/Meet-llm-d/page.md)
* [Putting It to Work](/docs/llm-d-A-Deep-Dive/Meet-llm-d/Putting-It-to-Work/page.md)
* [Inside the Cluster](/docs/llm-d-A-Deep-Dive/Meet-llm-d/Inside-the-Cluster/page.md)
* [How the Router Decides](/docs/llm-d-A-Deep-Dive/Meet-llm-d/How-the-Router-Decides/page.md)
* [How the Router Knows](/docs/llm-d-A-Deep-Dive/Meet-llm-d/How-the-Router-Knows/page.md)
* [Routing on a Prediction](/docs/llm-d-A-Deep-Dive/Meet-llm-d/Routing-on-a-Prediction/page.md)
* [When the Scratchpad Overflows](/docs/llm-d-A-Deep-Dive/Meet-llm-d/When-the-Scratchpad-Overflows/page.md)

## Tour the Running Stack

* [The Cluster Behind Every Demo](/docs/llm-d-A-Deep-Dive/Tour-the-Running-Stack/The-Cluster-Behind-Every-Demo/page.md)
* [Demo Tour the Running Stack and Send the First Request](/docs/llm-d-A-Deep-Dive/Tour-the-Running-Stack/Demo-Tour-the-Running-Stack-and-Send-the-First-Request/page.md)
* [Demo Watch One Request Happen](/docs/llm-d-A-Deep-Dive/Tour-the-Running-Stack/Demo-Watch-One-Request-Happen/page.md)

## Smart Routing, Live

* [The Scorers Behind Every Decision](/docs/llm-d-A-Deep-Dive/Smart-Routing-Live/The-Scorers-Behind-Every-Decision/page.md)
* [Demo The Same Prompt Lands on the Same Server](/docs/llm-d-A-Deep-Dive/Smart-Routing-Live/Demo-The-Same-Prompt-Lands-on-the-Same-Server/page.md)
* [Demo Cold Requests Spread Out](/docs/llm-d-A-Deep-Dive/Smart-Routing-Live/Demo-Cold-Requests-Spread-Out/page.md)

## Getting More From the KV Cache

* [Precise Prefix Cache Routing](/docs/llm-d-A-Deep-Dive/Getting-More-From-the-KV-Cache/Precise-Prefix-Cache-Routing/page.md)
* [Tiered Cache Spill Over to CPU and Disk](/docs/llm-d-A-Deep-Dive/Getting-More-From-the-KV-Cache/Tiered-Cache-Spill-Over-to-CPU-and-Disk/page.md)
* [Demo A Multi Turn Chat Reuses the Cache](/docs/llm-d-A-Deep-Dive/Getting-More-From-the-KV-Cache/Demo-A-Multi-Turn-Chat-Reuses-the-Cache/page.md)

## Serving Bigger Models With llm-d

* [Wide Expert Parallelism](/docs/llm-d-A-Deep-Dive/Serving-Bigger-Models-With-llm-d/Wide-Expert-Parallelism/page.md)
* [Demo Read the PD well lit Path Against Our Live Cluster](/docs/llm-d-A-Deep-Dive/Serving-Bigger-Models-With-llm-d/Demo-Read-the-PD-well-lit-Path-Against-Our-Live-Cluster/page.md)
* [What Really Needs GPUs](/docs/llm-d-A-Deep-Dive/Serving-Bigger-Models-With-llm-d/What-Really-Needs-GPUs/page.md)

## Running It Day to Day

* [Flow Control A Line for Requests](/docs/llm-d-A-Deep-Dive/Running-It-Day-to-Day/Flow-Control-A-Line-for-Requests/page.md)
* [Autoscaling on the Right Signals](/docs/llm-d-A-Deep-Dive/Running-It-Day-to-Day/Autoscaling-on-the-Right-Signals/page.md)
* [Rollouts With No Downtime](/docs/llm-d-A-Deep-Dive/Running-It-Day-to-Day/Rollouts-With-No-Downtime/page.md)
* [Demo A Rolling Update Under Live Traffic](/docs/llm-d-A-Deep-Dive/Running-It-Day-to-Day/Demo-A-Rolling-Update-Under-Live-Traffic/page.md)

## Seeing What Is Happening

* [The Numbers llm d Reports](/docs/llm-d-A-Deep-Dive/Seeing-What-Is-Happening/The-Numbers-llm-d-Reports/page.md)
* [Demo Read the Grafana Dashboard](/docs/llm-d-A-Deep-Dive/Seeing-What-Is-Happening/Demo-Read-the-Grafana-Dashboard/page.md)
* [Demo Run a Small Benchmark](/docs/llm-d-A-Deep-Dive/Seeing-What-Is-Happening/Demo-Run-a-Small-Benchmark/page.md)

## Wrap Up: Agents and Beyond

* [Agentic Serving](/docs/llm-d-A-Deep-Dive/Wrap-Up-Agents-and-Beyond/Agentic-Serving/page.md)
* [Multimodal Serving](/docs/llm-d-A-Deep-Dive/Wrap-Up-Agents-and-Beyond/Multimodal-Serving/page.md)
* [Demo An Agent Style Multi Turn Workload](/docs/llm-d-A-Deep-Dive/Wrap-Up-Agents-and-Beyond/Demo-An-Agent-Style-Multi-Turn-Workload/page.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.