Skip to main content
Welcome to KServe Fundamentals: Serving ML Models on Kubernetes. I’m Chris Short, and I’ll guide you through deploying and operating machine learning models on Kubernetes using KServe. As organizations adopt AI and machine learning, the ability to reliably deploy models at scale has become essential. Model development is only part of the journey — production delivery requires a platform that can handle deployment, autoscaling, routing, and operational observability. KServe is a Kubernetes-native model serving platform built to simplify these tasks for both traditional ML and generative AI (LLM) workloads. It standardizes model serving while leveraging Kubernetes’ orchestration features. This lesson covers KServe fundamentals and provides hands-on experience serving models on Kubernetes. We’ll prepare your environment, install KServe, and walk through both generative and predictive serving patterns. Prerequisites
  • A Kubernetes cluster with sufficient resources (CPU, RAM, and storage) for model serving.
  • kubectl configured to point to your target cluster.
  • Helm 3 installed.
  • Appropriate cluster permissions to create CRDs and namespace resources.
Install KServe (example)
  • The following Helm commands install the KServe CRDs and the KServe controller in RawDeployment mode. Adjust versions or values to match your environment.
We’ll start with the fundamentals: what model serving is, KServe architecture, and the core components (InferenceService, LLMISvc, predictors, and transformers) that power model serving on Kubernetes.
The image shows a man speaking into a microphone with a presentation slide titled "KServe Fundamentals" displaying a bulleted list of topics related to models and troubleshooting.
What you’ll do in this course
  • Deploy a quantized Qwen large language model using KServe, creating the appropriate KServe serving resources (LLMISvc or InferenceService) and sending inference requests.
  • Serve generative AI workloads and observe how request formats and response handling differ from traditional predictive models.
  • Deploy a text-classification model to learn patterns for predictive inference (batch vs. real-time, request/response schemas).
  • Inspect InferenceService status, identify deployment issues, and troubleshoot common problems in KServe environments.
  • Reinforce concepts with demonstrations, hands-on labs, and quizzes.
Comparing LLM and predictive model request formats
  • Generative LLMs often use chat-style prompts or structured inputs with streaming responses and token-level outputs.
  • Predictive/classification models typically accept plain text or feature vectors and return structured labels or probabilities.
The image is a slide from a presentation titled "KServe Fundamentals: Serving ML Models on Kubernetes," comparing two request formats: Qwen, which uses chat-style prompts, and Classifier, which takes plain text input. A person is shown in the corner.
Course outline (high level) Links and references Throughout the course you’ll practice deploying, serving, and troubleshooting both generative and predictive ML models using KServe on Kubernetes. Each hands-on module includes sample manifests and inference examples so you can reproduce the demos in your own cluster.
Before running the Helm commands, ensure your kubectl context points to the Kubernetes cluster where you intend to install KServe and that you have sufficient permissions to create cluster-level resources.

Watch Video