> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.
# What Changes With Predictive Models
> Explains differences between generative and predictive models, deploying a DistilBERT sentiment classifier with KServe, and how request and response shapes and manifests differ
Welcome to this module on serving predictive models with KServe. In the previous lesson we deployed a generative language model (Qwen) and invoked it with chat-style prompts. In this lesson we'll contrast that workflow with predictive (classification) models: what changes in the request/response shape, what stays the same in your deployment manifest, and why distilled models like DistilBERT are a great fit for production classification tasks.
This guide covers:
* The conceptual difference between generative and predictive models
* Why request and response payloads differ
* How KServe keeps the InferenceService manifest stable across model types
* A brief introduction to model distillation and the DistilBERT sentiment classifier we'll deploy
## Generative vs Predictive — core concept
A generative model composes new content token-by-token. Example: you ask "Tell me about Kubernetes" and the model generates an explanatory paragraph. Its output space is effectively open-ended.
A predictive model, by contrast, maps inputs to a predefined set of labels. It answers questions like "Which bucket does this input belong to?" or "Which label best describes this input?" The output is a discrete set (e.g., {spam, not_spam}) or a probability distribution over those labels.
Common predictive examples:
* Spam filtering: label an email `spam` or `not_spam`.
* Image classification: detect whether a photo contains a cat (`cat` / `no_cat`).
* Sentiment analysis: classify text as `positive` or `negative`.
The classifier we’ll deploy in this lesson performs sentiment analysis: it reads a sentence and predicts positive or negative sentiment. This is a canonical predictive task and useful to illustrate the differences from generative workflows.
## Distillation — why DistilBERT?
Distillation trains a smaller "student" model to mimic a larger "teacher". The result is a compact model that approximates the teacher's behavior while using far less memory, CPU, and storage. Distilled models trade a small amount of accuracy for much lower inference cost — ideal for production classifiers.
The model we’ll use is:
* Name: `distilbert-base-uncased-finetuned-sst-2-english`
* DistilBERT: lighter, faster variant of BERT
* `uncased`: ignores case differences
* `finetuned SST-2 English`: trained on the Stanford Sentiment Treebank (SST-2) for sentiment classification
When you send a sentence to this model it returns a label (positive/negative) and a confidence score. We pull the model directly from Hugging Face via an hf URI — no external object storage is required. The Hugging Face model URI is:
`hf://distilbert-base-uncased-finetuned-sst-2-english`
## KServe InferenceService — what stays the same
One of KServe's strengths is that the InferenceService manifest and predictor spec look the same whether you deploy a large generative model or a compact classifier. Only a few fields change: most notably the InferenceService `metadata.name` and the `storageUri` (model identifier). Resource requests/limits typically shrink for distilled classifiers.
Example InferenceService manifest (Hugging Face model):
```yaml theme={null}
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
name: sentiment-classifier
namespace: kserve-inference
spec:
predictor:
model:
modelFormat:
name: huggingface
storageUri: "hf://distilbert-base-uncased-finetuned-sst-2-english"
resources:
requests:
cpu: "500m"
memory: "1Gi"
limits:
cpu: "1"
memory: "2Gi"
```
Only a few manifest details differ compared to a generative model deployment:
* `metadata.name` — a descriptive name for the InferenceService
* `storageUri` — model identifier (e.g., the Hugging Face URI)
* `resources` — classifiers typically require smaller requests/limits than large generative LLMs
This consistency means you can reuse the same deployment workflow and tooling across generative and predictive models.
## Request / Response shape — what changes
The primary runtime difference is the inference payload. Generative models accept chat-style prompts (a list of messages, system and user roles) and produce open-ended text. Classifiers accept a single input (text or data tensor) and return a label or a probability distribution over labels.
Common payload examples (these are typical shapes — exact format depends on your predictor and KServe runtime configuration):
* Simple instances-style request:
```json theme={null}
{
"instances": [
"This course has been great!"
]
}
```
* Typical classifier response (probabilities for labels in fixed order `[negative, positive]`):
```json theme={null}
{
"predictions": [
[0.02, 0.98]
]
}
```
* Alternatively, some Hugging Face-style APIs return labeled results:
```json theme={null}
[
{
"label": "POSITIVE",
"score": 0.9873
}
]
```
Note: the exact JSON keys (`instances` vs `inputs` vs a Hugging Face-specific shape) can differ by predictor and the chosen protocol (V1/V2). KServe docs and your runtime configuration specify the exact expected format.
Key takeaway: predictive models select from a predefined set of labels (e.g., positive/negative). KServe uses the same InferenceService manifest and runtime for both generative and predictive models; only the inference payload shape changes.
## Quick comparison: generative vs predictive
| Aspect | Generative Model | Predictive (Classifier) |
| --------------- | -----------------------------: | ------------------------------------ |
| Output | Free-form text | Fixed set of labels or probabilities |
| Input | Chat-style messages, prompts | Single text or tensor input |
| Typical use | Content generation, completion | Classification, detection, ranking |
| Resource cost | Often very high (large LLMs) | Typically lower (distilled models) |
| KServe manifest | Same API & predictor spec | Same API & predictor spec |
## Summary
* Predictive models map inputs to predefined labels rather than generating free-form content.
* DistilBERT (fine-tuned on SST-2) is a compact, production-friendly sentiment classifier.
* KServe keeps the InferenceService manifest and runtime consistent across different model types; only the model URI and resource sizing change.
* The main difference is the runtime payload format: generative models accept prompts/messages, while classifiers expect a single input and return labels or probability distributions.
Next: we'll deploy the sentiment classifier and demonstrate the exact request/response payloads for your KServe setup.
References:
* KServe documentation: [https://kserve.github.io](https://kserve.github.io)
* Hugging Face models: [https://huggingface.co](https://huggingface.co)
* Stanford Sentiment Treebank (SST-2): [https://nlp.stanford.edu/sentiment/](https://nlp.stanford.edu/sentiment/)