Demo Reading InferenceService Status

If you’ve deployed models on Kubernetes before, you’ve probably seen READY: False and had to wait. That’s normal — downloading and initializing a model from remote storage can take time. But what if READY stays False for many minutes? Or never becomes True? Kubernetes contains the diagnostic information you need — you just need to know where to look and how to interpret it.

The image shows a user icon labeled "Users" connected to a text document with Kubernetes and a magnifying glass symbol, likely illustrating user interaction with Kubernetes documentation or logs.

This article walks through two realistic failure scenarios when serving the Qwen model with KServe, and shows how to read the Kubernetes/KServe output to find root causes and fixes. One scenario involves an authentication/path issue that can be misleading at first. Quick first step — check the InferenceService status:

kubectl get inferenceservice -n kserve-inference

If READY is False and AGE is only tens of seconds, the service is usually still initializing (storage initializer downloading files).
If several minutes pass with no URL and READY: False, start troubleshooting — something likely failed.

The AGE column is a helpful clue: if a service deployed five minutes ago is still False with no URL, it likely failed during initialization.

Below are two common scenarios, how the failure appears, and how to fix it.

Scenario 1 — Broken storage URI (typo or wrong repo)

Apply a deliberately broken InferenceService to reproduce the error:

kubectl apply -f qwen-model-broken.yaml
kubectl get inferenceservice -n kserve-inference

Short example output:

NAME         URL   READY   PREV   LATEST   PREVROLLEDOUTREVISION   LATESTREADYREVISION   AGE
qwen-model   False

Describe the InferenceService for more detail:

kubectl describe inferenceservice qwen-model -n kserve-inference

Look at the status section, especially conditions. A healthy condition shows Status: True and Type: PredictorReady (or Ready). For failures, you’ll see Status: False and reason + message fields with diagnostic text. Example storage-initializer failure (truncated for clarity):

Status: False
Type: Stopped
Deployment Mode: Standard
Model Status:
Copies: 
Failed Copies: 0
Last Failure Info:
  Exit Code: 1

2026-05-29 18:05:31.179  storage.initialize INFO  Initializing, args: (src_uri, dest_path)
2026-05-29 18:05:31.179  storage.initialize INFO  Copying contents of hf://Qwen/Qwen2.5-0.5B-Instruct to local
2026-05-29 18:05:31.615  storage.initialize ERROR Storage error when accessing hf://Qwen/Qwen2.5-0.5B-Instruct: 401 Client Error.
Repository Not Found for url: https://huggingface.co/api/models/Qwen/Qwen2.5-0.5B-Instruct/revision/main.
Please make sure you specified the correct `repo_id` and `repo_type`.
If you are trying to access a private or gated repo, make sure you are authenticated. For more details, see https://huggingface.co/docs/huggingface_hub/authentication
Invalid username or password.
2026-05-29 18:05:31.615  storage.initialize ERROR  Storage initialization failed: HuggingFace authentication failed. Set HF_TOKEN or request access to the gated repository.
Reason: ModelLoadFailed
States:
  Active Model State: FailedToLoad
  Target Model State: FailedToLoad
  Transition Status: BlockedByFailedLoad
Observed Generation: 1
Events:
  Warning  VirtualServiceCRDNotFound  Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true.

Important context: the combination of Repository Not Found and 401 can be confusing. It may indicate:

A typo in the storageUri (e.g., Instruct vs Instuct) — the path is wrong, so the API returns “not found”.
Or a private/gated repository that requires authentication (e.g., an HF_TOKEN).

When kubectl describe is ambiguous, inspect pod and init container logs:

kubectl get pods -n kserve-inference -w
# or
kubectl describe pod <pod-name> -n kserve-inference
kubectl logs <pod-name> -c storage-initializer -n kserve-inference

With Hugging Face URIs, if you must be logged in on huggingface.co to view the model, KServe also needs credentials (for example, set HF_TOKEN). If the path is wrong, correct the storageUri in your manifest and reapply.

Scenario 2 — Out of memory (OOMKilled)

ML models include weights that must be loaded into RAM. If a container is constrained to less memory than required, the kubelet will kill it (OOMKilled). This leads to restarts and the Revision/Deployment failing to reach minimum availability, causing the InferenceService to remain False. Apply a manifest that intentionally causes OOM:

kubectl apply -f qwen-model-oomkilled.yaml
kubectl get pods -n kserve-inference

You may observe init containers or the main container being restarted with OOMKilled. Describing the InferenceService will show PredictorReady: False and messages about minimum availability or scheduling:

Predictor:
  Conditions:
    - Last Transition Time: 2026-05-29T18:10:52Z
      Reason: Predictor ingress not created
      Status: False
      Type: IngressReady
    - Last Transition Time: 2026-05-29T18:10:52Z
      Message: Deployment does not have minimum availability.
      Reason: MinimumReplicasUnavailable
      Status: False
      Type: PredictorReady
    - Last Transition Time: 2026-05-29T18:11:24Z
      Reason: Predictor ingress not created
      Status: False
      Type: Ready
      Severity: Info
  Last Transition Time: 2026-05-29T18:10:52Z
  Status: False
  Type: Stopped
  Deployment Mode: Standard
  Model Status:
    Copies:
      Failed Copies: 0
    States:
      Active Model State:
      Target Model State: Pending
      Transition Status: InProgress
  Observed Generation: 1
Events:
  Warning  VirtualServiceCRDNotFound  Istio VirtualService CRD not present; VirtualService reconciliation skipped. If you do not use Istio, set ingress.disableIstioVirtualHost=true.

Deployment does not have minimum availability commonly indicates:

Kubernetes couldn’t schedule the pod because nodes lack sufficient memory.
Or the pod is repeatedly OOMKilled because requests/limits are too low.

Example manifest that intentionally sets too-small memory requests/limits:

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: qwen-model
  namespace: kserve-inference
spec:
  predictor:
    model:
      modelFormat:
        name: huggingface
      storageUri: "hf://Qwen/Qwen-2.5-0.5B-Instruct"
      args:
        - --backend=huggingface
    resources:
      requests:
        memory: "512Mi"
      limits:
        memory: "512Mi"

Setting both requests and limits to 512Mi is too small for the Qwen model and will likely produce OOMKilled. A common starting point for Qwen is requests: 2Gi and limits: 4Gi. Adjust these values according to the model size and available node resources, or schedule on nodes with more memory.

The image displays a terminal output with information on a deployment, showing status updates and reasons for issues related to a predictor ingress and minimum replicas.

When you see Deployment does not have minimum availability, either increase the pod memory request/limit or provision nodes with sufficient memory so Kubernetes can schedule the pod.

Troubleshooting checklist (quick reference)

Step	Command / Action	Purpose
Check InferenceService	`kubectl get inferenceservice -n kserve-inference`	See basic `READY`, `URL`, `AGE` status
Inspect conditions	`kubectl describe inferenceservice qwen-model -n kserve-inference`	Read `status.conditions` -> `reason` and `message`
Watch pods	`kubectl get pods -n kserve-inference -w`	See pod creation, restarts, scheduling events
Pod details	`kubectl describe pod <pod-name> -n kserve-inference`	Check events, status, and reasons (OOMKilled, scheduling)
Init container logs	`kubectl logs <pod-name> -c storage-initializer -n kserve-inference`	See storage initialization errors (auth, repo not found)
Predictor logs	`kubectl logs <pod-name> -c predictor -n kserve-inference`	Runtime errors from the serving process

Read the conditions -> message field first. It’s the single most useful piece of information KServe provides about why an InferenceService isn’t becoming ready.

Practical troubleshooting based on messages:

If message indicates storage or repository errors:
- Verify the storageUri for typos (e.g., Instruct vs Instuct).
- If the model is private/gated, set HF_TOKEN in your environment or use a repo you have access to. See Hugging Face authentication docs: https://huggingface.co/docs/huggingface_hub/authentication
- Inspect storage-initializer logs:
  kubectl logs <pod-name> -c storage-initializer -n kserve-inference
If message indicates scheduling/minimum availability or pods show OOMKilled:
- Check pod status and events:
  kubectl get pods -n kserve-inference kubectl describe pod <pod-name> -n kserve-inference
- Increase resources.requests.memory and resources.limits.memory in your InferenceService manifest to meet the model’s RAM needs (for example, requests: "2Gi", limits: "4Gi" for Qwen as a starting point), then reapply.

After applying fixes, watch the InferenceService until READY becomes True:

kubectl get inferenceservice -n kserve-inference -w

If the conditions/events don’t provide enough detail, inspect pod logs (init and main containers). Storage initializer and predictor logs typically reveal the root cause: bad path, authentication failure, or memory allocation problems. Useful links and references:

KServe documentation: https://kserve.github.io/ (KServe concepts and troubleshooting)
Kubernetes docs: https://kubernetes.io/docs/ (kubectl, pods, events)
Hugging Face authentication: https://huggingface.co/docs/huggingface_hub/authentication

By following this checklist and reading the reason and message fields in KServe conditions, you can distinguish a slow initial load from an actual failure and take the correct corrective action.

Foundations

Serving a Generative Model

Serving a Predictive Model

Troubleshooting Basics

Demo Reading InferenceService Status

Scenario 1 — Broken storage URI (typo or wrong repo)

Scenario 2 — Out of memory (OOMKilled)

Troubleshooting checklist (quick reference)

Watch Video

Practice Lab

​Scenario 1 — Broken storage URI (typo or wrong repo)

​Scenario 2 — Out of memory (OOMKilled)

​Troubleshooting checklist (quick reference)

Watch Video

Practice Lab

Scenario 1 — Broken storage URI (typo or wrong repo)

Scenario 2 — Out of memory (OOMKilled)

Troubleshooting checklist (quick reference)