Skip to main content
Earlier we deployed the Qwen model and watched its status change from not-ready to ready. That deployment provides a live inference endpoint inside the Kubernetes cluster. In this lesson you’ll open a connection to that endpoint using kubectl port-forward, send an HTTP request with curl, and inspect the JSON response. By the end you’ll understand how to call a KServe inference endpoint that exposes an OpenAI-compatible API.
The image shows the Qwen logo at the top. Below it are two status cards—on the left a red "False" (starting up) card with an X and on the right a green "True" (live endpoint) card with a check, connected by an arrow.
When running inside a cluster, production traffic is typically routed through an ingress controller or API gateway to reach services. For local testing and learning we can create a direct tunnel from a local port to the Kubernetes Service using kubectl port-forward. Traffic you send to localhost is forwarded to the service inside the cluster for the duration of the command. KServe creates an InferenceService plus a corresponding Kubernetes Service for the predictor. In this example the predictor service is named qwen-model-predictor and it lives in the kserve-inference namespace.
A diagram titled "Opening the Tunnel" showing a Kubernetes cluster and a namespace called kserve-inference. Inside are two components: an InferenceService named qwen-model and a Kubernetes Service named qwen-model-predictor.
Open a terminal and run this command (keep it running). It forwards local port 8080 to port 80 on the qwen-model-predictor service in the kserve-inference namespace:
You should see output similar to:
Keep this terminal open while you make requests. The port-forwarding tunnel stops when you close the command.
Open a second terminal for sending requests to the local tunnel. KServe’s Hugging Face runtime exposes an OpenAI-compatible REST API. The chat completions endpoint is:
A minimal JSON request body requires three fields: the model name, a list of messages, and a max_tokens limit to bound the response length. Example request (use this from the second terminal):
Quick notes on the curl options and headers:
  • -s silences curl’s progress output so only the response is printed.
  • -H "Content-Type: application/json" ensures the server interprets the body as JSON.
  • The JSON body includes the required model, messages, and max_tokens fields.
Request body fields (summary) The first request may take a few extra seconds as the model begins producing tokens. The server returns a JSON response that typically contains:
  • choices[0].message.content — the model’s textual reply you want to display or parse.
  • usage — token accounting for prompt/completion/total (useful for cost monitoring).
For improved readability, pipe the response through jq (if installed):
Now you’ve sent a real request to a language model running inside Kubernetes and received a real JSON response.
A simple diagram showing a client (laptop) sending a green "Request" arrow to a Kubernetes cluster hosting a "Live Language Model," which returns a pink "Response" arrow back to the client.
That completes the full loop: deploy the model, expose it, and call it. Next, we’ll inspect how KServe routes incoming requests inside the cluster and identify which container handles the inference traffic. Links and references

Watch Video