Skip to main content

Documentation Index

Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt

Use this file to discover all available pages before exploring further.

This guide demonstrates Istio Service Mesh traffic mirroring. Traffic mirroring (also called shadowing) sends a duplicate of client requests to a mirror target while the primary target serves the actual response. It’s useful for testing, telemetry, and validating new revisions without impacting live traffic. Prerequisites:
  • An Istio-enabled cluster with sidecar injection enabled for the namespace you use.
  • kubectl configured to communicate with the cluster.
Links and references:

​
1) Confirm Istio sidecar injection is enabled for the namespace

Check the namespace labels to ensure istio-injection=enabled is present: 
root@controlplane ~ ➜  kubectl get ns --show-labels
NAME               STATUS   AGE     LABELS
default            Active   2m38s   istio-injection=enabled,kubernetes.io/metadata.name=default
istio-system       Active   2m6s    kubernetes.io/metadata.name=istio-system
kube-node-lease    Active   2m38s   kubernetes.io/metadata.name=kube-node-lease
kube-public        Active   2m38s   kubernetes.io/metadata.name=kube-public
kube-system        Active   2m38s   kubernetes.io/metadata.name=kube-system

root@controlplane ~ ➜

​
2) Create echo-server deployments (v1 and v2)

We deploy two revisions of the same app. Note the shared app: echo-server label and distinct version labels — these version labels become DestinationRule subsets that Istio will use. 
# echo-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-server-v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo-server
      version: v1
  template:
    metadata:
      labels:
        app: echo-server
        version: v1
    spec:
      containers:
      - name: echo
        image: ealen/echo-server
        ports:
        - containerPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-server-v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo-server
      version: v2
  template:
    metadata:
      labels:
        app: echo-server
        version: v2
    spec:
      containers:
      - name: echo
        image: ealen/echo-server
        ports:
        - containerPort: 80
Apply and validate the deployments: 
root@controlplane ~ ➜ kubectl apply -f echo-deployment.yaml
deployment.apps/echo-server-v1 created
deployment.apps/echo-server-v2 created

root@controlplane ~ ➜ kubectl get pods --show-labels
NAME                                     READY   STATUS    RESTARTS   AGE   LABELS
echo-server-v1-59ff75d58-t4dq6           1/2     Running   0          7s    app=echo-server,pod-template-hash=59ff75d58,security.istio.io/tlsMode=istio,service.istio.io/canonical-name=echo-server,service.istio.io/canonical-revision=v1,version=v1
echo-server-v2-5698db4f99-lm5ss          2/2     Running   0          7s    app=echo-server,pod-template-hash=5698db4f99,security.istio.io/tlsMode=istio,service.istio.io/canonical-name=echo-server,service.istio.io/canonical-revision=v2,version=v2

​
3) Create a ClusterIP Service that selects both revisions

A single Service selects pods by the shared app: echo-server label so client traffic can hit either revision. 
# echo-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: echo-server
  labels:
    app: echo-server
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: echo-server
Apply the Service and confirm: 
root@controlplane ~ ➜ kubectl apply -f echo-svc.yaml
service/echo-server created

root@controlplane ~ ➜ kubectl get svc --show-labels
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE   LABELS
echo-server  ClusterIP   10.96.123.45   <none>        80/TCP    10s   app=echo-server

​
4) Inspect pod logs to see baseline behavior

Open two terminals (or tmux panes). Tail v1 logs: 
root@controlplane ~ ➜ kubectl logs -f echo-server-v1-59ff75d58-t4dq6
Listening on port 80.
{"name":"echo-server","hostname":"echo-server-v1-59ff75d58-t4dq6","msg":"[GET] - /","time":"2025-04-11T16:26:41.095Z",...}
{"name":"echo-server","hostname":"echo-server-v1-59ff75d58-t4dq6","msg":"[GET] - /","time":"2025-04-11T16:26:42.074Z",...}
Tail v2 logs: 
root@controlplane ~ ➜ kubectl logs -f echo-server-v2-5698db4f99-lm5ss
Listening on port 80.
{"name":"echo-server","hostname":"echo-server-v2-5698db4f99-lm5ss","msg":"[GET] - /","time":"2025-04-11T16:26:23.173Z",...}
{"name":"echo-server","hostname":"echo-server-v2-5698db4f99-lm5ss","msg":"[GET] - /","time":"2025-04-11T16:26:36.848Z",...}

​
5) From a test pod, call the service to see which pod responds

If you don’t yet have a test pod, create one (for example, a busybox or alpine pod with curl). Exec into it and call the ClusterIP service. The echo-server JSON includes a HOSTNAME field — extract it to determine which pod served the response: 
root@controlplane ~ ➜ kubectl exec -ti test -- /bin/bash
root@test:/# curl -s http://echo-server | grep -o '"HOSTNAME":"[^"]*"' | sed 's/"HOSTNAME":"\(.*\)"/HOSTNAME: \1/'
HOSTNAME: echo-server-v2-5698db4f99-lm5ss
root@test:/# curl -s http://echo-server | grep -o '"HOSTNAME":"[^"]*"' | sed 's/"HOSTNAME":"\(.*\)"/HOSTNAME: \1/'
HOSTNAME: echo-server-v2-5698db4f99-lm5ss
root@test:/# curl -s http://echo-server | grep -o '"HOSTNAME":"[^"]*"' | sed 's/"HOSTNAME":"\(.*\)"/HOSTNAME: \1/'
HOSTNAME: echo-server-v1-59ff75d58-t4dq6
root@test:/# exit
You’ll see requests appear in the corresponding pod logs shown earlier.

​
6) Define DestinationRule subsets for v1 and v2

Create a DestinationRule that exposes v1 and v2 subsets for the VirtualService to reference: 
# dr.yaml
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: echo-server
spec:
  host: echo-server
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
Apply and confirm: 
root@controlplane ~ ➜ kubectl apply -f dr.yaml
destinationrule.networking.istio.io/echo-server created

root@controlplane ~ ➜ kubectl get destinationrules.networking.istio.io
NAME        HOST         AGE
echo-server echo-server  3s

​
7) Create a VirtualService that routes to v1 and mirrors to v2

The VirtualService below routes 100% of client traffic to subset v1 and mirrors 100% of the requests to subset v2. Mirrored requests are sent in parallel to the mirror target; the mirror’s responses are discarded by the proxy. 
# vs-mirror.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: echo-server
spec:
  hosts:
  - echo-server
  http:
  - route:
    - destination:
        host: echo-server
        subset: v1
      weight: 100
    mirror:
      host: echo-server
      subset: v2
    mirrorPercentage:
      value: 100.0
Apply the VirtualService: 
root@controlplane ~ ➜ kubectl apply -f vs-mirror.yaml
virtualservice.networking.istio.io/echo-server created

​
8) Test mirroring behavior

From the test pod, invoke the service again. All client responses should now come from v1, while identical mirrored requests are visible in v2 logs: 
root@controlplane ~ ➜ kubectl exec -ti test -- /bin/bash
root@test:/# curl -s http://echo-server | grep -o '"HOSTNAME":"[^"]*"' | sed 's/"HOSTNAME":"\(.*\)"/HOSTNAME: \1/'
HOSTNAME: echo-server-v1-59ff75d58-t4dq6
root@test:/# curl -s http://echo-server | grep -o '"HOSTNAME":"[^"]*"' | sed 's/"HOSTNAME":"\(.*\)"/HOSTNAME: \1/'
HOSTNAME: echo-server-v1-59ff75d58-t4dq6
root@test:/# curl -s http://echo-server | grep -o '"HOSTNAME":"[^"]*"' | sed 's/"HOSTNAME":"\(.*\)"/HOSTNAME: \1/'
HOSTNAME: echo-server-v1-59ff75d58-t4dq6
root@test:/# exit
Check the pod logs you left open — you’ll see matching request entries in both v1 and v2 logs. v1 serves the client; v2 receives mirrored traffic.
Traffic mirroring is a safe way to validate a new revision or collect telemetry. Mirrored requests are sent to the mirror target, but the proxy ignores mirror responses — client behavior is unaffected.
If examples reference Gateway API CRDs (common in some docs), ensure the Gateway CRDs are installed before applying manifests that rely on them. Example command to install a Gateway API release if CRDs are missing:
kubectl get crd gateways.gateway.networking.k8s.io &> /dev/null || \
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/standard/gateway.networking.k8s.io_crd.yaml

​
Quick reference — key resources demonstrated

ResourcePurposeExample
DeploymentTwo revisions with version labels used as subsetsecho-server-v1, echo-server-v2
Service (ClusterIP)Selects pods by shared app label so traffic reaches both revisionsecho-server
DestinationRuleDefines v1 and v2 subsets that VirtualService referencesdr.yaml
VirtualServiceRoutes client traffic to v1 and mirrors to v2vs-mirror.yaml
This completes the mirroring demonstration. Try adjusting weights or mirror percentages to experiment with partial mirroring and collect telemetry from mirrored revisions without altering client-facing behavior.

Watch Video

Practice Lab