Demo Fault Injection

In this lesson you’ll implement two Istio fault-injection patterns: a delay and an abort. These traffic-shaping capabilities are applied at the VirtualService level to simulate latency and failures so you can observe and validate how your application behaves under adverse conditions. Prerequisites

A Kubernetes cluster with Istio installed.
The target namespace must be labeled for Istio sidecar injection (the example uses the default namespace).

Ensure your namespace is labeled for Istio sidecar injection before deploying sample apps. You can label a namespace with:

kubectl label namespace default istio-injection=enabled --overwrite

Step-by-step demo

Verify the default namespace is auto-injected

root@controlplane ~ ➜ kubectl get ns --show-labels
NAME                STATUS   AGE    LABELS
default             Active   21m    istio-injection=enabled,kubernetes.io/metadata.name=default
istio-system        Active   51s    kubernetes.io/metadata.name=istio-system
kube-node-lease     Active   21m    kubernetes.io/metadata.name=kube-node-lease
kube-public         Active   21m    kubernetes.io/metadata.name=kube-public
kube-system         Active   21m    kubernetes.io/metadata.name=kube-system

Deploy the helloworld sample app and confirm resources are created

root@controlplane ~ ➜ kubectl apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/helloworld/helloworld.yaml
service/helloworld created
deployment.apps/helloworld-v1 created
deployment.apps/helloworld-v2 created

Create a test Pod (a utility pod with curl that stays running)

root@controlplane ~ ➜ kubectl run test --image=curlimages/curl --command -- sleep 1d
pod/test created

Confirm pods and services are running

root@controlplane ~ ➜ kubectl get pods
NAME                                    READY   STATUS    RESTARTS   AGE
helloworld-v1-7459d7b54b-lqt16         2/2     Running   0          25s
helloworld-v2-654d97458-7vpz4          2/2     Running   0          25s
test                                    1/1     Running   0          10s

root@controlplane ~ ➜ kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
helloworld   ClusterIP   10.111.116.180  <none>        5000/TCP   37s
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP    22m

Test the service from the test pod — the sample alternates responses between v1 and v2

root@controlplane ~ ➜ kubectl exec -ti test -- curl http://helloworld:5000/hello
Hello version: v2, instance: helloworld-v2-654d97458-7vpz4

root@controlplane ~ ➜ kubectl exec -ti test -- curl http://helloworld:5000/hello
Hello version: v1, instance: helloworld-v1-7459d7b54b-lqt16

Important note about configuration location

Fault injection is configured on VirtualService HTTP routes. DestinationRule is used for connection-level settings (circuit breaking, TLS settings, subsets), not for HTTP fault injection.

Confirm there are no existing VirtualServices or DestinationRules for this service

root@controlplane ~ ➜ kubectl get vs
No resources found in default namespace.

root@controlplane ~ ➜ kubectl get destinationrules.networking.istio.io
No resources found in default namespace.

Fault injection patterns overview

Fault type	Purpose	VirtualService snippet / example file
Delay	Simulate added latency (fixed or percentage of traffic)	`vs-delay.yaml` (example below)
Abort	Return an HTTP error for all or part of requests	`vs-abort-500.yaml`, `vs-abort-50.yaml`

Delay fault injection

Create a VirtualService that injects a 5-second fixed delay for 100% of requests. Save this as vs-delay.yaml:

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: hello-world-vs
spec:
  hosts:
  - helloworld
  http:
  - fault:
      delay:
        percentage:
          value: 100.0
        fixedDelay: 5s
    route:
    - destination:
        host: helloworld
        port:
          number: 5000

Apply the VirtualService:

root@controlplane ~ ➜ kubectl apply -f vs-delay.yaml
virtualservice.networking.istio.io/hello-world-vs created

Test the delay from the test pod — you should observe roughly a 5-second pause on each request:

root@controlplane ~ ➜ kubectl exec -ti test -- curl http://helloworld:5000/hello
# ~5 second pause
Hello version: v1, instance: helloworld-v1-7459d7b54b-lqt16

Abort fault injection

Modify the VirtualService to abort requests with a chosen HTTP error status. The example below returns HTTP 500 for 100% of requests; save it as vs-abort-500.yaml:

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: hello-world-vs
spec:
  hosts:
    - helloworld
  http:
    - fault:
        abort:
          percentage:
            value: 100.0
          httpStatus: 500
      route:
        - destination:
            host: helloworld
            port:
              number: 5000

Apply the updated VirtualService:

root@controlplane ~ ➜ kubectl apply -f vs-abort-500.yaml
virtualservice.networking.istio.io/hello-world-vs configured

Testing the abort:

A plain curl may contain Envoy’s abort/filter text.
Use curl --head to inspect the HTTP status:

root@controlplane ~ ➜ kubectl exec -ti test -- curl http://helloworld:5000/hello
root@controlplane ~ ➜ kubectl exec -ti test -- curl --head http://helloworld:5000/hello
HTTP/1.1 500 Internal Server Error
content-length: 18
content-type: text/plain
date: Tue, 15 Apr 2025 15:58:03 GMT
server: envoy

Partial aborts (example: 50% traffic)

To abort 50% of requests with HTTP 404, use this VirtualService definition and save it as vs-abort-50.yaml:

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: hello-world-vs
spec:
  hosts:
  - helloworld
  http:
  - fault:
      abort:
        percentage:
          value: 50.0
        httpStatus: 404
    route:
    - destination:
        host: helloworld
        port:
          number: 5000

Apply it:

root@controlplane ~ ➜ kubectl apply -f vs-abort-50.yaml
virtualservice.networking.istio.io/hello-world-vs configured

Run a short loop from the test pod to observe a mix of 200 and 404 responses:

root@controlplane ~ ➜ kubectl exec -ti test -- /bin/sh -c 'for i in $(seq 1 10); do curl --head helloworld.default.svc:5000/hello; echo "----" ; done'

Sample mixed output (200 and 404 responses interleaved):

HTTP/1.1 200 OK
server: envoy
date: Tue, 15 Apr 2025 15:59:21 GMT
content-type: text/html; charset=utf-8
content-length: 59
x-envoy-upstream-service-time: 119

----
HTTP/1.1 404 Not Found
content-length: 18
content-type: text/plain
date: Tue, 15 Apr 2025 15:59:21 GMT
server: envoy

----
# ...and so on

Partial delays

You can apply delays to a percentage of traffic the same way you apply partial aborts — replace the abort section with the delay configuration and specify the percentage.

Documentation and exam tip

When you hear “fault injection” think “VirtualService”. Fault injection (delay and abort) is configured on VirtualService HTTP routes. Circuit breaking and connection-level settings belong in DestinationRule.

Useful references

Istio VirtualService HTTPFaultInjection docs: https://istio.io/latest/docs/reference/config/networking/virtual-service/#HTTPFaultInjection
Istio Traffic Management concepts: https://istio.io/latest/docs/concepts/traffic-management/

Wrap-up

This lesson demonstrated how to:

Inject a fixed delay for all or a percentage of requests.
Abort requests with a specific HTTP status for all or a percentage of requests.
Test fault injection behavior from a test pod.

Next steps

Try combining fault injection with retries and timeouts to simulate real-world resilience patterns.
Explore weighted routing and subset routing in VirtualService/DestinationRule to target faults at specific versions.

Demo Fault Injection

Delay fault injection

Abort fault injection

Partial aborts (example: 50% traffic)

Documentation and exam tip

Wrap-up

Watch Video

Practice Lab

Documentation Index

​Delay fault injection

​Abort fault injection

​Partial aborts (example: 50% traffic)

​Documentation and exam tip

​Wrap-up

Watch Video

Practice Lab

Delay fault injection

Abort fault injection

Partial aborts (example: 50% traffic)

Documentation and exam tip

Wrap-up