> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Demo Fault Injection

> Demo of Istio VirtualService fault injection (delay and abort) to simulate latency and failures, with step-by-step examples, testing commands, and best-practice guidance.

In this lesson you'll implement two Istio fault-injection patterns: a delay and an abort. These traffic-shaping capabilities are applied at the VirtualService level to simulate latency and failures so you can observe and validate how your application behaves under adverse conditions.

Prerequisites

* A Kubernetes cluster with Istio installed.
* The target namespace must be labeled for Istio sidecar injection (the example uses the `default` namespace).

<Callout icon="lightbulb" color="#1CB2FE">
  Ensure your namespace is labeled for Istio sidecar injection before deploying sample apps. You can label a namespace with:

  ```bash theme={null}
  kubectl label namespace default istio-injection=enabled --overwrite
  ```
</Callout>

Step-by-step demo

1. Verify the default namespace is auto-injected

```bash theme={null}
root@controlplane ~ ➜ kubectl get ns --show-labels
NAME                STATUS   AGE    LABELS
default             Active   21m    istio-injection=enabled,kubernetes.io/metadata.name=default
istio-system        Active   51s    kubernetes.io/metadata.name=istio-system
kube-node-lease     Active   21m    kubernetes.io/metadata.name=kube-node-lease
kube-public         Active   21m    kubernetes.io/metadata.name=kube-public
kube-system         Active   21m    kubernetes.io/metadata.name=kube-system
```

2. Deploy the helloworld sample app and confirm resources are created

```bash theme={null}
root@controlplane ~ ➜ kubectl apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/helloworld/helloworld.yaml
service/helloworld created
deployment.apps/helloworld-v1 created
deployment.apps/helloworld-v2 created
```

3. Create a test Pod (a utility pod with curl that stays running)

```bash theme={null}
root@controlplane ~ ➜ kubectl run test --image=curlimages/curl --command -- sleep 1d
pod/test created
```

4. Confirm pods and services are running

```bash theme={null}
root@controlplane ~ ➜ kubectl get pods
NAME                                    READY   STATUS    RESTARTS   AGE
helloworld-v1-7459d7b54b-lqt16         2/2     Running   0          25s
helloworld-v2-654d97458-7vpz4          2/2     Running   0          25s
test                                    1/1     Running   0          10s

root@controlplane ~ ➜ kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
helloworld   ClusterIP   10.111.116.180  <none>        5000/TCP   37s
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP    22m
```

5. Test the service from the test pod — the sample alternates responses between v1 and v2

```bash theme={null}
root@controlplane ~ ➜ kubectl exec -ti test -- curl http://helloworld:5000/hello
Hello version: v2, instance: helloworld-v2-654d97458-7vpz4

root@controlplane ~ ➜ kubectl exec -ti test -- curl http://helloworld:5000/hello
Hello version: v1, instance: helloworld-v1-7459d7b54b-lqt16
```

Important note about configuration location

* Fault injection is configured on VirtualService HTTP routes. DestinationRule is used for connection-level settings (circuit breaking, TLS settings, subsets), not for HTTP fault injection.

6. Confirm there are no existing VirtualServices or DestinationRules for this service

```bash theme={null}
root@controlplane ~ ➜ kubectl get vs
No resources found in default namespace.

root@controlplane ~ ➜ kubectl get destinationrules.networking.istio.io
No resources found in default namespace.
```

Fault injection patterns overview

| Fault type | Purpose                                                 | VirtualService snippet / example file   |
| ---------- | ------------------------------------------------------- | --------------------------------------- |
| Delay      | Simulate added latency (fixed or percentage of traffic) | `vs-delay.yaml` (example below)         |
| Abort      | Return an HTTP error for all or part of requests        | `vs-abort-500.yaml`, `vs-abort-50.yaml` |

## Delay fault injection

Create a VirtualService that injects a 5-second fixed delay for 100% of requests. Save this as `vs-delay.yaml`:

```yaml theme={null}
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: hello-world-vs
spec:
  hosts:
  - helloworld
  http:
  - fault:
      delay:
        percentage:
          value: 100.0
        fixedDelay: 5s
    route:
    - destination:
        host: helloworld
        port:
          number: 5000
```

Apply the VirtualService:

```bash theme={null}
root@controlplane ~ ➜ kubectl apply -f vs-delay.yaml
virtualservice.networking.istio.io/hello-world-vs created
```

Test the delay from the test pod — you should observe roughly a 5-second pause on each request:

```bash theme={null}
root@controlplane ~ ➜ kubectl exec -ti test -- curl http://helloworld:5000/hello
# ~5 second pause
Hello version: v1, instance: helloworld-v1-7459d7b54b-lqt16
```

## Abort fault injection

Modify the VirtualService to abort requests with a chosen HTTP error status. The example below returns HTTP 500 for 100% of requests; save it as `vs-abort-500.yaml`:

```yaml theme={null}
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: hello-world-vs
spec:
  hosts:
    - helloworld
  http:
    - fault:
        abort:
          percentage:
            value: 100.0
          httpStatus: 500
      route:
        - destination:
            host: helloworld
            port:
              number: 5000
```

Apply the updated VirtualService:

```bash theme={null}
root@controlplane ~ ➜ kubectl apply -f vs-abort-500.yaml
virtualservice.networking.istio.io/hello-world-vs configured
```

Testing the abort:

* A plain curl may contain Envoy’s abort/filter text.
* Use `curl --head` to inspect the HTTP status:

```bash theme={null}
root@controlplane ~ ➜ kubectl exec -ti test -- curl http://helloworld:5000/hello
root@controlplane ~ ➜ kubectl exec -ti test -- curl --head http://helloworld:5000/hello
HTTP/1.1 500 Internal Server Error
content-length: 18
content-type: text/plain
date: Tue, 15 Apr 2025 15:58:03 GMT
server: envoy
```

## Partial aborts (example: 50% traffic)

To abort 50% of requests with HTTP 404, use this VirtualService definition and save it as `vs-abort-50.yaml`:

```yaml theme={null}
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: hello-world-vs
spec:
  hosts:
  - helloworld
  http:
  - fault:
      abort:
        percentage:
          value: 50.0
        httpStatus: 404
    route:
    - destination:
        host: helloworld
        port:
          number: 5000
```

Apply it:

```bash theme={null}
root@controlplane ~ ➜ kubectl apply -f vs-abort-50.yaml
virtualservice.networking.istio.io/hello-world-vs configured
```

Run a short loop from the test pod to observe a mix of 200 and 404 responses:

```bash theme={null}
root@controlplane ~ ➜ kubectl exec -ti test -- /bin/sh -c 'for i in $(seq 1 10); do curl --head helloworld.default.svc:5000/hello; echo "----" ; done'
```

Sample mixed output (200 and 404 responses interleaved):

```text theme={null}
HTTP/1.1 200 OK
server: envoy
date: Tue, 15 Apr 2025 15:59:21 GMT
content-type: text/html; charset=utf-8
content-length: 59
x-envoy-upstream-service-time: 119

----
HTTP/1.1 404 Not Found
content-length: 18
content-type: text/plain
date: Tue, 15 Apr 2025 15:59:21 GMT
server: envoy

----
# ...and so on
```

Partial delays

* You can apply delays to a percentage of traffic the same way you apply partial aborts — replace the `abort` section with the `delay` configuration and specify the percentage.

## Documentation and exam tip

<Callout icon="lightbulb" color="#1CB2FE">
  When you hear "fault injection" think "VirtualService". Fault injection (delay and abort) is configured on VirtualService HTTP routes. Circuit breaking and connection-level settings belong in DestinationRule.
</Callout>

Useful references

* Istio VirtualService HTTPFaultInjection docs: [https://istio.io/latest/docs/reference/config/networking/virtual-service/#HTTPFaultInjection](https://istio.io/latest/docs/reference/config/networking/virtual-service/#HTTPFaultInjection)
* Istio Traffic Management concepts: [https://istio.io/latest/docs/concepts/traffic-management/](https://istio.io/latest/docs/concepts/traffic-management/)

## Wrap-up

This lesson demonstrated how to:

* Inject a fixed delay for all or a percentage of requests.
* Abort requests with a specific HTTP status for all or a percentage of requests.
* Test fault injection behavior from a test pod.

Next steps

* Try combining fault injection with retries and timeouts to simulate real-world resilience patterns.
* Explore weighted routing and subset routing in VirtualService/DestinationRule to target faults at specific versions.

<CardGroup>
  <Card title="Watch Video" icon="video" cta="Learn more" href="https://learn.kodekloud.com/user/courses/istio-certified-associate/module/f3f5ca4b-b8d6-4788-9553-9ed765709933/lesson/3216232f-aca0-45ff-812a-35b9253cb0b4" />

  <Card title="Practice Lab" icon="flask-conical" cta="Learn more" href="https://learn.kodekloud.com/user/courses/istio-certified-associate/module/f3f5ca4b-b8d6-4788-9553-9ed765709933/lesson/4b560f64-c293-4ed4-8c86-497a50ef68df" />
</CardGroup>
