Skip to main content
Canary deployments route a small percentage of production traffic to a new release while monitoring metrics. Kubernetes Services alone cannot perform precise, HTTP-level weighted routing, so a service mesh (e.g., Istio) is commonly used to enable L7 (layer-seven) traffic control like weighted routing, header-based routing, traffic mirroring, and fault injection. In this lesson we’ll:
  • Explain why Kubernetes Services are insufficient for precise canary percentages.
  • Show how sidecar proxies enable L7 routing.
  • Configure Istio VirtualService and DestinationRule for canary routing.
  • Demonstrate how Argo Rollouts automates canary weight changes and analysis.
The image lists four learning objectives related to Kubernetes, service mesh, Istio, and Argo Rollouts. It includes understanding routing, traffic control, and integration features.

​
Scenario: Why a 5% Canary Failed

A platform team attempted a 5% canary using only a Kubernetes Service. Their cluster had 19 stable pods (v1) and 1 canary pod (v2). They expected roughly 5% of requests to go to v2 because the canary made up 1/20 endpoints. However, Kubernetes Services only load-balance at layer four (L4), so any percentage is only an approximation based on endpoint count—not a precise, HTTP-aware distribution.
The image illustrates that Kubernetes services, using round-robin load balancing, cannot split traffic effectively between stable and canary versions, with 95% going to stable and 5% to canary.

​
Why the L4-only approach fails

  • No precise traffic control: Services balance connections across endpoints; percentages can only be approximated by changing replica counts.
  • No user-based routing: Services cannot target specific users (for example, based on cookies or headers).
  • No request-level mirroring or observability: You cannot copy HTTP requests to a canary without traffic-aware proxies.
  • Limited routing logic: Services cannot inspect HTTP paths, headers, or implement weighted percentages at L7.
Table: Limitations of Kubernetes Service for Canary Routing A Kubernetes Service will route to pods roughly equally: e.g., two v1 pods and one v2 pod -> v1 ≈ 66%, v2 ≈ 33%. You cannot reliably enforce 95%/5% with a Service alone.

​
How a Service Mesh Solves This

Istio (as an example) injects an Envoy sidecar proxy into each pod. Envoy intercepts all inbound/outbound traffic and applies HTTP-aware routing configured by Istiod (the control plane). Because Envoy operates at L7 it can:
  • Apply exact weighted routing (e.g., 95% stable, 5% canary).
  • Route based on headers, cookies, or query params.
  • Mirror traffic to canary for testing without impacting users.
  • Inject faults (latency, errors) for resilience testing.
The image illustrates a service mesh traffic control setup with a control plane (Istiod) configuring proxies for two pods, each using Envoy. It explains that Envoy intercepts all traffic and applies routing rules.
Service mesh features commonly used for progressive delivery:
The image outlines four service mesh traffic control capabilities: weighted routing, header-based routing, traffic mirroring, and fault injection, each with a brief description.
  • Weighted routing: precise percentage-based routing independent of replica counts.
  • Header-based routing: target specific users or groups via cookies/headers.
  • Traffic mirroring: duplicate requests to canary for testing/perf analysis.
  • Fault injection: simulate failures to validate resilience.
These capabilities are declared as YAML resources (e.g., Istio VirtualService and DestinationRule), which can be stored in Git for GitOps-style deployments.

​
Istio objects used for Canary Routing

Three Istio resources typically participate in a canary: Key insight: the mesh can send exactly 5% of traffic to a single canary pod while the remaining 95% goes to stable pods, because Envoy applies weights at request time.
The image is a diagram explaining Istio VirtualService for Canary deployments, highlighting components like Virtual Service, Destination Rule, and Service, along with their roles in traffic management.

​
Example Istio configuration

VirtualService — the http route array contains destination entries with host, subset, and weight. Weights are interpreted as percentages and typically add up to 100 (proxies will normalize values if needed).
DestinationRule — define subsets that map to pod labels. The VirtualService references these subset names.
Weights in the VirtualService are interpreted as percentages (typically summing to 100); the routing decision is made by the sidecar proxies (Envoy) based on these weights, not by the Kubernetes Service.
Ensure sidecar injection is enabled for the workloads and that subset labels in the DestinationRule match Pod labels exactly. Otherwise, istiod/Envoy cannot route to the intended subset and traffic will fall back to the default route.

​
Automating canaries with Argo Rollouts

Argo Rollouts can orchestrate canary progression by updating Istio VirtualService weights and running automated analysis. Typical flow:
  1. Argo Rollouts creates a canary ReplicaSet alongside the stable ReplicaSet.
  2. At each step, Argo updates the Istio VirtualService weights (e.g., 5%, then 50%, then 100%).
  3. After each weight change, Argo runs analysis (for example, Prometheus queries) to evaluate metrics like error rate or latency.
  4. If analysis fails, Argo Rollouts can automatically roll back by reverting VirtualService weights and scaling down the canary ReplicaSet.
The image outlines the integration of Argo Rollouts with Istio, highlighting features like creating canary ReplicaSets, updating VirtualService weights, running analysis with Prometheus, and enabling auto-rollback.
This combination — Istio’s L7 routing (Envoy) and Argo Rollouts’ automation and analysis — enables safe, automated progressive delivery with precise traffic control and request-level observability.

Watch Video