Explains Cilium routing modes, comparing encapsulation and native routing, their requirements, tradeoffs, configuration, and operational implications for pod-to-pod networking.
This document explains the routing modes supported by Cilium and how they affect pod-to-pod networking across different physical network topologies. You’ll learn the differences between encapsulation (tunnel) mode and native routing mode, the trade-offs for each, configuration examples, and operational requirements.
If Pod 1 (10.244.1.1) sends a packet to Pod 2 (10.244.2.1) and the physical router R1 does not know how to reach 10.244.2.0/24, the packet will be dropped. This is a common real-world issue: underlay networks typically do not know cluster-internal pod CIDRs by default.
CNI solutions (including Cilium) address this mismatch between the cluster overlay and the physical underlay using one of two general approaches:
Encapsulation (tunnel) mode
Native routing mode
We’ll cover encapsulation first, then native routing.
In encapsulation mode, the original pod-originated packet (the inner packet) is wrapped inside an outer packet whose source and destination are the physical node IPs. The underlay only needs to route the outer IPs (the node IPs), not the pod CIDRs.
When the outer packet reaches Node 2, the Cilium agent decapsulates it and forwards the inner packet to the destination pod.
Reply traffic follows the same process in reverse: Pod 2 → Cilium on Node 2 encapsulates (node2_IP → node1_IP) → underlay routes the outer packet → Node 1 decapsulates → Pod 1 receives the inner packet.
Encapsulation typically adds ~50 bytes of overhead (outer IP/UDP + tunnel header for VXLAN/Geneve). That reduces the effective MTU for the inner packet and can cause fragmentation.Mitigation strategies:
Reduce the inner MTU on host interfaces (e.g., subtract tunnel overhead).
Works with simple underlays — only node IP reachability required
Adds overhead and slightly higher latency
No special routing configuration required on the physical network
Reduced effective MTU and possible fragmentation
Portable across on-prem and cloud environments
Packet visibility and debugging are harder (tunnel hides inner packet)
Lower dependency on provider features
More CPU/network processing for encapsulation/decapsulation
Encapsulation is often simpler to deploy because the physical network only needs to reach node IPs. Ensure firewall rules and MTU settings are adjusted for the chosen tunnel protocol (VXLAN/Geneve).
In native routing mode, pods’ IPs are routed across the physical network without encapsulation. The inner packet travels across the underlay unchanged, so routers must know how to reach each node’s pod CIDRs.If Pod 1 sends to Pod 2, the packet leaves Node 1 with src=10.244.1.1 and dst=10.244.2.1. The physical router R1 must have a route pointing to Node 2’s pod CIDR (10.244.2.0/24) for the packet to be delivered.
To make native routing practical at scale, the underlay must learn pod CIDRs. Common approaches:
Static routes — simple but not scalable or fault-tolerant.
Dynamic routing (BGP) — Cilium can advertise pod CIDRs into the physical fabric using BGP so routers learn where to forward pod traffic.
SDN/orchestration solutions that distribute routes to the underlay.
When Cilium advertises pod routes (for example via BGP), physical routers (R1/R2) learn the pod CIDRs and forward pod-to-pod traffic natively without encapsulation.
Cilium defaults to encapsulation. To enable native routing, set the routing mode to “native” and configure which CIDR ranges should be treated as natively routable. This allows you to mix modes: some pod CIDRs can be native while others remain encapsulated.Example Cilium configuration snippet:
Copy
# cilium-config snippet (example)routingMode: "native"# IPv4 CIDR(s) that will use native routing (e.g., your cluster Pod CIDR)ipv4NativeRoutingCIDR: "10.244.0.0/16"ipv6NativeRoutingCIDR: ""
Notes about these settings:
ipv4NativeRoutingCIDR defines which IP range(s) Cilium advertises or treats as natively routed.
Traffic within that CIDR will be forwarded natively by the underlay; traffic outside falls back to encapsulation.
You can restrict the range (for example, a /17) to mix native and encapsulated traffic across different pod groups.
Less portable where providers restrict route advertisement or BGP
Native routing requires careful underlay configuration. If your physical network lacks routes for the pod CIDRs (or your cloud provider blocks route advertisement), pod-to-pod traffic can be dropped. Plan route distribution (e.g., BGP) before switching to native routing.
Encapsulation (tunneling) is simpler to deploy and portable, but adds overhead and MTU considerations.
Native routing removes encapsulation overhead and improves performance but requires route distribution into the physical network (BGP or static routes).
Cilium supports both modes and allows mixing them by CIDR. Choose based on your underlay capabilities, performance goals, and operational constraints.
