1. You Can Calculate Subnets, So Why Won't Your Pods Connect?
You can handle subnet mask calculations, VLAN design, and BGP peering without breaking a sweat. And yet, the moment you start touching a Kubernetes cluster, you suddenly can't figure out why two Pods won't talk to each other. This is far from a rare experience.
Kubernetes networking is built on an abstraction layer that differs fundamentally from traditional network design. Pods are created and destroyed constantly, IP addresses aren't fixed, and there's little room to manually edit routing tables. As the official Kubernetes documentation explains, a Kubernetes Service provides "a stable network endpoint decoupled from the dynamically changing IP addresses of Pods"—a mechanism that overturns the very premise of "fixed IPs plus static routing" that underlies traditional network design.
In other words, understanding Kubernetes networking means learning how to translate a network engineer's existing knowledge of "address design" into Kubernetes' "declarative, dynamic" world. This article walks through the three core components—CNI, Service Mesh, and NetworkPolicy—by contrasting each with its traditional networking counterpart.
That said, you don't necessarily have to work through this translation process from scratch on your own every time. With a managed Kubernetes service like Kubo, CNI selection and baseline NetworkPolicy configuration already come standardized out of the box. Let's start by laying out the fundamentals of Kubernetes networking.
2. The Three Layers of Kubernetes Networking — CNI, Service, and Ingress
Kubernetes networking becomes much easier to grasp as a whole once you break it down into three layers.
CNI (Container Network Interface) — The Foundation of Pod-to-Pod Communication
CNI is the layer responsible for assigning network interfaces to Pods and handling the actual packet forwarding between them. The CNI spec itself is plugin-based, and the choice of CNI plugin at cluster build time significantly changes how Pod-to-Pod communication is implemented—whether through an overlay network or BGP-based routing, for example.
In traditional networking terms, CNI plays a role similar to "a combination of switches and routers." The key difference is that configuration is fundamentally declarative, driven by Kubernetes manifests, rather than physical port configuration.
Service — A Stable Endpoint That Hides a Group of Pods
Pods scale up and down frequently, and their IP addresses change along with them. The Service resource is what absorbs this instability. According to the official Kubernetes documentation, when a Service is created it's assigned a fixed ClusterIP, and a controller continuously scans for Pods matching the selector to keep the corresponding EndpointSlice updated automatically. Clients never need to be aware of individual Pod IPs—the same concept as multiple servers hidden behind a load balancer.
Ingress — The Point of Contact With the Outside World
Ingress handles routing HTTP/HTTPS traffic from outside the cluster. It's conceptually similar to a traditional reverse proxy or load balancer configuration, but in Kubernetes it too is managed declaratively as a manifest.
3. Choosing a CNI in Practice — How Cilium, Calico, and Flannel Differ in Design Philosophy
Choosing a CNI plugin is the area where a network engineer's existing knowledge is most directly applicable—or, alternatively, where the biggest shift in thinking is required. Let's look at three representative plugins.
Cilium — Observability and Security Powered by eBPF
Cilium is a CNI plugin built on eBPF, a Linux kernel feature. According to the official CNCF announcement, Cilium was accepted as a CNCF Incubating project in October 2021 and was promoted to Graduated status in October 2023. The same announcement reports adoption by more than 100 organizations and participation from over 800 individual contributors, making it one of the most active projects in the CNCF behind Kubernetes itself. The CNCF project page likewise positions Cilium as an eBPF-based networking, security, and observability solution.
For traditional network engineers, Cilium's strength lies in its ability to "visualize the flow of packets." Being able to observe Pod-to-Pod communication in real time, in a way that feels similar to tcpdump or NetFlow, is a significant advantage.
Calico — The BGP-Native, "Familiar" Choice
Calico's defining feature is its ability to use BGP (Border Gateway Protocol)-based routing. According to the official Calico documentation, Calico is described as a networking and security solution that supports containers, virtual machines, and bare-metal hosts—extending well beyond Kubernetes to OpenStack and bare-metal environments as well.
BGP-based routing is one of the rare areas where knowledge built up from data center networking transfers directly, making Calico arguably the most "familiar" CNI for network engineers.
Flannel — Focused on Simple Connectivity
Flannel, as described in its official repository, is "a simple layer 3 network fabric designed for Kubernetes." A flanneld agent runs on each node and assigns subnets from a pre-configured address space. Unlike Cilium and Calico, it has no policy engine and focuses purely on providing connectivity. In K3s, Flannel is adopted as the default CNI plugin, with vxlan configured as the default backend.
Kubo Cloud, which is also K3s-based, builds on this same lightweight network foundation while bundling NetworkPolicy support and monitoring as standard—reducing the effort needed to separately assemble the policy control and observability layers that tend to be thin with Flannel alone.
4. What's the Difference Between NetworkPolicy and Service Mesh?
One area that trips up many engineers learning Kubernetes networking is the distinction between NetworkPolicy and Service Mesh.
NetworkPolicy — Access Control at L3/L4
According to the official Kubernetes documentation, NetworkPolicy is a resource that achieves "traffic control at the IP address or port level (OSI layer 3 or 4)," and supports protocols including TCP, UDP, and SCTP. Importantly, the official documentation explicitly states that "network policies are implemented by the network plugin"—meaning that if you're using a CNI plugin that doesn't support NetworkPolicy (Flannel, mentioned earlier, being a prime example), creating a NetworkPolicy resource will have no effect whatsoever.
This is conceptually close to configuring traditional firewall rules. It's easiest to understand as a mechanism for declaratively defining "which Pod (source) is allowed to communicate with which Pod (destination) on which port."
Service Mesh — Encryption, Observability, and Traffic Control at L7
Service Mesh, on the other hand, operates at L7 (the application layer). In the case of Istio, a representative implementation, the official documentation explains that "Istio provides mutual TLS (mTLS) as a full-stack transport authentication solution without requiring any changes to application code." Envoy proxies on both the client and server sides perform a handshake, mutually verify identity, and then encrypt the traffic that follows.
Put another way: if NetworkPolicy is the firewall that decides "whether communication is allowed or not," Service Mesh is the layer that "encrypts the communication itself and makes visible who is talking to whom." The two aren't competing technologies—combining them is what achieves defense covering both L3/L4 and L7.
5. Conclusion — Subnet Thinking Is a Foundation, But It Requires a Shift in Mindset
Kubernetes networking is by no means a technology that renders traditional network design knowledge useless. Understanding of BGP transfers directly to operating Calico, and the mindset behind firewall rule design connects directly to designing NetworkPolicies. At the same time, there's no avoiding a shift in premises: Pods are created and destroyed dynamically, IPs aren't fixed, and configuration is fundamentally managed declaratively through manifests.
Choosing a CNI, and working out the division of labor between NetworkPolicy and Service Mesh—these are design and verification tasks that require real, non-trivial learning investment. Selecting a CNI from scratch on EKS or AKS, confirming NetworkPolicy support, and building out a full monitoring setup all take considerable operational effort.
Kubo is a managed Kubernetes service built on K3s, with baseline CNI and network policy configuration already standardized, so you don't need to work through the complexity of network design from zero. For a 4 vCPU / 8 GB / 40 GB × 3-node configuration, you can run a full-spec Kubernetes environment starting at ¥48,000 per month according to the pricing plan—a substantial cost advantage compared to AWS EKS at ¥82,700 per month.
Of course, Kubo uses the standard Kubernetes API as-is, so any knowledge or experience with Calico, Cilium, or BGP carries over directly. Learning the fundamentals of network design while handing off day-to-day operations to a managed service is a genuinely practical option. If you don't have the bandwidth to work through the details of network design in-house, reaching out to discuss your current setup is one way forward.
Kubernetes networking isn't "a foreign language" to a traditional network engineer—it's "a familiar language with a different grammar."