No SDN Kubernetes

  • Pods are routable on a flat network
  • Pods should see their own routable IP address
  • Nodes can communicate with all containers

Route tables

For this example we have two Kubernetes nodes. The nodes are joined together on the network and each has Docker running using a specified NAT address (using -bip option).

  • You can’t reuse 172.17.1.0/24 anywhere else in your network unless your network routes are segmented (as with a VPC on a cloud provider).
  • You need to manually assign the container subnet on each node and update the route table which can be tedious.
  • If nodes are on separate subnets you may need to setup routes on multiple routers.
  • It may involve a team/resource outside of your control or expertise.

Host routes

Likewise, you can do the same route options locally on each node in the Kubernetes cluster. Let’s look at our two example nodes again.

ip route add $DOCKER_SUBNET via $NODE_IP
  • Nodes need to be connected via layer 2 (single hop)
  • You need to keep track of container subnets and node IP addresses

Other options

While I find it conceptually easier to manually define routes, I understand there are situations that would benefit from having an SDN

  • You don’t have access to route tables (on the router or on the host)
  • Your hosts fluctuate (dynamic scaling group?) and route management is cumbersome

Conclusion

Kubernetes network requirements help solve real world problems with distributed applications. The important thing to know is

--

--

Trying new things. Breaking stuff. Likes open source.

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store