ipv6 routed pods with identity.

Link https://hackmd.io/@mauilion/cilium-ipv6

Youtube link https://www.youtube.com/watch?v=IgPqIi67hSA

Summary:

Setup an cluster and deploy cilium in routed mode.

Show connectivity between pods and then use tcpdump to show identity labels encoded in the flow label header for ipv6 traffic.

Setup.

We will use a kind cluster for this learn more about kind here

the kind configuration we will use is here:

kind-config.yaml

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  disableDefaultCNI: true
  kubeProxyMode: "none"
  ipFamily: dual
nodes:
- role: control-plane
- role: worker
- role: worker
- role: worker

We will use the cilium cli installer to deploy cilium to this cluster.

instructions on getting the installer here

to install cilium into the target cluster configured for our purposes use the folling install command.

This command will employ the cilium cli binary to deploy cilium configured to support a dual stack kubernetes configuration. Specificed above in the kind config file.

cilium install \
  --helm-set \
  k8sServiceHost=kind-control-plane,\
  k8sServicePort=6443,\
  kubeProxyReplacement=strict,\
  ipv6.enabled=true,\ 
  tunnel=disabled,\ 
  ipv4NativeRoutingCIDR=172.18.0.0/16,\
  ipv6NativeRoutingCIDR=fc00:f853:ccd:e793::/64,\ 
  autoDirectNodeRoutes=true

At this point let's deploy some test pods to ensure that we can show the flowlabel being set to the identity of the source traffic.

For this test we are going to use a simple debug image that has an echo server running inside of it.

The following pernode.sh script will deploy one pod per node in the cluster and expose them as a service so that we can ping and send traffic easily across nodes.

The Dockerfile for the mauilion/debug image is here

pernode.sh

#!/bin/bash

# The purpose of this script is to deploy to each node in the cluster 2 pods. 
# Each pod will have an env var that shows it's zone.

function netpod () {
  kubectl run net${2}-${1} \
     --image overridden  --labels app=net,pod=net${2}-${1},node=${1}  --overrides \
    '{
      "spec":{
        "hostname": "net'${2}-${1}'",
	      "subdomain": "net",
        "nodeName": "'$1'",
        "containers":[{
          "name":"net",
          "image":"mauilion/debug"
        }]
      }
    }'
}

for worker in $(kubectl get nodes -o name | sed s/node.//)
  do
    for i in {1..2}
      do netpod $worker $i
    done
  done

kubectl create service clusterip net --tcp 8080 

Once deployed we can send traffic back and forth and see that the flow label tcp option is set to the source identity. It's encoded in hex in the output from tcpdump to convert this and see the decimal value in bash we can use a simple conversion.

echo $(( 16#02e76 ))

we will use kubectl get ciliumendpoints to see the identity value.