Configure the Kong Mesh CNI

Uses: Kong Mesh

Installation

Below are the details of how to set up Kong Mesh CNI in different environments using both kumactl and helm.

    
      
      
      
    
      
      
      
    
  kumactl install control-plane \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/opt/cni/bin" \
  --set "kuma.cni.confName=05-cilium.conflist" \
  | kubectl apply -f -
# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.10/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/opt/cni/bin" \
  --set "kuma.cni.confName=05-cilium.conflist" \
  kong-mesh kong-mesh/kong-mesh

You need to set the Cilium config value cni-exclusive or the corresponding Helm chart value cni.exclusive to false in order to use Cilum and Kong Mesh together. This is necessary starting with the release of Cilium v1.14.

For installing Kong Mesh CNI with Cilium on GKE, you should follow the Google - GKE section.

For Cilium versions < 1.14 you should use kuma.cni.confName=05-cilium.conf as this has changed for version starting from Cilium 1.14.

    
      
      
      
    
      
      
      
    
  kumactl install control-plane \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/opt/cni/bin" \
  --set "kuma.cni.confName=10-calico.conflist" \
  | kubectl apply -f -
# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.10/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/opt/cni/bin" \
  --set "kuma.cni.confName=10-calico.conflist" \
  kong-mesh kong-mesh/kong-mesh

For installing Kong Mesh CNI with Calico on GKE, you should follow the Google - GKE section.

    
      
      
      
    
      
      
      
    
  kumactl install control-plane \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/var/lib/rancher/k3s/agent/etc/cni/net.d" \
  --set "kuma.cni.binDir=/bin" \
  --set "kuma.cni.confName=10-flannel.conflist" \
  | kubectl apply -f -
# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.10/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/var/lib/rancher/k3s/agent/etc/cni/net.d" \
  --set "kuma.cni.binDir=/bin" \
  --set "kuma.cni.confName=10-flannel.conflist" \
  kong-mesh kong-mesh/kong-mesh

    
      
      
      
    
      
      
      
    
  kumactl install control-plane \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/opt/cni/bin" \
  --set "kuma.cni.confName=10-kindnet.conflist" \
  | kubectl apply -f -
# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.10/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/opt/cni/bin" \
  --set "kuma.cni.confName=10-kindnet.conflist" \
  kong-mesh kong-mesh/kong-mesh

    
      
      
      
    
      
      
      
    
  kumactl install control-plane \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/opt/cni/bin" \
  --set "kuma.cni.confName=10-azure.conflist" \
  | kubectl apply -f -
# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.10/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/opt/cni/bin" \
  --set "kuma.cni.confName=10-azure.conflist" \
  kong-mesh kong-mesh/kong-mesh

    
      
      
      
    
      
      
      
    
  kumactl install control-plane \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/opt/cni/bin" \
  --set "kuma.cni.confName=15-azure-swift-overlay.conflist" \
  | kubectl apply -f -
# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.10/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/opt/cni/bin" \
  --set "kuma.cni.confName=15-azure-swift-overlay.conflist" \
  kong-mesh kong-mesh/kong-mesh

    
      
      
      
    
      
      
      
    
  kumactl install control-plane \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/opt/cni/bin" \
  --set "kuma.cni.confName=10-aws.conflist" \
  --set "kuma.controlPlane.envVars.KUMA_RUNTIME_KUBERNETES_INJECTOR_SIDECAR_CONTAINER_IP_FAMILY_MODE=ipv4" \
  | kubectl apply -f -
# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.10/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/opt/cni/bin" \
  --set "kuma.cni.confName=10-aws.conflist" \
  --set "kuma.controlPlane.envVars.KUMA_RUNTIME_KUBERNETES_INJECTOR_SIDECAR_CONTAINER_IP_FAMILY_MODE=ipv4" \
  kong-mesh kong-mesh/kong-mesh

Add KUMA_RUNTIME_KUBERNETES_INJECTOR_SIDECAR_CONTAINER_IP_FAMILY_MODE=ipv4 as EKS has IPv6 disabled by default.

You need to enable network-policy in your cluster (for existing clusters this redeploys the nodes).

Define the Variable CNI_CONF_NAME by your CNI, like: export CNI_CONF_NAME=05-cilium.conflist or export CNI_CONF_NAME=10-calico.conflist

    
      
      
      
    
      
      
      
    
  kumactl install control-plane \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/home/kubernetes/bin" \
  --set "kuma.cni.confName=${CNI_CONF_NAME}" \
  | kubectl apply -f -
# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.10/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.chained=true" \
  --set "kuma.cni.netDir=/etc/cni/net.d" \
  --set "kuma.cni.binDir=/home/kubernetes/bin" \
  --set "kuma.cni.confName=${CNI_CONF_NAME}" \
  kong-mesh kong-mesh/kong-mesh

Follow the instructions in OpenShift 3.11 installation to get the MutatingAdmissionWebhook and ValidatingAdmissionWebhook enabled (this is required for regular Kong Mesh installation).
You need to grant privileged permission to kuma-cni service account:

oc adm policy add-scc-to-user privileged -z kuma-cni -n kube-system

    
  kumactl install control-plane \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.containerSecurityContext.privileged=true" \
  | kubectl apply -f -
# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.10/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.containerSecurityContext.privileged=true" \
  kong-mesh kong-mesh/kong-mesh

    
  kumactl install control-plane \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.containerSecurityContext.privileged=true" \
  | kubectl apply -f -
# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.10/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.cni.enabled=true" \
  --set "kuma.cni.containerSecurityContext.privileged=true" \
  kong-mesh kong-mesh/kong-mesh

Kong Mesh CNI taint controller

To prevent a race condition described in this issue a new controller was implemented. The controller will taint a node with NoSchedule taint to prevent scheduling before the CNI DaemonSet is running and ready. Once the CNI DaemonSet is running and ready it will remove the taint and allow other pods to be scheduled into the node.

To disable the taint controller use the following env variable:

KUMA_RUNTIME_KUBERNETES_NODE_TAINT_CONTROLLER_ENABLED="false"

Merbridge CNI with eBPF

To install merbridge CNI with eBPF append the following options to the command from installation:

To use Merbridge CNI with eBPF your environment has to use Kernel >= 5.7 and have cgroup2 available

--set ... \
--set "kuma.cni.enabled=true" \
--set "kuma.experimental.ebpf.enabled=true"

Kong Mesh CNI logs

Logs of the are available via kubectl logs.

eBPF CNI currently doesn’t have support for exposing its logs.

Configure the Kong Mesh CNI

Installation

Kong Mesh CNI taint controller

Merbridge CNI with eBPF

Kong Mesh CNI logs

Help us make these docs great!

Still need help