Kubernetes built-in gateways with Kong Mesh

Kong Mesh supports the Kubernetes Gateway API for configuring built-in gateways as well as traffic routing using the experimental GAMMA routing spec.

To learn how to use the Kubernetes Gateway API to deploy a built-in gateway, see Set up a built-in Kubernetes gateway with Kong Mesh.

The Gateway API provides the parametersRef field on GatewayClass.spec to add implementation-specific configuration to the Gateway resource. When using the Kubernetes Gateway API with Kong Mesh, you can refer to a MeshGatewayConfig resource:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: kuma
spec:
  controllerName: gateways.kuma.io/controller
  parametersRef:
    kind: MeshGatewayConfig
    group: kuma.io
    name: kuma

This resource has the same structure as the MeshGatewayInstance resource, but the tags field is optional. With a MeshGatewayConfig, you can then customize the generated Service and Deployment resources.

You can specify a Mesh for Gateway and HTTPRoute resources by setting the kuma.io/mesh annotation

HTTPRoutes must also have the annotation to reference a Gateway from a non-default Mesh.

Cross-mesh gateways are supported with the Gateway API. Create a corresponding GatewayClass pointing to a MeshGatewayConfig that sets crossMesh: true:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: kuma-cross-mesh
spec:
  controllerName: gateways.kuma.io/controller
  parametersRef:
    group: kuma.io
    kind: MeshGatewayConfig
    name: default-cross-mesh
---
apiVersion: kuma.io/v1alpha1
kind: MeshGatewayConfig
metadata:
  name: default-cross-mesh
spec:
  crossMesh: true

Then reference it in your Gateway:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kuma
  namespace: default
spec:
  gatewayClassName: kuma-cross-mesh
  listeners:
  - name: proxy
    port: 8080
    protocol: HTTP

The Gateway API supports multi-zone deployments, but with some limitations:

• Gateway API resources like Gateway, ReferenceGrant, and HTTPRoute must be created in non-global zones.

• Only services deployed within the same Kubernetes cluster, such as the HTTPRoute, can be referenced via backendRef.

  This limitation exists because Kong Mesh currently only allows referencing as backendRefs Kubernetes Services.

  This is a temporary limitation. We’re actively working on extending backendRef to support Kong Mesh’s MeshServices. Once this feature is complete, you’ll be able to reference services across different clusters within your mesh.

  To better visualize this limitation, here’s an example scenario that describes how you could configure multi-zone deployments with the Gateway API. In this example, you have the following resources:

  • Two zones (zone-1 and zone-2) in separate Kubernetes clusters

  • A Gateway with a listener on port 8080 deployed in zone-1

  • Two services:

    • A service named backend deployed in each zone

    • A service named db deployed only in zone-2

   
   flowchart TD
       subgraph c2["k8s-cluster-2"]
           subgraph z2["zone-2"]
               subgraph c1z2s1["Service"]
                   b2(backend)
               end
               subgraph c1z2s2["Service"]
                   db(db)
               end
           end
       end
       subgraph c1["k8s-cluster-1"]
           subgraph z1["zone-1"]
               subgraph Gateway
                   listener(:8080)
               end
               subgraph Service
                   b1(backend)
               end
           end
       end
     

  If you deploy multi-zone with the Gateway API, the following will occur:

  • If you create an HTTPRoute with a backendRef targeting the backend service in k8s-cluster-1, it will only route traffic to the backend service in k8s-cluster-1.

   
   flowchart TD
       subgraph c2["k8s-cluster-2"]
           subgraph z2["zone-2"]
               subgraph c1z2s1["Service"]
                   backend2(backend)
               end
               subgraph c1z2s2["Service"]
                   db(db)
               end
           end
       end
       subgraph c1["k8s-cluster-1"]
           subgraph z1["zone-1"]
               subgraph Service
                   backend1(backend)
               end
               subgraph Gateway
                   listener(:8080)
               end
               subgraph HTTPRoute
                   route1(/)
               end
               route1--"❌"-->backend2
               linkStyle 0 stroke:red,color:red,stroke-dasharray: 5 5;
               route1-->backend1
               listener-->route1
           end
       end
     

  • Similarly, if you create an HTTPRoute with a backendRef pointing to the db service in k8s-cluster-1, it will result in a HTTPRoute with a ResolvedRefs status condition of BackendNotFound because service db is not present in k8s-cluster-1.

   
   flowchart TD
       subgraph c2["k8s-cluster-2"]
           subgraph z2["zone-2"]
               subgraph c1z2s1["Service"]
                   backend2(backend)
               end
               subgraph c1z2s2["Service"]
                   db(db)
               end
           end
       end
       subgraph c1["k8s-cluster-1"]
           subgraph z1["zone-1"]
               subgraph Gateway
                   listener(:8080)
               end
               subgraph HTTPRoute
                   route1(/)
               end
               subgraph Service
                   backend1(backend)
               end
               route1--"❌"-->db
               linkStyle 0 stroke:red,color:red,stroke-dasharray: 5 5;
               listener-->route1
           end
       end
     

Kong Mesh also supports routing between services with HTTPRoute in conformance with the GAMMA specifications.

GAMMA is a Gateway API project focused on mesh implementations of the Gateway API and extending the Gateway API resources to mesh use cases.

The key feature of HTTPRoute for mesh routing is specifying a Kubernetes Service as the parentRef, as opposed to a Gateway. All requests to this Service are then filtered and routed as specified in the HTTPRoute.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: canary-demo-app
  namespace: kuma-demo
spec:
  parentRefs:
  - name: demo-app
    port: 5000
    kind: Service
  rules:
  - backendRefs:
    - name: demo-app-v1
      port: 5000
    - name: demo-app-v2
      port: 5000

The namespace of the HTTPRoute is key. If the route’s namespace and the parentRef’s namespace are identical, Kong Mesh applies the route to requests from all workloads. If the route’s namespace differs from its parentRef’s namespace, the HTTPRoute applies only to requests from workloads in the route’s namespace.

Remember to tag your Service ports with appProtocol: http to use them in an HTTPRoute.

Because of how Kong Mesh currently maps resources, the combination of the HTTPRoute name and namespace and the parent Service name and namespace must be no more than 249 characters.

Kong Mesh includes controllers that reconcile the Gateway API CRDs and convert them into the corresponding Kong Mesh CRDs. This is why in the GUI, Kong Mesh MeshGateways/MeshHTTPRoutes/MeshTCPRoutes are visible and not Kubernetes Gateway API resources.

Kubernetes Gateway API resources serve as the source of truth for Kong Mesh gateways and routes. Any edits to the corresponding Kong Mesh resources are overwritten.