Configuring your mesh and multi-tenancy

Uses: Kong Mesh

Creating a mesh

To create a Mesh resource, you only need to specify a name:

apiVersion: kuma.io/v1alpha1
kind: Mesh
metadata:
  name: default

Copied!

Apply the configuration with kubectl apply -f [..].

type: Mesh
name: default

Copied!

Apply the configuration with kumactl apply -f [..] or via the HTTP API.

Creating resources in a mesh

When creating other resources, you can add them to the mesh in the following ways.

Data plane proxies

When starting a data plane proxy, specify which mesh it belongs to:

Use the kuma.io/mesh annotation in a Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  template:
    metadata:
      annotations:
        # indicates to Kong Mesh which Mesh the data plane proxy belongs to
        kuma.io/mesh: default

Copied!

A Mesh resource can span multiple Kubernetes namespaces. Any Kong Mesh resource in the cluster which specifies a particular mesh will be part of that mesh.

v2.13+ Namespace-mesh constraint

A single Kubernetes namespace can’t contain Pods in multiple meshes. This limitation exists because Workload resources are mesh-scoped and generated from the app.kubernetes.io/name label, which can cause resource collisions when the same workload name is used across different meshes in the same namespace.

When Kong Mesh detects multiple meshes in a single namespace, it:

Emits a Kubernetes warning event on the namespace

Skips Workload resource generation for affected workloads

Logs an error message

To prevent this configuration issue proactively, you can enable the runtime flag runtime.kubernetes.disallowMultipleMeshesPerNamespace. When enabled, the admission webhook rejects Pod creation and Pod updates if the namespace already contains data planes in a different mesh.

We recommend keeping all pods in a single namespace within the same mesh.

Use the -m or --mesh argument when running kuma-dp:

kuma-dp run \
  --name=backend-1 \
  --mesh=default \
  --cp-address=https://127.0.0.1:5678 \
  --dataplane-token-file=/tmp/kuma-dp-backend-1-token

Copied!

You can control which data plane proxies are allowed to join the mesh using mesh constraints.

Policies

When creating new policies, you must also specify which mesh they belong to:

Use the kuma.io/mesh label:

apiVersion: kuma.io/v1alpha1
kind: MeshHTTPRoute
metadata:
  name: route-1
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default

Copied!

Kong Mesh consumes all policies on the cluster and joins each to an individual mesh, based on this label.

Use the mesh property:

type: MeshHTTPRoute
name: route-1
mesh: default

Copied!

Default resources

To help users get started, Kong Mesh creates the following default policies:

MeshTimeout for all gateways
MeshTimeout for all sidecar resources
MeshRetry
MeshCircuitBreaker

apiVersion: kuma.io/v1alpha1
kind: MeshTimeout
metadata:
  name: mesh-gateways-timeout-all-default
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Mesh
    proxyTypes:
    - Gateway
  to:
  - targetRef:
      kind: Mesh
    default:
      idleTimeout: 1h
      http:
        streamIdleTimeout: 5s
  from:
  - targetRef:
      kind: Mesh
    default:
      idleTimeout: 5m
      http:
        streamIdleTimeout: 5s
        requestHeadersTimeout: 500ms
---
apiVersion: kuma.io/v1alpha1
kind: MeshTimeout
metadata:
  name: mesh-timeout-all-default
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Mesh
    proxyTypes:
    - Sidecar
  to:
  - targetRef:
      kind: Mesh
    default:
      connectionTimeout: 5s
      idleTimeout: 1h
      http:
        requestTimeout: 15s
        streamIdleTimeout: 30m
  from:
  - targetRef:
      kind: Mesh
    default:
      connectionTimeout: 10s
      idleTimeout: 2h
      http:
        requestTimeout: 0s
        streamIdleTimeout: 1h
        maxStreamDuration: 0s
---
apiVersion: kuma.io/v1alpha1
kind: MeshRetry
metadata:
  name: mesh-retry-all-default
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Mesh
  to:
  - targetRef:
      kind: Mesh
    default:
      tcp:
        maxConnectAttempt: 5
      http:
        numRetries: 5
        perTryTimeout: 16s
        backOff:
          baseInterval: 25ms
          maxInterval: 250ms
      grpc:
        numRetries: 5
        perTryTimeout: 16s
        backOff:
          baseInterval: 25ms
          maxInterval: 250ms
---
apiVersion: kuma.io/v1alpha1
kind: MeshCircuitBreaker
metadata:
  name: mesh-circuit-breaker-all-default
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Mesh
  to:
  - targetRef:
      kind: Mesh
    default:
      connectionLimits:
        maxConnections: 1024
        maxPendingRequests: 1024
        maxRetries: 3
        maxRequests: 1024

Copied!

type: MeshTimeout
mesh: default
name: mesh-gateways-timeout-all-default
spec:
  targetRef:
    kind: Mesh
    proxyTypes:
    - Gateway
  to:
  - targetRef:
      kind: Mesh
    default:
      idleTimeout: 1h
      http:
        streamIdleTimeout: 5s
  from:
  - targetRef:
      kind: Mesh
    default:
      idleTimeout: 5m
      http:
        streamIdleTimeout: 5s
        requestHeadersTimeout: 500ms
---
type: MeshTimeout
mesh: default
name: mesh-timeout-all-default
spec:
  targetRef:
    kind: Mesh
    proxyTypes:
    - Sidecar
  to:
  - targetRef:
      kind: Mesh
    default:
      connectionTimeout: 5s
      idleTimeout: 1h
      http:
        requestTimeout: 15s
        streamIdleTimeout: 30m
  from:
  - targetRef:
      kind: Mesh
    default:
      connectionTimeout: 10s
      idleTimeout: 2h
      http:
        requestTimeout: 0s
        streamIdleTimeout: 1h
        maxStreamDuration: 0s
---
type: MeshRetry
mesh: default
name: mesh-retry-all-default
spec:
  targetRef:
    kind: Mesh
  to:
  - targetRef:
      kind: Mesh
    default:
      tcp:
        maxConnectAttempt: 5
      http:
        numRetries: 5
        perTryTimeout: 16s
        backOff:
          baseInterval: 25ms
          maxInterval: 250ms
      grpc:
        numRetries: 5
        perTryTimeout: 16s
        backOff:
          baseInterval: 25ms
          maxInterval: 250ms
---
type: MeshCircuitBreaker
mesh: default
name: mesh-circuit-breaker-all-default
spec:
  targetRef:
    kind: Mesh
  to:
  - targetRef:
      kind: Mesh
    default:
      connectionLimits:
        maxConnections: 1024
        maxPendingRequests: 1024
        maxRetries: 3
        maxRequests: 1024

Copied!

Adjust konnect_mesh_control_plane.my_meshcontrolplane.id and konnect_mesh.my_mesh.name according to your current configuration.

resource "konnect_mesh_timeout" "mesh_gateways_timeout_all_default" {
  provider = konnect-beta
  type = "MeshTimeout"
  name = "mesh-gateways-timeout-all-default"
  spec = {
    target_ref = {
      kind = "Mesh"
      proxy_types = [
         = "Gateway"
      ]
    }
    to = [
      {
        target_ref = {
          kind = "Mesh"
        }
        default = {
          idle_timeout = "1h"
          http = {
            stream_idle_timeout = "5s"
          }
        }
      }
    ]
    from = [
      {
        target_ref = {
          kind = "Mesh"
        }
        default = {
          idle_timeout = "5m"
          http = {
            stream_idle_timeout = "5s"
            request_headers_timeout = "500ms"
          }
        }
      }
    ]
  }
  labels   = {
  "kuma.io/mesh" = konnect_mesh.my_mesh.name
  }
  cp_id    = konnect_mesh_control_plane.my_meshcontrolplane.id
  mesh     = konnect_mesh.my_mesh.name
}
resource "konnect_mesh_timeout" "mesh_timeout_all_default" {
  provider = konnect-beta
  type = "MeshTimeout"
  name = "mesh-timeout-all-default"
  spec = {
    target_ref = {
      kind = "Mesh"
      proxy_types = [
         = "Sidecar"
      ]
    }
    to = [
      {
        target_ref = {
          kind = "Mesh"
        }
        default = {
          connection_timeout = "5s"
          idle_timeout = "1h"
          http = {
            request_timeout = "15s"
            stream_idle_timeout = "30m"
          }
        }
      }
    ]
    from = [
      {
        target_ref = {
          kind = "Mesh"
        }
        default = {
          connection_timeout = "10s"
          idle_timeout = "2h"
          http = {
            request_timeout = "0s"
            stream_idle_timeout = "1h"
            max_stream_duration = "0s"
          }
        }
      }
    ]
  }
  labels   = {
  "kuma.io/mesh" = konnect_mesh.my_mesh.name
  }
  cp_id    = konnect_mesh_control_plane.my_meshcontrolplane.id
  mesh     = konnect_mesh.my_mesh.name
}
resource "konnect_mesh_retry" "mesh_retry_all_default" {
  provider = konnect-beta
  type = "MeshRetry"
  name = "mesh-retry-all-default"
  spec = {
    target_ref = {
      kind = "Mesh"
    }
    to = [
      {
        target_ref = {
          kind = "Mesh"
        }
        default = {
          tcp = {
            max_connect_attempt = "5"
          }
          http = {
            num_retries = "5"
            per_try_timeout = "16s"
            back_off = {
              base_interval = "25ms"
              max_interval = "250ms"
            }
          }
          grpc = {
            num_retries = "5"
            per_try_timeout = "16s"
            back_off = {
              base_interval = "25ms"
              max_interval = "250ms"
            }
          }
        }
      }
    ]
  }
  labels   = {
  "kuma.io/mesh" = konnect_mesh.my_mesh.name
  }
  cp_id    = konnect_mesh_control_plane.my_meshcontrolplane.id
  mesh     = konnect_mesh.my_mesh.name
}
resource "konnect_mesh_circuit_breaker" "mesh_circuit_breaker_all_default" {
  provider = konnect-beta
  type = "MeshCircuitBreaker"
  name = "mesh-circuit-breaker-all-default"
  spec = {
    target_ref = {
      kind = "Mesh"
    }
    to = [
      {
        target_ref = {
          kind = "Mesh"
        }
        default = {
          connection_limits = {
            max_connections = "1024"
            max_pending_requests = "1024"
            max_retries = "3"
            max_requests = "1024"
          }
        }
      }
    ]
  }
  labels   = {
  "kuma.io/mesh" = konnect_mesh.my_mesh.name
  }
  cp_id    = konnect_mesh_control_plane.my_meshcontrolplane.id
  mesh     = konnect_mesh.my_mesh.name
}

Copied!

Skipping default resource creation

To prevent these policies from being added when creating a mesh, set skipCreatingInitialPolicies:

apiVersion: kuma.io/v1alpha1
kind: Mesh
metadata:
  name: default
spec:
  skipCreatingInitialPolicies: ['*']

Copied!

type: Mesh
name: default
skipCreatingInitialPolicies: ['*']

Copied!

You can also skip creating the default mesh by setting the following parameter when configuring the control plane: KUMA_DEFAULTS_SKIP_MESH_CREATION=true.

Configuring your mesh and multi-tenancy

Creating a mesh

Creating resources in a mesh

Data plane proxies

Policies

Default resources

Skipping default resource creation

Mesh schema

Help us make these docs great!

Still need help?