Home / Kong Ingress Controller / How To
Edit
Report

Rate limiting with Kong Ingress Controller

Uses: Kong Ingress Controller
Deployment Platform
Related Documentation
All KIC Documentation
Tags
#rate-limiting
TL;DR

Create a rate-limiting KongPlugin instance and annotate your Service with the konghq.com/plugins annotation.

Prerequisites

Kong Konnect

If you don’t have a Konnect account, you can get started quickly with our onboarding wizard.

  1. The following Konnect items are required to complete this tutorial:
    • Personal access token (PAT): Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.

  2. Set the personal access token as an environment variable:

    export KONNECT_TOKEN='YOUR KONNECT TOKEN'

Enable the Gateway API

  1. Install the Gateway API CRDs before installing Kong Ingress Controller.

    kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.3.0/standard-install.yaml

  2. Create a Gateway and GatewayClass instance to use.

echo "
apiVersion: v1
kind: Namespace
metadata:
  name: kong
---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: kong
  annotations:
    konghq.com/gatewayclass-unmanaged: 'true'
spec:
  controllerName: konghq.com/kic-gateway-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kong
spec:
  gatewayClassName: kong
  listeners:
  - name: proxy
    port: 80
    protocol: HTTP
    allowedRoutes:
      namespaces:
         from: All
" | kubectl apply -n kong -f -

Create a KIC Control Plane

Use the Konnect API to create a new CLUSTER_TYPE_K8S_INGRESS_CONTROLLER Control Plane:

CONTROL_PLANE_DETAILS=$( curl  -X POST "https://us.api.konghq.com/v2/control-planes" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN" \
     --json '{
       "name": "My KIC CP",
       "cluster_type": "CLUSTER_TYPE_K8S_INGRESS_CONTROLLER"
     }')

We’ll need the id and telemetry_endpoint for the values.yaml file later. Save them as environment variables:

CONTROL_PLANE_ID=$(echo $CONTROL_PLANE_DETAILS | jq -r .id)
CONTROL_PLANE_TELEMETRY=$(echo $CONTROL_PLANE_DETAILS | jq -r '.config.telemetry_endpoint | sub("https://";"")')

Create mTLS certificates

Kong Ingress Controller talks to Konnect over a connected secured with TLS certificates.

Generate a new certificate using openssl:

openssl req -new -x509 -nodes -newkey rsa:2048 -subj "/CN=kongdp/C=US" -keyout ./tls.key -out ./tls.crt

The certificate needs to be a single line string to send it to the Konnect API with curl. Use awk to format the certificate:

export CERT=$(awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' tls.crt);

Next, upload the certificate to Konnect:

 curl  -X POST "https://us.api.konghq.com/v2/control-planes/$CONTROL_PLANE_ID/dp-client-certificates" \
     --no-progress-meter --fail-with-body  \
     -H "Authorization: Bearer $KONNECT_TOKEN" \
     --json '{
       "cert": "'$CERT'"
     }'

Finally, store the certificate in a Kubernetes secret so that Kong Ingress Controller can read it:

kubectl create namespace kong -o yaml --dry-run=client | kubectl apply -f -
kubectl create secret tls konnect-client-tls -n kong --cert=./tls.crt --key=./tls.key

Kong Ingress Controller running (attached to Konnect)

  1. Add the Kong Helm charts:

    helm repo add kong https://charts.konghq.com
helm repo update

  2. Create a values.yaml file:

    cat <<EOF > values.yaml
controller:
  ingressController:
    image:
      tag: "3.5"
    env:
      feature_gates: "FillIDs=true"
    konnect:
      license:
        enabled: true
      enabled: true
      controlPlaneID: "$CONTROL_PLANE_ID"
      tlsClientCertSecretName: konnect-client-tls
      apiHostname: "us.kic.api.konghq.com"
gateway:
  image:
    repository: kong
    tag: "3.9.1"
  env:
    konnect_mode: 'on'
    vitals: "off"
    cluster_mtls: pki
    cluster_telemetry_endpoint: "$CONTROL_PLANE_TELEMETRY:443"
    cluster_telemetry_server_name: "$CONTROL_PLANE_TELEMETRY"
    cluster_cert: /etc/secrets/konnect-client-tls/tls.crt
    cluster_cert_key: /etc/secrets/konnect-client-tls/tls.key
    lua_ssl_trusted_certificate: system
    proxy_access_log: "off"
    dns_stale_ttl: "3600"
  secretVolumes:
     - konnect-client-tls
EOF

  3. Install Kong Ingress Controller using Helm:

    helm install kong kong/ingress -n kong --create-namespace --values ./values.yaml

  4. Set $PROXY_IP as an environment variable for future commands:

    export PROXY_IP=$(kubectl get svc --namespace kong kong-gateway-proxy -o jsonpath='{range .status.loadBalancer.ingress[0]}{@.ip}{@.hostname}{end}')
echo $PROXY_IP

Kong Ingress Controller running

  1. Add the Kong Helm charts:

    helm repo add kong https://charts.konghq.com
helm repo update

  2. Install Kong Ingress Controller using Helm:

    helm install kong kong/ingress -n kong --create-namespace

  3. Set $PROXY_IP as an environment variable for future commands:

    export PROXY_IP=$(kubectl get svc --namespace kong kong-gateway-proxy -o jsonpath='{range .status.loadBalancer.ingress[0]}{@.ip}{@.hostname}{end}')
echo $PROXY_IP

Required Kubernetes resources

This how-to requires some Kubernetes services to be available in your cluster. These services will be used by the resources created in this how-to.

kubectl apply -f https://developer.konghq.com/manifests/kic/echo-service.yaml -n kong

This how-to also requires 1 pre-configured route:

echo "
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: echo
  namespace: kong
  annotations:
    konghq.com/strip-path: 'true'
spec:
  parentRefs:
  - name: kong
    namespace: kong
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /echo
    backendRefs:
    - name: echo
      kind: Service
      port: 1027
" | kubectl apply -f -

Create a Rate Limiting Plugin

To add rate limiting to the echo Service, create a new Rate Limiting KongPlugin:

echo "
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: rate-limit-5-min
  namespace: kong
  annotations:
    kubernetes.io/ingress.class: kong
plugin: rate-limiting
config:
  minute: 5
  policy: local
" | kubectl apply -f -

Next, apply the KongPlugin resource by annotating the service resource:

kubectl annotate -n kong service echo konghq.com/plugins=rate-limit-5-min

Validate your configuration

Send repeated requests to decrement the remaining limit headers, and block requests after the fifth request:

for _ in {1..6}; do
  curl  -i $PROXY_IP/echo  
  echo
done


The RateLimit-Remaining header indicates how many requests are remaining before the rate limit is enforced. The first five responses return HTTP/1.1 200 OK which indicates that the request is allowed. The final request returns HTTP/1.1 429 Too Many Requests and the request is blocked.

If you receive an HTTP 429 from the first request, wait 60 seconds for the rate limit timer to reset.


for _ in {1..6}; do
  curl  -i $PROXY_IP/echo  
  echo
done


The RateLimit-Remaining header indicates how many requests are remaining before the rate limit is enforced. The first five responses return HTTP/1.1 200 OK which indicates that the request is allowed. The final request returns HTTP/1.1 429 Too Many Requests and the request is blocked.

If you receive an HTTP 429 from the first request, wait 60 seconds for the rate limit timer to reset.


Scale to multiple pods

The policy: local setting in the plugin configuration tracks request counters in each Pod’s local memory separately. Counters are not synchronized across Pods, so clients can send requests past the limit without being throttled if they route through different Pods.

To test this, scale your Deployment to three replicas:

kubectl scale --replicas 3 -n kong deployment kong-gateway

It may take up to 30 seconds for the new replicas to come online. Run kubectl get pods -n kong and check the Ready column to validate that the replicas are online

Sending requests to this Service does not reliably decrement the remaining counter:

for _ in {1..10}; do
  curl  -sv $PROXY_IP/echo  2>&1 | grep -E "(< RateLimit-Remaining)"
  echo
done


The requests are distributed across multiple Pods, each with their own in memory rate limiting counter:


< RateLimit-Remaining: 4

< RateLimit-Remaining: 4

< RateLimit-Remaining: 3

< RateLimit-Remaining: 4

< RateLimit-Remaining: 3

< RateLimit-Remaining: 2

< RateLimit-Remaining: 3

< RateLimit-Remaining: 2

< RateLimit-Remaining: 1

< RateLimit-Remaining: 1

for _ in {1..10}; do
  curl  -sv $PROXY_IP/echo  2>&1 | grep -E "(< RateLimit-Remaining)"
  echo
done


The requests are distributed across multiple Pods, each with their own in memory rate limiting counter:


< RateLimit-Remaining: 4

< RateLimit-Remaining: 4

< RateLimit-Remaining: 3

< RateLimit-Remaining: 4

< RateLimit-Remaining: 3

< RateLimit-Remaining: 2

< RateLimit-Remaining: 3

< RateLimit-Remaining: 2

< RateLimit-Remaining: 1

< RateLimit-Remaining: 1

Using a load balancer that distributes client requests to the same Pod can alleviate this somewhat, but changes to the number of replicas can still disrupt accurate accounting. To consistently enforce the limit, the plugin needs to use a shared set of counters across all Pods. The redis policy can do this when a Redis instance is available.

Deploy Redis to your Kubernetes cluster

Redis provides an external database for Kong Gateway components to store shared data, such as rate limiting counters. There are several options to install it.

Bitnami provides a Helm chart for Redis with turnkey options for authentication.

  1. Create a password Secret and replace PASSWORD with a password of your choice.

    kubectl create -n kong secret generic redis-password-secret --from-literal=redis-password=PASSWORD

  2. Install Redis

     helm install -n kong redis oci://registry-1.docker.io/bitnamicharts/redis \
   --set auth.existingSecret=redis-password-secret \
   --set architecture=standalone

    Helm displays the instructions that describes the new installation.

    If Redis is not accessible, Kong Gateway will allow incoming requests. Run kubectl get pods -n kong redis-master-0 and check the Ready column to ensure that Redis is ready before continuing.

  3. Update your plugin configuration with the redis policy, Service, and credentials. Replace PASSWORD with the password that you set for Redis.

     kubectl patch -n kong kongplugin rate-limit-5-min --type json --patch '[
   {
     "op":"replace",
     "path":"/config/policy",
     "value":"redis"
   },
   {
     "op":"add",
     "path":"/config/redis_host",
     "value":"redis-master"
   },
   {
     "op":"add",
     "path":"/config/redis_password",
     "value":"PASSWORD"
   }
 ]'

    If the redis_username is not set, it uses the default redis user.

Test rate limiting in a multi-node deployment

Send the following request to test the rate limiting functionality in the multi-Pod deployment:

for _ in {1..6}; do
  curl  -sv $PROXY_IP/echo  2>&1 | grep -E "(< RateLimit-Remaining|< HTTP)"
  echo
done


The counters decrement sequentially regardless of the Kong Gateway replica count.


< HTTP/1.1 200 OK
< RateLimit-Remaining: 4

< HTTP/1.1 200 OK
< RateLimit-Remaining: 3

< HTTP/1.1 200 OK
< RateLimit-Remaining: 2

< HTTP/1.1 200 OK
< RateLimit-Remaining: 1

< HTTP/1.1 200 OK
< RateLimit-Remaining: 0

< HTTP/1.1 429 Too Many Requests
< RateLimit-Remaining: 0

for _ in {1..6}; do
  curl  -sv $PROXY_IP/echo  2>&1 | grep -E "(< RateLimit-Remaining|< HTTP)"
  echo
done


The counters decrement sequentially regardless of the Kong Gateway replica count.


< HTTP/1.1 200 OK
< RateLimit-Remaining: 4

< HTTP/1.1 200 OK
< RateLimit-Remaining: 3

< HTTP/1.1 200 OK
< RateLimit-Remaining: 2

< HTTP/1.1 200 OK
< RateLimit-Remaining: 1

< HTTP/1.1 200 OK
< RateLimit-Remaining: 0

< HTTP/1.1 429 Too Many Requests
< RateLimit-Remaining: 0

Cleanup

Delete created Kubernetes resources

 
kubectl delete -n kong -f https://developer.konghq.com/manifests/kic/echo-service.yaml

Uninstall KIC from your cluster

 
helm uninstall kong -n kong
Something wrong?
Report an Issue | Edit this Page

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!
Contribute

Still need help

Ask in our Forum
Contact Support