Home / How-to Guides
Edit
Report

Restart Kong Gateway when a mounted certificate changes

Uses: Kong Gateway
Incompatible with
konnect
Related Documentation
Gateway Documentation
Tags
#certificates #cert-manager #kubernetes #helm #tls
Related Resources
Rotate TLS certificates without restarting Kong Gateway
Automate TLS certificate provisioning and rotation with cert-manager
Restart Kong Gateway on Kubernetes
Using SSL certificates in Kong Gateway
TL;DR

Certificates set through kong.conf parameters such as ssl_cert or cluster_cert are read once at startup, so a rotated file on disk is ignored. Install Stakater Reloader and annotate your Deployment with secret.reloader.stakater.com/reload. Reloader triggers a rolling restart whenever the Secret changes. Certificates served from Certificate entities don’t need this.

Prerequisites

A running Kubernetes cluster

You need a running Kubernetes cluster with kubectl configured to point at it. Any cluster works, whether it runs locally or in a cloud provider.

If you don’t have one, create a local cluster with minikube:

minikube start -p kong-demo

Helm

You will need Helm, a package manager for Kubernetes.

Install cert-manager

Install cert-manager in your cluster to issue and rotate certificates automatically:

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm upgrade --install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set crds.enabled=true
kubectl wait -n cert-manager --for=condition=ready pod --all --timeout=90s

A Kong Gateway data plane

Deploy a standalone DB-less Kong Gateway data plane into the kong namespace. The rest of this guide uses the kong-dp release and the values-dp.yaml file created here.

  1. Add the Kong Helm charts:

    helm repo add kong https://charts.konghq.com
helm repo update

  2. Create the kong namespace:

    kubectl create namespace kong --dry-run=client -o yaml | kubectl apply -f -

  3. Create a file named license.json containing your Kong Gateway Enterprise license and store it in a Kubernetes secret:

    kubectl create secret generic kong-enterprise-license --from-file=license=./license.json -n kong

  4. Create a values-dp.yaml file:

    cat <<EOF > values-dp.yaml
# Do not use Kong Ingress Controller
ingressController:
  enabled: false

image:
  repository: kong/kong-gateway
  tag: "3.15"

env:
  # Run without a database
  database: "off"
  LICENSE_DATA:
    valueFrom:
      secretKeyRef:
        name: kong-enterprise-license
        key: license

# In DB-less mode, Kong Gateway only reports itself ready once it has
# built a router, which requires at least one Route in the declarative config.
dblessConfig:
  config: |
    _format_version: "3.0"
    services:
      - name: example
        url: http://example.internal
        routes:
          - name: example-route
            paths:
              - /example

# The data plane handles proxy traffic only
proxy:
  enabled: true
  # This guide reaches the proxy with kubectl port-forward, so it doesn't
  # need an external address
  type: ClusterIP

admin:
  enabled: false

manager:
  enabled: false
EOF

  5. Install the release and wait for it to become ready:

    helm upgrade --install kong-dp kong/kong -n kong --values ./values-dp.yaml --wait

  6. Confirm that the data plane is running:

    kubectl get pods -n kong -l app.kubernetes.io/instance=kong-dp

    You should see one pod with a Running status:

    NAME                            READY   STATUS    RESTARTS   AGE
kong-dp-kong-7cfbc49585-2v4qr   1/1     Running   0          45s

Some Kong Gateway certificates can only be configured through kong.conf. Those values are rendered into the NGINX configuration when a node boots, so if cert-manager rotates the Secret you mounted, the file on disk changes but Kong Gateway keeps serving the old certificate. The node has to be replaced before it reads the new file.

Stakater Reloader watches ConfigMaps and Secrets and performs a rolling restart of the workloads that consume them. In this guide, you’ll mount a cert-manager Secret into a data plane, watch the rotation fail to take effect, then set up Reloader and confirm that the pods are replaced automatically.

This guide uses ssl_cert, the default certificate for the proxy listener, because it’s the easiest to observe with openssl. Reloader restarts pods, so use it only when a certificate has no entity equivalent:

  • admin_ssl_cert
  • admin_gui_ssl_cert
  • status_ssl_cert
  • client_ssl_cert
  • cluster_cert
  • lua_ssl_trusted_certificate

For proxy certificates, including the ssl_cert used in this guide, use Certificate and SNI entities in production instead. They rotate with no restart at all.

Create a cert-manager Issuer

The Issuer resource represents the certificate authority that signs your certificates. Create a self-signed issuer in the kong namespace:

echo '
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: selfsigned-issuer
  namespace: kong
spec:
  selfSigned: {}' | kubectl apply -f -

A self-signed issuer keeps this guide self-contained. In production, use an ACME issuer such as Let’s Encrypt, or a CA issuer. For all issuer types, see the cert-manager configuration documentation.

Issue a certificate

  1. Create a Certificate for the demo.example.com hostname. cert-manager writes the key pair to a Secret named demo-example-com and renews it automatically:

    echo '
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: demo-example-com
  namespace: kong
spec:
  secretName: demo-example-com
  duration: 24h
  renewBefore: 12h
  issuerRef:
    name: selfsigned-issuer
    kind: Issuer
  dnsNames:
    - demo.example.com' | kubectl apply -f -

  2. Wait for the certificate to be issued:

    kubectl wait --for=condition=Ready certificate/demo-example-com -n kong --timeout=90s

Mount the Secret into the data plane

  1. Create a values-cert.yaml file that mounts the Secret and serves it from the proxy listener:

    cat <<EOF > values-cert.yaml
# Mount the certificate cert-manager issued
secretVolumes:
  - demo-example-com

env:
  ssl_cert: /etc/secrets/demo-example-com/tls.crt
  ssl_cert_key: /etc/secrets/demo-example-com/tls.key
EOF

    Keep this in a separate file and layer it on top of values-dp.yaml rather than editing values-dp.yaml in place. Helm merges multiple --values files from left to right, so the data plane keeps the settings it was installed with.

    Don’t point status_ssl_cert at a mounted certificate in this setup. The chart’s readiness and liveness probes target the status port over plain HTTP, so turning it into a TLS listener stops the pod from becoming ready.

  2. Apply the change and wait for the rollout:

    helm upgrade kong-dp kong/kong -n kong --values ./values-dp.yaml --values ./values-cert.yaml --wait

  3. Store the pod name and confirm that the certificate is mounted:

    export DP_POD=$(kubectl get pods -n kong -l app.kubernetes.io/instance=kong-dp --sort-by=.metadata.creationTimestamp -o jsonpath='{.items[-1:].metadata.name}')
kubectl exec -n kong $DP_POD -- ls /etc/secrets/demo-example-com

    You should see the three files from the Secret:

    ca.crt
tls.crt
tls.key

Verify that the rotation is broken

  1. Open a port forward to the proxy’s TLS listener and record the serial number of the certificate it serves:

    kubectl port-forward -n kong $DP_POD 8443:8443 > /dev/null &
sleep 3
export OLD_SERIAL=$(kubectl exec -n kong $DP_POD -- openssl x509 -in /etc/secrets/demo-example-com/tls.crt -noout -serial)
echo "" | openssl s_client -connect localhost:8443 -servername demo.example.com 2>/dev/null | openssl x509 -noout -serial -dates

    The output looks like this:

    serial=5B2C9A1E7F04D8B3A6E1C0F29D3847BA
notBefore=Aug  7 10:12:04 2026 GMT
notAfter=Aug  8 10:12:04 2026 GMT

  2. Force cert-manager to reissue the certificate by deleting the Secret. cert-manager detects that the Certificate no longer has a valid Secret and reissues it immediately:

    kubectl delete secret demo-example-com -n kong
kubectl wait --for=condition=Ready certificate/demo-example-com -n kong --timeout=90s

  3. Wait for the kubelet to sync the new Secret into the pod:

    until [ "$(kubectl exec -n kong $DP_POD -- openssl x509 -in /etc/secrets/demo-example-com/tls.crt -noout -serial)" != "$OLD_SERIAL" ]; do
  echo "waiting for the kubelet to sync the new Secret..."
  sleep 10
done

  4. Compare the file on disk with what the listener serves:

    kubectl exec -n kong $DP_POD -- openssl x509 -in /etc/secrets/demo-example-com/tls.crt -noout -serial
echo "" | openssl s_client -connect localhost:8443 -servername demo.example.com 2>/dev/null | openssl x509 -noout -serial

    The file on disk has the new serial number, but the listener still serves the old one. Kong Gateway read the file at startup and hasn’t looked at it since.

  5. Close the port forward:

    kill %1

Install Reloader

Install Reloader into its own namespace and wait for it to be ready:

helm repo add stakater https://stakater.github.io/stakater-charts
helm repo update
helm upgrade --install reloader stakater/reloader -n reloader --create-namespace \
  --set reloader.reloadOnCreate=true
kubectl wait -n reloader --for=condition=ready pod --all --timeout=90s

Annotate the deployment

Add the secret.reloader.stakater.com/reload annotation to the data plane deployment, listing the Secrets it should watch.

  1. Append the annotation to values-cert.yaml:

    cat <<EOF >> values-cert.yaml

deploymentAnnotations:
  secret.reloader.stakater.com/reload: "demo-example-com"
EOF

  2. Apply the change:

    helm upgrade kong-dp kong/kong -n kong --values ./values-dp.yaml --values ./values-cert.yaml --wait

  3. Confirm that the annotation reached the deployment:

    kubectl get deployment kong-dp-kong -n kong -o jsonpath='{.metadata.annotations}'

    The output should include your annotation:

    {"secret.reloader.stakater.com/reload":"demo-example-com"}

Validate the rotation

  1. Record the current pod name and age:

    kubectl get pods -n kong -l app.kubernetes.io/instance=kong-dp

  2. Force another renewal:

    kubectl delete secret demo-example-com -n kong
kubectl wait --for=condition=Ready certificate/demo-example-com -n kong --timeout=90s

  3. Watch the rollout. Reloader detects the change and updates the deployment, which replaces the pods:

    kubectl rollout status deployment/kong-dp-kong -n kong --timeout=300s

  4. Confirm that the pod was replaced:

    kubectl get pods -n kong -l app.kubernetes.io/instance=kong-dp

    The pod name and age have changed and the number of restarts is 0 because the pod is new.

  5. Confirm that the new certificate is now being served:

    export DP_POD=$(kubectl get pods -n kong -l app.kubernetes.io/instance=kong-dp --sort-by=.metadata.creationTimestamp -o jsonpath='{.items[-1:].metadata.name}')
kubectl port-forward -n kong $DP_POD 8443:8443 > /dev/null &
sleep 3
echo "" | openssl s_client -connect localhost:8443 -servername demo.example.com 2>/dev/null | openssl x509 -noout -serial -dates
kubectl exec -n kong $DP_POD -- openssl x509 -in /etc/secrets/demo-example-com/tls.crt -noout -serial
kill %1

    The two serial numbers should now match. Kong Gateway is serving the rotated certificate.

Reloader restarts the whole deployment, so every kong.conf value is re-read, not just the certificate. Roll out configuration changes deliberately, and keep at least two replicas so a rotation doesn’t take your data plane offline. If you’re using a single replica, like in this guide, expect a gap in service while the pod is replaced.

Reloader triggers the rollout by patching an environment variable into the pod template, so you’ll see a STAKATER_ variable on the deployment that isn’t in your values file. It’s reset by your next helm upgrade, and Reloader adds it again on the following rotation.

kong reload would also pick up the rotated file, without dropping connections, but it only affects the container you run it in and the change is lost when the pod is next rescheduled. Replacing the pods is what keeps the running data plane and its manifest in agreement. For more information, see Restart Kong Gateway on Kubernetes.

Cleanup

Uninstall Reloader

 
helm uninstall reloader -n reloader
kubectl delete namespace reloader

Uninstall Kong Gateway

 
helm uninstall kong-dp -n kong
kubectl delete namespace kong

Uninstall cert-manager

 
helm uninstall cert-manager -n cert-manager
kubectl delete namespace cert-manager
Something wrong?
Report an Issue | Edit this Page
View all Kong documentation

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!
Contribute

Still need help?

Ask in our Slack Community
Contact Support