Kong Gateway Control Plane and Data Plane communication

When Kong Gateway deployments run either in either managed or hybrid mode, there is a separate Control Plane attached to a Data Plane consisting of one or more Data Plane nodes. Data traveling between Control Planes and Data Planes is secured through a mutual TLS handshake. Data Plane nodes initiate the connection to the Kong Gateway Control Plane. Once the connection is established, the Control Plane can send configuration data to the connected Data Plane nodes. Normally, each Data Plane node maintains a persistent connection with the Control Plane. The node sends a heartbeat to the Control Plane every 30 seconds to keep the connection alive. If it doesn’t receive an answer, it tries to reconnect to the Control Plane after a 5-10 second delay. If communication is interrupted and either side can’t send or receive config, Data Plane nodes can still continue proxying traffic to clients.

Whenever a Data Plane node receives new configuration from the Control Plane, it immediately loads that config into memory. At the same time, it caches the config to the file system. By default, Data Plane nodes store their configuration in an unencrypted LMDB database directory, dbless.lmdb, in Kong Gateway’s prefix path.

When operating as a Data Plane (DP) node, Kong Gateway loads configuration in the following order:

1. Local config cache: If the dbless.lmdb file exists in the kong_prefix directory (default: /usr/local/kong), the Data Plane node uses it as its configuration.
2. Declarative config file: If the cache is missing and declarative_config is set, the node loads the specified file.
3. No configuration: If neither the cache nor a declarative config file is available, the node starts with an empty configuration and returns 404 for all requests.
4. Contact control plane: In all cases, the Data Plane node attempts to fetch the latest configuration from the Control Plane. If successful, the configuration is saved to the local cache (dbless.lmdb).

Kong Gateway Control Planes support Data Plane authentication using either a pinned certificate or a certificate signed by a certificate authority (CA).

• Pinned certificates: Data Planes authenticate using a shared certificate. Both the Control Plane and Data Plane nodes are provisioned with the same certificate. The Control Plane verifies that the Data Plane connected using the pinned certificate.

• Public Key Infrastructure (PKI) certificates: Data Planes authenticate using certificates signed by a CA. The Control Plane must be provisioned with the CA certificate to verify the trust chain. If intermediate certificates are involved, Data Plane nodes must include them when connecting to the Control Plane.

Certificate chain

You must upload enough of the certificate chain in the Control Plane so that the Control Plane can trust the certificate in the Data Plane request and authenticate.

Consider the following scenarios with this example cert chain:

• Upload only cert1 to the Control Plane: This is the pinned certificate. You can include just cert1 in your Data Plane request and not include the chain. The Control Plane doesn’t need to evaluate the issuer because it trusts the cert itself.
• Upload only cert2 to the Control Plane: This would mean any cert coming in that has (issuer: intermediary) would be trusted. You can include just cert1 in your Data Plane request. The Control Plane would trust any certificate issued by the intermediary public key.
• Upload only cert3 to the Control Plane: This is the typical PKI case. It means any cert signed by the root is trusted. However, since cert1 is signed by an intermediary and cert2 is signed by root, you need to include both cert1 and cert2 in your Data Plane request. The Control Plane would trust the whole chain because cert2 is issued by cert3 and cert1 is issued by cert2.

Konnect provides several options to generate or add a certificate for your Data Plane nodes.

Generate a certificate key pair

When you use the Konnect wizard to create a Data Plane node, it generates a certificate key pair. Data Planes can establish a connection with this certificate key pair (pinned cert).

1. Navigate to Gateway Manager in Konnect.
2. Click on the Control Plane you want to create a Data Plane node for.
3. Click Data Plane Nodes in the sidebar.
4. Click Create a New Data Plane Node.
5. Follow the instructions in the wizard to create a Data Plane node and generate the certificate key pair.

Generate a CA-signed certificate

Using the Konnect UI, you can generate a CA certificate, which allows Data Planes to connect using a certificate signed by that CA (PKI). Alternatively you can upload your own CA using the upload option.

1. Navigate to Gateway Manager in Konnect.
2. Click on the Control Plane you want to create a Data Plane node for.
3. From the Action menu, select Data Plane Certificates.
4. Either upload or generate a certificate.

Renew a certificate in Konnect

Certificates generated by Konnect are valid for 10 years. If you bring your own certificates, make sure to review the expiration date and associated metadata.

If you originally created your Data Plane node container using one of the Docker options in Gateway Manager, we recommend creating a new Data Plane node with renewed certificates.

1. Stop the Data Plane node container.
2. Open Gateway Manager, select a Control Plane, open Data Plane Nodes from the side menu, and click New Data Plane Node.
3. Run the script to create a new Data Plane node with updated certificates.
4. Remove the old Data Plane node container.

If your Data Plane nodes are running on Linux or Kubernetes, or if you have a Docker container that was not created using the quick setup script, you must generate new certificates and replace them on the existing nodes. For more information, see the Data Plane reference.

In hybrid mode, a mutual TLS handshake (mTLS) is used for authentication so the actual private key is never transferred on the network, and communication between Control Plane and Data Plane nodes is secure.

Before using hybrid mode, you need a certificate/key pair. Kong Gateway provides two modes for handling certificate/key pairs:

• Shared mode: (Default) Use the Kong Gateway CLI to generate a certificate/key pair, then distribute copies across nodes. The certificate/key pair is shared by both Control Plane and Data Plane nodes.
• PKI mode: Provide certificates signed by a central certificate authority (CA). Kong Gateway validates both sides by checking if they are from the same CA. This eliminates the risks associated with transporting private keys.

Warning: If you have a TLS-aware proxy between the Data Plane and Control Plane nodes, you must use PKI mode and set cluster_server_name to the Control Plane hostname in kong.conf. Do not use shared mode, as it uses a non-standard value for TLS server name indication, and this will confuse TLS-aware proxies that rely on SNI to route traffic.

Shared mode

Warning: Protect the private key. Ensure the private key file can only be accessed by Kong Gateway nodes that belong to the cluster. If the key is compromised, you must regenerate and replace certificates and keys on all Control Plane and Data Plane nodes.

1. On an existing Kong Gateway instance, create a certificate/key pair:

    kong hybrid gen_cert
   

   This will generate cluster.crt and cluster.key files and save them to the current directory. By default, the certificate/key pair is valid for three years, but can be adjusted with the --days option. See kong hybrid --help for more usage information.

2. Copy the cluster.crt and cluster.key files to the same directory on all Kong Gateway Control Plane and Data Plane nodes; e.g., /cluster/cluster. Set appropriate permissions on the key file so it can only be read by Kong Gateway.

PKI mode

With PKI mode, the Hybrid cluster can use certificates signed by a central certificate authority (CA).

In this mode, the Control Plane and Data Plane don’t need to use the same cluster_cert and cluster_cert_key. Instead, Kong Gateway validates both sides by checking if they are from the same CA. Certificates on the Control Plane and Data Plane must contain the TLS Web Server Authentication and TLS Web Client Authentication as X509v3 Extended Key Usage extension, respectively.

Kong Gateway doesn’t validate the CommonName (CN) in the Data Plane certificate, it can take an arbitrary value.

If your Control Plane and Data Planes are separated by a firewall that routes external communications through a proxy, you can configure Kong Gateway to authenticate with the proxy server and allow traffic to pass through.

To use a forward proxy for Control Plane and Data Plane communication, you need to configure the following parameters in kong.conf:

proxy_server = 'http://<username>:<password>@<proxy-host>:<proxy-port>'
proxy_server_ssl_verify = off
cluster_use_proxy = on

proxy_server = 'https://<username>:<password>@<proxy-host>:<proxy-port>'
proxy_server_ssl_verify = on
cluster_use_proxy = on
lua_ssl_trusted_certificate = system  # or <certificate> or <path-to-cert>

Reload Kong Gateway for the connection to take effect.

The following table explains what each forward proxy parameter does: