Traditional mode

In a traditional deployment, you run Kong Gateway nodes in clusters. A Kong Gateway cluster allows you to scale the system horizontally by adding more machines to handle more incoming requests. They will all share the same configuration since they point to the same database. Kong Gateway nodes pointing to the same datastore will be part of the same Kong Gateway cluster.

You need a load balancer in front of your Kong Gateway cluster to distribute traffic across your available nodes.

 
flowchart TD

A[(Database)]
B( Kong Gateway instance)
C( Kong Gateway instance)
D( Kong Gateway instance)

A <---> B & C & D

  

Figure 1: In a traditional deployment, all Kong Gateway nodes connect to the database. Each node manages its own configuration.

Data Plane nodes in the same Kong Gateway cluster share the same configuration.

To load balance traffic across Data Plane nodes in a cluster, configure a load balancer in front of your nodes to distribute the traffic.

For performance reasons, Kong Gateway avoids database connections when proxying requests, and caches the contents of your database in memory. The cached entities include Gateway Services, Routes, Consumers, plugins, credentials, and so on. Since those values are stored in memory, any change made via the Admin API of one of the nodes must be propagated to the other nodes.

A single Kong Gateway node connected to a supported database creates a Kong Gateway cluster of one node. Any changes applied via the Admin API of this node will instantly take effect.

For example, consider a single Kong Gateway node A. If we delete a previously registered Service, then any subsequent request to A would instantly return 404 Not Found, as the node purged it from its local cache.

In a cluster of multiple Kong Gateway nodes, other nodes connected to the same database wouldn’t be instantly notified that the Service was deleted by node A. While the Service is not in the database anymore (it was deleted by node A), it is still in node B’s memory.

All nodes perform a periodic background job to synchronize with changes that may have been triggered by other nodes. The frequency of this job can be configured using the db_update_frequency parameter in kong.conf.

Every db_update_frequency seconds, all running Kong Gateway nodes will poll the database for any update, and will purge the relevant entities from their cache if necessary.

If we delete a Service from node A, this change will not be effective in node B until node B’s next database poll, which will occur up to db_update_frequency seconds later (though it could happen sooner).

This makes Kong Gateway clusters eventually consistent.

When using PostgreSQL as the backend storage, you can optionally enable Kong Gateway to serve read queries from a separate database instance.

Enabling the read-only connection support in Kong Gateway greatly reduces the load on the main database instance since read-only queries are no longer sent to it.

To learn more about how to configure this feature, refer to the Datastore section of the Kong Gateway configuration reference.

For performance reasons, Kong Gateway avoids database connections when proxying requests, and caches the contents of your database in memory. All Gateway entities are cached.

Since these values are stored in memory, any change made via the Admin API of one of the nodes must be propagated to the other nodes.

Additionally, Kong Gateway also caches database misses. This means that if you configure a Service with no plugin, Kong Gateway will cache this information.

All CRUD operations trigger cache invalidations. Creation (POST, PUT) will invalidate cached database misses, and update/deletion (PATCH, DELETE) will invalidate cached database hits.

Using parameters in kong.conf, you can configure where Kong Gateway nodes stand on the performance versus consistency trade-off.

Kong Gateway comes with default values tuned for consistency so that you can experiment with its clustering capabilities while avoiding surprises. As you prepare a production setup, you should consider tuning those values to ensure that your performance constraints are respected.

If you want to investigate the cached values, or manually invalidate a value cached by Kong Gateway (a cached hit or miss), you can do so via the Admin API /cache endpoint.

• Inspect a cached value: /cache/{key}
• Purge a cached value: /cache/{cache_key}

• Purge a node’s cache: /cache

  Note: Be wary of using this endpoint on a node running in production with warm cache. If the node is receiving a lot of traffic, purging its cache at the same time will trigger many requests to your database, and could cause a cache stampede.