Kafka Consume

This plugin consumes messages from Apache Kafka topics and makes them available through HTTP endpoints. For more information, see Kafka topics.

Note: This plugin has the following known limitations:

• Message compression is not supported.
• The message format is not customizable.
• Kong Gateway does not support Kafka 4.0.

Kong also provides Kafka plugins for publishing messages:

• Kafka Log
• Kafka Upstream

The plugin supports two modes of operation:

• http-get: Consume messages via an HTTP GET requests (default)
• server-sent-events: Stream messages using server-sent events

When running multiple Data Plane nodes, there is no thread-safe behavior between nodes. In high-load scenarios, you may observe the same message being delivered multiple times across different Data Plane nodes

To minimize duplicate message delivery in a multi-node setup, consider:

• Using a single Data Plane node for consuming messages from specific topics
• Implementing idempotency handling in your consuming application
• Monitoring Consumer Group offsets across your Data Plane nodes