AI Rate Limiting Advanced

Release date 2026/04/07

Deprecation

• Removed the parse-request shared filter and deprecated the llm_format configuration field.

Feature

• Added a policy-based rate limiting mode with multi-dimensional match conditions and request counting strategy.

• Added prompt token estimation for prompt_tokens, total_tokens, and cost strategies in policies mode, enabling early request rejection before the upstream responds.

Bugfix

• Fixed an issue where the plugin did not validate whether the configured dictionary_name exists before use, which could lead to runtime errors.

• Extended deferred rate limiting callback to support model-partitioned policies and cost strategy.

Release date 2025/12/18

Feature

• added support to count cost for routes with dynamic AI models.

Bugfix

• Fixed an issue where the plugin decreased requests by whole numbers when using Redis. This is an opt-in fix and can be enabled by setting decrease_by_fractions_in_redis to true in the plugin configuration.

Release date 2025/11/18

Bugfix

• Fixed an issue where the plugin decreased requests by whole numbers when using Redis. This is an opt-in fix and can be enabled by setting decrease_by_fractions_in_redis to true in the plugin configuration.

Release date 2025/10/23

Bugfix

• Fixed an issue where the plugin decreased requests by whole numbers when using Redis. This is an opt-in fix and can be enabled by setting decrease_by_fractions_in_redis to true in the plugin configuration.

Release date 2025/07/16

Bugfix

• Fixed an issue where the llm license migration could fail if the license counter contained more than one week of data.

Release date 2025/07/03

Feature

• Added support for the namespace field in the AI Rate Limiting Advanced plugin. This allows users to specify a ratelimiting namespace for the plugin, Similar to the existing namespace field in the rate-limiting-advanced plugin.

Bugfix

• Fixed an issue where some of ai metrics was missed in analytics

• Fixed an issue where AI rate limiting advanced plugin might panic when use redis strategy and sync_rate is set to 0.

• Fixed an issue where the rate limiting plugins can’t handle decimal numbers when using Redis strategy.

Known Issues

• If any AI Gateway plugin has been enabled in a self-managed Kong Gateway deployment for more than a week, upgrades from 3.10 versions to 3.11.0.0 will fail due to a license migration issue. This does not affect Konnect deployments.

  A fix will be provided in 3.11.0.1.

  See breaking changes in 3.11 for a temporary workaround.

Release date 2025/12/09

Bugfix

• Fixed an issue where the plugin decreased requests by whole numbers when using Redis. This is an opt-in fix and can be enabled by setting decrease_by_fractions_in_redis to true in the plugin configuration.

Release date 2025/08/07

Bugfix

• Fixed an issue where AI rate limiting advanced plugin might panic when use redis strategy and sync_rate is set to 0.

Release date 2025/03/27

Breaking Change

• window_size and limit now require an array of numbers instead of a single number. If you configured the plugin before 3.10 and use kong migrations to upgrade to 3.10, it will be automatically migrated to use the array.

Feature

• Added support for allowing multiple rate limits for the same providers.

Release date 2024/12/12

Feature

• Added support for Huggingface provider to the AI Rate Limiting Advanced plugin.

Bugfix

• Updated the error message for rate limit exceeded to include AI-related information.

• Fixed an issue where the plugin yielded an error when incrementing the rate limit counters in non-yieldable phases.

• Fixed an issue where the plugin may fail to authenticate to Redis correctly with vault-referenced redis configuration.

Release date 2024/11/04

Bugfix

• Fixed an issue where the plugin yielded an error when incrementing the rate limit counters in non-yieldable phases.

Release date 2024/09/11

Deprecation

• Switched to sentinel_nodes and cluster_nodes for redis configuration.

• Deprecated timeout config field in redis config in favor of connect_/send_/read_timeout (timeout field will be removed in 4.0).

Feature

• allow AI plugin to read request from buffered file

• Add the cost strategy to AI rate Limiting plugin.

• Added the bedrock and gemini providers to the providers list in the ai-rate-limiting-advanced plugin.

• Add the stats when reaching limit and exiting AI rate Limiting plugin.

• Added Redis cluster_max_redirections configuration option.

Bugfix

• Edit the logic for the window ajustement and fix missing passing window to shm

Release date 2024/06/18

Bugfix

• Edit the logic for the window ajustement and fix missing passing window to shm

Release date 2024/05/28

Feature

• Introduced the new AI Rate Limiting Advanced plugin that allow to implement a rate limit by AI provider.