Release date 2025/11/18
Fixed an issue where Cohere and Huggingface models did not work with the semantic cache because of the polluted request format.
Fixed an issue where the plugin did not report time to first token (TTFT) metrics when hit.
Release date 2025/10/01
Fixed an issue where the DELETE /ai-semantic-cache endpoint returned 500 errors for successful deletions when using pgvector as the vector database.
Fixed an issue where the DELETE /ai-semantic-cache endpoint returned an incorrect 204 status for non-existent keys.
Fixed an issue where AI inference requests to non-OpenAI providers in the stream mode were not supported.
Fixed an issue where the SSE terminator could have incorrect ending characters.
Release date 2025/10/23
Fixed an issue where the DELETE /ai-semantic-cache returned 500 even deletion was successful, when using pgvector as vectordb.
Fixed an issue where the DELETE /ai-semantic-cache with non-existent key returned incorrect status 204.
Fixed an issue where AI inference requests to non-OpenAI providers in the stream mode were not supported.
Release date 2025/07/28
Fixed an issue where Gemini Vertex AI embeddings failed due to incorrect URL construction and response parsing.
Release date 2025/07/16
Fixed an issue where the llm license migration could fail if the license counter contained more than one week of data.
Fixed an issue where SSE terminator may not have correct ending characters.
Release date 2025/07/03
Fixed an issue where some of ai metrics was missed in analytics
Fixed an issue where SSE terminator may not have correct ending characters.
Blocked plugins to execute retry logic. Also improve testing functions
Fixed an issue where some of ai metrics was missed in analytics
If any AI Gateway plugin has been enabled in a self-managed Kong Gateway deployment for more than a week, upgrades from 3.10 versions to 3.11.0.0 will fail due to a license migration issue. This does not affect Konnect deployments.
A fix will be provided in 3.11.0.1.
See breaking changes in 3.11 for a temporary workaround.
Release date 2025/08/07
Fixed an issue where the Gemini provider could not use Anthropic ‘rawPredict’ endpoint models hosted in Vertex.
Fixed an issue where Gemini Vertex AI embeddings failed due to incorrect URL construction and response parsing.
Fixed an issue where SSE terminator may not have correct ending characters.
Release date 2025/07/06
Fixed an issue where SSE terminator may not have correct ending characters.
Release date 2025/03/27
Added support for boto3 SDKs for Bedrock provider, and for Google GenAI SDKs for Gemini provider.
Added the huggingface, azure, vertex, and bedrock providers to embeddings. They can be used by the ai-proxy-advanced, ai-semantic-cache, ai-semantic-prompt-guard, and ai-rag-injector plugins.
Allow authentication to Bedrock services with assume roles in AWS.
Fixed an issue where AI upstream URL trailing would be empty.
Fixed an issue where the Refresh header wasn’t properly sent to the client.
Fixed issue where the SSE body may have extra trailing.
Release date 2024/12/12
Made the
embeddings.model.name config field a free text entry, enabling use of a
self-hosted (or otherwise compatible) model.
Added ignore_tool configuration option to discard tool role prompts from the input text.
Plugin can now be enabled on Consumer Groups.
Fixed an bug that AI semantic cache can’t use request provided models
Fixed the exact matching to catch everything including embeddings.
Fixed an issue where the ai-semantic-cache plugin would abort in stream mode when another plugin enable the buffering proxy mode.
Fixed an issue where the ai-semantic-cache plugin put the wrong type value in the metrics when using the prometheus plugin.
Fixed an issue where the plugin failed when handling requests with multiple models.
Release date 2024/11/04
Fixed an bug that AI semantic cache can’t use request provided models
Fixed an issue where the ai-semantic-cache plugin would abort in stream mode when another plugin enable the buffering proxy mode.
Fixed an issue where the ai-semantic-cache plugin put the wrong type value in the metrics when using the prometheus plugin.
Release date 2024/09/11
allow AI plugin to read request from buffered file
Introduced AI Semantic Caching plugin, enabling you to configure an embeddings-based caching system for Large Language Model responses.
Fix the ai-semantic-caching plugin with a condition for calculating latencies when no embeddings, add deep copy for the request table and fix countback.