AI Gateway Enterprise: This plugin is only available as part of our AI Gateway Enterprise offering.
The AI Semantic Prompt Guard plugin extends the AI Prompt Guard plugin by allowing you to permit or block prompts based on a list of similar prompts, helping to prevent misuse of llm/v1/chat or llm/v1/completions requests.
You can use a combination of allow and deny rules to maintain integrity and compliance when serving an LLM service using Kong AI Gateway.
The matching behavior is as follows:
deny prompts are set and the request matches a prompt in the deny list, the caller receives a 400 response.allow prompts are set, but the request matches none of the allowed prompts, the caller also receives a 400 response.allow prompts are set and the request matches one of the allow prompts, the request passes through to the LLM.deny and allow prompts set, the deny condition takes precedence over allow. Any request that matches a prompt in the deny list will return a 400 response, even if it also matches a prompt in the allow list. If the request doesn’t match a prompt in the deny list, then it must match a prompt in the allow list to be passed through to the LLM.A vector database can be used to store vector embeddings, or numerical representations, of data items. For example, a response would be converted to a numerical representation and stored in the vector database so that it can compare new requests against the stored vectors to find relevant cached items.
The AI Semantic Prompt Guard plugin supports the following vector databases:
config.vectordb.strategy: redis and parameters in config.vectordb.redis:
config.vectordb.strategy: pgvector and parameters in config.vectordb.pgvector:
To learn more about vector databases in AI Gateway, see Embedding-based similarity matching in Kong AI gateway plugins.
Does the AI Semantic Prompt Guard plugin support multilingual input?
Yes, the AI Semantic Prompt Guard plugin supports multilingual input—depending on the capabilities of the configured embedding model. The plugin sends raw UTF-8 text to the embedding provider supported by AI Gateway (such as Azure, Bedrock, Gemini, Hugging Face, Mistral, or OpenAI). As long as the model supports multiple languages, semantic comparisons and rule enforcement will work as expected without requiring additional plugin configuration.
How do I resolve the MemoryDB error Number of indexes exceeds the limit?
If you see the following error in the logs:
failed to create memorydb instance failed to create index: LIMIT Number of indexes (11) exceeds the limit (10)
This means that the hardcoded MemoryDB instance limit has been reached. To resolve this, create more MemoryDB instances to handle multiple AI Semantic Prompt Guard plugin instances.