AI Semantic Prompt Guard

AI License Required

Overview Examples Configuration reference Changelog

How it works

The matching behavior is as follows:

If any deny prompts are set and the request matches a prompt in the deny list, the caller receives a 400 response.
If any allow prompts are set, but the request matches none of the allowed prompts, the caller also receives a 400 response.
If any allow prompts are set and the request matches one of the allow prompts, the request passes through to the LLM.
If there are both deny and allow prompts set, the deny condition takes precedence over allow. Any request that matches a prompt in the deny list will return a 400 response, even if it also matches a prompt in the allow list. If the request doesn’t match a prompt in the deny list, then it must match a prompt in the allow list to be passed through to the LLM.

A vector database can be used to store vector embeddings, or numerical representations, of data items. For example, a response would be converted to a numerical representation and stored in the vector database so that it can compare new requests against the stored vectors to find relevant cached items.

The AI Semantic Prompt Guard plugin supports the following vector databases:

Using config.vectordb.strategy: redis and parameters in config.vectordb.redis:
- Redis with Vector Similarity Search (VSS)
- AWS MemoryDB for Redis v3.12+
Using config.vectordb.strategy: pgvector and parameters in config.vectordb.pgvector:
- PostgreSQL with pgvector v3.10+

To learn more about vector databases in AI Gateway, see Embedding-based similarity matching in Kong AI gateway plugins.

Using cloud authentication with Redis v3.13+

Starting in Kong Gateway 3.13, you can authenticate with a cloud Redis provider for your Redis strategy. This allows you to seamlessly rotate credentials without relying on static passwords.

The following providers are supported:

AWS ElastiCache
Azure Managed Redis
Google Cloud Memorystore (with or without Valkey)

Each provider also supports an instance and cluster configuration.

Important: Kong Gateway open source plugins do not support any Redis cloud provider cluster configurations.

To configure cloud authentication with Redis, add the following parameters to your plugin configuration:

You need:

A running Redis instance on an AWS ElastiCache instance for Valkey 7.2 or later or ElastiCache for Redis OSS version 7.0 or later
The ElastiCache user needs to set “Authentication mode” to “IAM”

The following policy assigned to the IAM user/IAM role that is used to connect to the ElastiCache:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "elasticache:Connect"
            ],
            "Resource": [
                "arn:aws:elasticache:ARN_OF_THE_ELASTICACHE",
                "arn:aws:elasticache:ARN_OF_THE_ELASTICACHE_USER"
            ]
        }
    ]
}

Copied!

config:
  storage: redis
  storage_config:
    redis:
      host: $INSTANCE_ADDRESS
      username: $INSTANCE_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: aws
        aws_cache_name: $AWS_CACHE_NAME
        aws_is_serverless: false
        aws_region: $AWS_REGION
        aws_access_key_id: $AWS_ACCESS_KEY_ID
        aws_secret_access_key: $AWS_ACCESS_SECRET_KEY

Copied!

Replace the following with your actual values:

$INSTANCE_ADDRESS: The ElastiCache instance address.
$INSTANCE_USERNAME: The ElastiCache username with IAM Auth mode configured.
$AWS_CACHE_NAME: Name of your AWS ElastiCache instance.
$AWS_REGION: Your AWS ElastiCache instance region.
$AWS_ACCESS_KEY_ID: (Optional) Your AWS access key ID.
$AWS_ACCESS_SECRET_KEY: (Optional) Your AWS secret access key.

You need:

A running Redis instance on an AWS ElastiCache cluster for Valkey 7.2 or later or ElastiCache for Redis OSS version 7.0 or later
The ElastiCache user needs to set “Authentication mode” to “IAM”

The following policy assigned to the IAM user/IAM role that is used to connect to the ElastiCache:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "elasticache:Connect"
            ],
            "Resource": [
                "arn:aws:elasticache:ARN_OF_THE_ELASTICACHE",
                "arn:aws:elasticache:ARN_OF_THE_ELASTICACHE_USER"
            ]
        }
    ]
}

Copied!

config:
  storage: redis
  storage_config:
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379
      username: $CLUSTER_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: aws
        aws_cache_name: $AWS_CACHE_NAME
        aws_is_serverless: false
        aws_region: $AWS_REGION 
        aws_access_key_id: $AWS_ACCESS_KEY_ID
        aws_secret_access_key: $AWS_ACCESS_SECRET_KEY

Copied!

Replace the following with your actual values:

$CLUSTER_ADDRESS: The ElastiCache cluster address.
$CLUSTER_USERNAME: The ElastiCache username with IAM Auth mode configured.
$AWS_CACHE_NAME: Name of your AWS ElastiCache cluster.
$AWS_REGION: Your AWS ElastiCache cluster region.
$AWS_ACCESS_KEY_ID: (Optional) Your AWS access key ID.
$AWS_ACCESS_SECRET_KEY: (Optional) Your AWS secret access key.

You need:

A running Redis instance on an Azure Managed Redis instance with Entra authentication configured
Add the user/service principal/identity to the “Microsoft Entra Authentication Redis user” list for the Azure Managed Redis instance

config:
  storage: redis
  storage_config:
    redis:
      host: $INSTANCE_ADDRESS
      username: $INSTANCE_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: azure
        azure_client_id: $AZURE_CLIENT_ID
        azure_client_secret: $AZURE_CLIENT_SECRET
        azure_tenant_id: $AZURE_TENANT_ID

Copied!

Replace the following with your actual values:

$INSTANCE_ADDRESS: The Azure Managed Redis instance address.
$INSTANCE_USERNAME: The object (principal) ID of the Principal/Identity with essential access.
$AZURE_CLIENT_ID: The client ID of the Principal/Identity.
$AZURE_CLIENT_SECRET: (Optional) The client secret of the Principal/Identity.
$AZURE_TENANT_ID: (Optional) The tenant ID of the Principal/Identity.

You need:

A running Redis instance on an Azure Managed Redis cluster with Entra authentication configured
Add the user/service principal/identity to the “Microsoft Entra Authentication Redis user” list for the Azure Managed Redis instance

config:
  storage: redis
  storage_config:
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379
      username: $CLUSTER_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: azure
        azure_client_id: $AZURE_CLIENT_ID
        azure_client_secret: $AZURE_CLIENT_SECRET
        azure_tenant_id: $AZURE_TENANT_ID

Copied!

Replace the following with your actual values:

$CLUSTER_ADDRESS: The Azure Managed Redis cluster address.
$CLUSTER_USERNAME: The object (principal) ID of the Principal/Identity with essential access.
$AZURE_CLIENT_ID: The client ID of the Principal/Identity.
$AZURE_CLIENT_SECRET: (Optional) The client secret of the Principal/Identity.
$AZURE_TENANT_ID: (Optional) The tenant ID of the Principal/Identity.

You need:

A running Redis instance on an Google Cloud Memorystore instance
Assign the principal to the corresponding role:
- Cloud Memorystore Redis DB Connection User(roles/redis.dbConnectionUser) for Memorystore for Redis Cluster
- Memorystore DB Connector User (roles/memorystore.dbConnectionUser) for Memorystore for Valkey

config:
  storage: redis
  storage_config:
    redis:
      host: $INSTANCE_ADDRESS
      port: 6379
      cloud_authentication:
        auth_provider: gcp
        gcp_service_account_json: $GCP_SERVICE_ACCOUNT

Copied!

Replace the following with your actual values:

$INSTANCE_ADDRESS: The Memorystore instance address.
$GCP_SERVICE_ACCOUNT: (Optional) The GCP service account JSON.

You need:

A running Redis instance on an Google Cloud Memorystore cluster
Assign the principal to the corresponding role:
- Cloud Memorystore Redis DB Connection User(roles/redis.dbConnectionUser) for Memorystore for Redis Cluster
- Memorystore DB Connector User (roles/memorystore.dbConnectionUser) for Memorystore for Valkey

config:
  storage: redis
  storage_config:
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379 
      port: 6379
      cloud_authentication:
        auth_provider: gcp
        gcp_service_account_json: $GCP_SERVICE_ACCOUNT

Copied!

Replace the following with your actual values:

$CLUSTER_ADDRESS: The Memorystore cluster address.
$GCP_SERVICE_ACCOUNT: The GCP service account JSON.

FAQs

Does the AI Semantic Prompt Guard plugin support multilingual input?

Yes, the AI Semantic Prompt Guard plugin supports multilingual input—depending on the capabilities of the configured embedding model. The plugin sends raw UTF-8 text to the embedding provider supported by AI Gateway (such as Azure, Bedrock, Gemini, Hugging Face, Mistral, or OpenAI). As long as the model supports multiple languages, semantic comparisons and rule enforcement will work as expected without requiring additional plugin configuration.

How do I resolve the MemoryDB error Number of indexes exceeds the limit?

If you see the following error in the logs:

failed to create memorydb instance failed to create index: LIMIT Number of indexes (11) exceeds the limit (10)

Copied!

This means that the hardcoded MemoryDB instance limit has been reached. To resolve this, create more MemoryDB instances to handle multiple AI Semantic Prompt Guard plugin instances.

Next Steps

Use AI Semantic Prompt Guard plugin to govern your LLM traffic

AI Semantic Prompt Guard

How it works

Vector databases

Using cloud authentication with Redis v3.13+

FAQs

Next Steps

Help us make these docs great!

Still need help