AI Semantic Response Guard

AI License Required

Overview Examples Guides Configuration reference Changelog

How it works

The plugin analyzes the semantic content of the full LLM response before it is returned to the client. The matching behavior is as follows:

If any deny_responses are set and the response matches a pattern in the deny list, the response is blocked with a 400 Bad response.
If any allow_responses are set, but the response matches none of the allowed patterns, the response is also blocked with a 400 Bad response.
If any allow_responses are set and the response matches one of the allowed patterns, the response is permitted.
If both deny_responses and allow_responses are set, the deny condition takes precedence. A response that matches a deny pattern will be blocked, even if it also matches an allow pattern. If the response does not match any deny pattern, it must still match an allow pattern to be permitted.

Response processing

To enforce these rules, the plugin:

Disables streaming (stream=false) to ensure the full response body is buffered before analysis.
Intercepts the response body using the guard-response filter.
Extracts response text, supporting JSON parsing of multiple LLM formats and gzipped content.
Generates embeddings for the extracted text.
Searches the vector database (Redis, Pgvector, or other) against configured allow_responses or deny_responses.
Applies the decision rules described above.

If a response is blocked or if a system error occurs during evaluation, the plugin returns a 400 Bad Request to the client without exposing that the Semantic Response Guard blocked it.

A vector database can be used to store vector embeddings, or numerical representations, of data items. For example, a response would be converted to a numerical representation and stored in the vector database so that it can compare new requests against the stored vectors to find relevant cached items.

The AI Semantic Response Guard plugin supports the following vector databases:

Using config.vectordb.strategy: redis and parameters in config.vectordb.redis:
- Redis with Vector Similarity Search (VSS)
- AWS MemoryDB for Redis v3.12+
Using config.vectordb.strategy: pgvector and parameters in config.vectordb.pgvector:
- PostgreSQL with pgvector v3.10+

To learn more about vector databases in AI Gateway, see Embedding-based similarity matching in Kong AI gateway plugins.

Using cloud authentication with Redis v3.13+

If your plugin uses a Redis datastore, you can authenticate to it with a cloud Redis provider. This allows you to seamlessly rotate credentials without relying on static passwords.

The following providers are supported:

AWS ElastiCache
Azure Managed Redis
Google Cloud Memorystore (with or without Valkey)

You need:

A running Redis instance on an AWS ElastiCache instance for Valkey 7.2 or later or ElastiCache for Redis OSS version 7.0 or later
The ElastiCache user needs to set “Authentication mode” to “IAM”

The following policy assigned to the IAM user/IAM role that is used to connect to the ElastiCache:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "elasticache:Connect"
            ],
            "Resource": [
                "arn:aws:elasticache:ARN_OF_THE_ELASTICACHE",
                "arn:aws:elasticache:ARN_OF_THE_ELASTICACHE_USER"
            ]
        }
    ]
}

Copied!

To configure cloud authentication with Redis, add the following parameters to your plugin configuration:

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      username: $INSTANCE_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: aws
        aws_cache_name: $AWS_CACHE_NAME
        aws_is_serverless: false
        aws_region: $AWS_REGION
        aws_access_key_id: $AWS_ACCESS_KEY_ID
        aws_secret_access_key: $AWS_ACCESS_SECRET_KEY

Copied!

Replace the following with your actual values:

$INSTANCE_ADDRESS: The ElastiCache instance address.
$INSTANCE_USERNAME: The ElastiCache username with IAM Auth mode configured.
$AWS_CACHE_NAME: Name of your AWS ElastiCache instance.
$AWS_REGION: Your AWS ElastiCache instance region.
$AWS_ACCESS_KEY_ID: (Optional) Your AWS access key ID.
$AWS_ACCESS_SECRET_KEY: (Optional) Your AWS secret access key.

You need:

A running Redis instance on an AWS ElastiCache cluster for Valkey 7.2 or later or ElastiCache for Redis OSS version 7.0 or later
The ElastiCache user needs to set “Authentication mode” to “IAM”

The following policy assigned to the IAM user/IAM role that is used to connect to the ElastiCache:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "elasticache:Connect"
            ],
            "Resource": [
                "arn:aws:elasticache:ARN_OF_THE_ELASTICACHE",
                "arn:aws:elasticache:ARN_OF_THE_ELASTICACHE_USER"
            ]
        }
    ]
}

Copied!

To configure cloud authentication with Redis, add the following parameters to your plugin configuration:

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379
      username: $CLUSTER_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: aws
        aws_cache_name: $AWS_CACHE_NAME
        aws_is_serverless: false
        aws_region: $AWS_REGION 
        aws_access_key_id: $AWS_ACCESS_KEY_ID
        aws_secret_access_key: $AWS_ACCESS_SECRET_KEY

Copied!

Replace the following with your actual values:

$CLUSTER_ADDRESS: The ElastiCache cluster address.
$CLUSTER_USERNAME: The ElastiCache username with IAM Auth mode configured.
$AWS_CACHE_NAME: Name of your AWS ElastiCache cluster.
$AWS_REGION: Your AWS ElastiCache cluster region.
$AWS_ACCESS_KEY_ID: (Optional) Your AWS access key ID.
$AWS_ACCESS_SECRET_KEY: (Optional) Your AWS secret access key.

You need:

A running Redis instance on an Azure Managed Redis instance with Entra authentication configured
Add the user/service principal/identity to the “Microsoft Entra Authentication Redis user” list for the Azure Managed Redis instance

To configure cloud authentication with Redis, add the following parameters to your plugin configuration:

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      username: $INSTANCE_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: azure
        azure_client_id: $AZURE_CLIENT_ID
        azure_client_secret: $AZURE_CLIENT_SECRET
        azure_tenant_id: $AZURE_TENANT_ID

Copied!

Replace the following with your actual values:

$INSTANCE_ADDRESS: The Azure Managed Redis instance address.
$INSTANCE_USERNAME: The object (principal) ID of the Principal/Identity with essential access.
$AZURE_CLIENT_ID: The client ID of the Principal/Identity.
$AZURE_CLIENT_SECRET: (Optional) The client secret of the Principal/Identity.
$AZURE_TENANT_ID: (Optional) The tenant ID of the Principal/Identity.

You need:

A running Redis instance on an Azure Managed Redis cluster with Entra authentication configured
Add the user/service principal/identity to the “Microsoft Entra Authentication Redis user” list for the Azure Managed Redis instance

To configure cloud authentication with Redis, add the following parameters to your plugin configuration:

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379
      username: $CLUSTER_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: azure
        azure_client_id: $AZURE_CLIENT_ID
        azure_client_secret: $AZURE_CLIENT_SECRET
        azure_tenant_id: $AZURE_TENANT_ID

Copied!

Replace the following with your actual values:

$CLUSTER_ADDRESS: The Azure Managed Redis cluster address.
$CLUSTER_USERNAME: The object (principal) ID of the Principal/Identity with essential access.
$AZURE_CLIENT_ID: The client ID of the Principal/Identity.
$AZURE_CLIENT_SECRET: (Optional) The client secret of the Principal/Identity.
$AZURE_TENANT_ID: (Optional) The tenant ID of the Principal/Identity.

You need:

A running Redis instance on an Google Cloud Memorystore instance
Assign the principal to the corresponding role:
- Cloud Memorystore Redis DB Connection User(roles/redis.dbConnectionUser) for Memorystore for Redis Cluster
- Memorystore DB Connector User (roles/memorystore.dbConnectionUser) for Memorystore for Valkey

To configure cloud authentication with Redis, add the following parameters to your plugin configuration:

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      port: 6379
      cloud_authentication:
        auth_provider: gcp
        gcp_service_account_json: $GCP_SERVICE_ACCOUNT

Copied!

Replace the following with your actual values:

$INSTANCE_ADDRESS: The Memorystore instance address.
$GCP_SERVICE_ACCOUNT: (Optional) The GCP service account JSON.

You need:

A running Redis instance on an Google Cloud Memorystore cluster
Assign the principal to the corresponding role:
- Cloud Memorystore Redis DB Connection User(roles/redis.dbConnectionUser) for Memorystore for Redis Cluster
- Memorystore DB Connector User (roles/memorystore.dbConnectionUser) for Memorystore for Valkey

To configure cloud authentication with Redis, add the following parameters to your plugin configuration:

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379 
      port: 6379
      cloud_authentication:
        auth_provider: gcp
        gcp_service_account_json: $GCP_SERVICE_ACCOUNT

Copied!

Replace the following with your actual values:

$CLUSTER_ADDRESS: The Memorystore cluster address.
$GCP_SERVICE_ACCOUNT: The GCP service account JSON.

Next Steps

Use AI Semantic Prompt Guard plugin to govern your LLM traffic

Use AI Prompt Guard plugin to govern your LLM traffic

AI Semantic Response Guard

How it works

Response processing

Vector databases

Using cloud authentication with Redis v3.13+

Next Steps

Help us make these docs great!

Still need help?