AI Semantic Response Guard

The AI Semantic Response Guard plugin extends the AI Prompt Guard plugin by filtering LLM responses based on semantic similarity to predefined rules. It helps prevent unwanted or unsafe responses when serving llm/v1/chat, llm/v1/completions, or llm/v1/embeddings requests through AI Gateway.

You can use a combination of allow and deny response rules to maintain integrity and compliance when returning responses from an LLM service.

The plugin analyzes the semantic content of the full LLM response before it is returned to the client. The matching behavior is as follows:

• If any deny_responses are set and the response matches a pattern in the deny list, the response is blocked with a 403 Forbidden.
• If any allow_responses are set, but the response matches none of the allowed patterns, the response is also blocked with a 403 Forbidden.
• If any allow_responses are set and the response matches one of the allowed patterns, the response is permitted.
• If both deny_responses and allow_responses are set, the deny condition takes precedence. A response that matches a deny pattern will be blocked, even if it also matches an allow pattern. If the response does not match any deny pattern, it must still match an allow pattern to be permitted.

To enforce these rules, the plugin:

1. Disables streaming (stream=false) to ensure the full response body is buffered before analysis.
2. Intercepts the response body using the guard-response filter.
3. Extracts response text, supporting JSON parsing of multiple LLM formats and gzipped content.
4. Generates embeddings for the extracted text.
5. Searches the vector database (Redis, Pgvector, or other) against configured allow_responses or deny_responses.
6. Applies the decision rules described above.

If a response is blocked or if a system error occurs during evaluation, the plugin returns a 403 Forbidden to the client without exposing that the Semantic Response Guard blocked it.

This plugin supports vectordb and embeddings Partials, which let you define shared vector database and embeddings configuration once and reuse it across multiple AI Gateway plugins. This is useful when running this plugin alongside others that use the same vector database and embeddings model, such as AI Semantic Cache, AI RAG Injector, AI Semantic Prompt Guard, AI Semantic Response Guard, and AI Proxy Advanced.

For setup instructions, see AI plugin Partials.

A vector database can be used to store vector embeddings, or numerical representations, of data items. For example, a response would be converted to a numerical representation and stored in the vector database so that it can compare new requests against the stored vectors to find relevant cached items.

The AI Semantic Response Guard plugin supports the following vector databases:

• Using config.vectordb.strategy: redis and parameters in config.vectordb.redis:
  • Redis with Redis Vector Search
  • Redis Cloud
  • Valkey v3.14+: When you configure vectordb.strategy: redis, Kong Gateway queries the server and checks the server name field. If it detects Valkey request, it automatically uses the Valkey-specific driver.
  • Managed Redis with cloud authentication:
    • AWS ElastiCache (auth_provider: aws)
    • Azure Managed Redis (auth_provider: azure)
    • Google Cloud Memorystore (auth_provider: gcp)

    For configuration details, see Using cloud authentication with Redis.

• Using config.vectordb.strategy: pgvector and parameters in config.vectordb.pgvector:
  • PostgreSQL with pgvector v3.10+

To learn more about vector databases in AI Gateway, see Embedding-based similarity matching in Kong AI gateway plugins.

Redis version and module requirements

Kong’s semantic features use the Redis Query Engine (RediSearch) to create and search vector indexes, and the Redis JSON data type to store indexed documents. Your Redis or Valkey deployment must support both, or vector index creation fails.

• Self-hosted Redis: Use Redis Open Source 8.0 or later, which bundles the Redis Query Engine and JSON data type by default, or a distribution that adds them as modules, such as Redis Stack, Redis Enterprise, or Redis Cloud.
• Self-hosted Valkey: Use Valkey 8.1 or later with the valkey-search and valkey-json modules enabled. A vanilla Valkey build without these modules can’t be used for semantic features.
• AWS ElastiCache: ElastiCache doesn’t support custom Redis modules, so ElastiCache for Redis OSS can’t be used for semantic features at any version. Use ElastiCache for Valkey 8.2 or later, which has vector search built in natively.
• Azure Managed Redis: Enable the RediSearch and RedisJSON modules when you create the instance. Modules can only be selected at creation time and can’t be added afterward.
• Google Cloud Memorystore: Use Memorystore for Redis Cluster 8.0 or later or Memorystore for Valkey 8.0 or later, both of which include the required modules automatically. Standard (non-cluster) Memorystore for Redis instances, including the Basic tier, are frozen at Redis 7.2 and don’t support RediSearch or the JSON data type, so they can’t be used for semantic features.

Using cloud authentication with Redis v3.13+

If your plugin uses a Redis datastore, you can authenticate to it with a cloud Redis provider, or with an OAuth 2.0 token endpoint. This allows you to seamlessly rotate credentials without relying on static passwords.

The following providers are supported:

• AWS ElastiCache
• Azure Managed Redis
• Google Cloud Memorystore (with or without Valkey)
• OAuth 2.0 v3.16+, using the client_credentials or password grant type

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      username: $INSTANCE_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: aws
        aws_cache_name: $AWS_CACHE_NAME
        aws_is_serverless: false
        aws_region: $AWS_REGION
        aws_access_key_id: $AWS_ACCESS_KEY_ID
        aws_secret_access_key: $AWS_ACCESS_SECRET_KEY

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379
      username: $CLUSTER_USERNAME
      port: 6379
      cloud_authentication:
        auth_provider: aws
        aws_cache_name: $AWS_CACHE_NAME
        aws_is_serverless: false
        aws_region: $AWS_REGION
        aws_access_key_id: $AWS_ACCESS_KEY_ID
        aws_secret_access_key: $AWS_ACCESS_SECRET_KEY

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      username: $INSTANCE_USERNAME
      port: 10000
      cloud_authentication:
        auth_provider: azure
        azure_client_id: $AZURE_CLIENT_ID
        azure_client_secret: $AZURE_CLIENT_SECRET
        azure_tenant_id: $AZURE_TENANT_ID

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 10000
      username: $CLUSTER_USERNAME
      port: 10000
      cloud_authentication:
        auth_provider: azure
        azure_client_id: $AZURE_CLIENT_ID
        azure_client_secret: $AZURE_CLIENT_SECRET
        azure_tenant_id: $AZURE_TENANT_ID

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      port: 6379
      cloud_authentication:
        auth_provider: gcp
        gcp_service_account_json: $GCP_SERVICE_ACCOUNT

config:
  vectordb:
    strategy: redis
    redis:
      cluster_nodes:
      - ip: $CLUSTER_ADDRESS
        port: 6379 
      port: 6379
      cloud_authentication:
        auth_provider: gcp
        gcp_service_account_json: $GCP_SERVICE_ACCOUNT

config:
  vectordb:
    strategy: redis
    redis:
      host: $INSTANCE_ADDRESS
      port: 6379
      cloud_authentication:
        auth_provider: oauth
        oauth:
          token_endpoint: $OAUTH_TOKEN_ENDPOINT
          grant_type: client_credentials
          client_id: $OAUTH_CLIENT_ID
          client_secret: $OAUTH_CLIENT_SECRET