AI Custom Guardrail Policy

The AI Custom Guardrail Policy enforces introspection on both inbound requests and outbound responses handled by an AI Model. The Policy can be attached to an AI Model, AI Consumer, AI Consumer Group, or applied globally (see AI Policy scopes). It can integrate with any HTTP-based guardrail service. This ensures all data exchanged between clients and upstream LLMs adheres to the configured security standards.

The AI Custom Guardrail Policy can be applied to:

• Input data (requests)
• Output data (responses)
• Both input and output data

Here’s how it works if you apply it to both requests and responses:

1. The AI Custom Guardrail Policy intercepts the request and sends the request body to the guardrail service.
   • The guardrail service analyzes the request against configured moderation categories and allows or blocks the request.
2. If allowed, the request is forwarded upstream by the AI Model.
3. On the way back, the Policy intercepts the response and sends the response body to the guardrail service.
   • The guardrail service analyzes the response against configured moderation categories and allows or blocks the response.
4. If allowed, the response is forwarded to the client.

The following diagram shows the request and response flow when the AI Custom Guardrail Policy inspects both directions:

 
sequenceDiagram
    autonumber
    participant Client
    participant Policy as AI Custom Guardrail Policy
    participant Svc as Guardrail service
    participant Model as AI Model
    participant LLM as Upstream LLM provider

    Client->>Policy: Send request
    Policy->>Svc: Send request body
    Svc->>Policy: Return verdict
    alt Request blocked
        Policy->>Client: Return error, request is not sent upstream
    else Request allowed
        Policy->>Model: Forward request
        Model->>LLM: Send request
        LLM->>Model: Return response
        Model->>Policy: Forward response
        Policy->>Svc: Send response body
        Svc->>Policy: Return verdict
        alt Response blocked
            Policy->>Client: Return error
        else Response allowed
            Policy->>Policy: Mask content if configured
            Policy->>Client: Return response
        end
    end
  

Figure 1: Request and response flow with the AI Custom Guardrail Policy.

To configure the AI Custom Guardrail Policy to work with your guardrail service, you must define your guardrail vendor API’s required parameters under config.params. The key is the parameter name, and the value can be a string or a Lua expression.

Additionally, the following built-in variables are available in Lua expressions. They can be used as arguments in functions, but not in the function body:

• $(source): The current phase on which the Policy is running. The value is INPUT if the Policy is currently inspecting the request, and OUTPUT if it’s inspecting the response.
• $(conf): A Lua table that corresponds to the Policy’s config field, meaning it has the same values as the Policy’s configuration, which allows to access sub-fields under config.
• $(content): The text content being inspected, extracted from the request body in the INPUT phase and the response body in the OUTPUT phase.

• $(resp): The response from the guardrail service.

  This variable is a Lua table corresponding to the request body if the AI Custom Guardrail Policy is inspecting the request, but it’s a string when inspecting the response. Make sure to configure your functions accordingly.

Request

The config.request field is used to configure the request that will be sent to your guardrail service. You can set the URL, request body, headers, query parameters, and authentication. You can use the parameters defined under config.params using the following syntax: $(conf.params.<PARAM_KEY>).

Response

The config.response field is used to define how to parse the response received by the guardrail service.

You must define:

• config.response.block
• config.response.block_message

These fields can be defined using functions defined in config.functions, Lua expressions, or strings. For example, to use the value of a field named action in the guardrail service’s response body, you can set config.response.block to $(resp.action).

Metrics

The config.metrics field allows you to define metrics to be logged by Kong Gateway.

The following standard metrics are available:

• block_reason: The reason why the request or response was blocked.
• block_details: Additional details about the blocked request or response.
• masked: Whether content was masked in the request or response.

The values can be set to Lua expressions. You can also use the config.custom_metrics field to define additional metrics.

Set config.proxy on this entity to route its outbound requests through an HTTP forward proxy. Use this in network-isolated deployments where AI Gateway cannot open direct connections to LLM providers or auxiliary services.

The proxy record is identical for AI Model, AI MCP Server, and supported AI Policy entities. Existing capabilities such as load balancing, health checking, streaming, WebSocket, and HTTP/2 continue to work when the proxy is active.

For the full field reference, traffic flow, and limitations, see Forward proxy support.