Home / How-to Guides
Edit
Report

Use the AI AWS Guardrails Policy

Uses: AI Gateway kongctl Konnect API
Incompatible with
on-prem
Related Documentation
AI Gateway Documentation
Tags
#ai #openai #aws #safety
Related Resources
Azure AI Content Safety
AI Gateway
Minimum Version
AI Gateway - 2.0
Previous Versions of this page
See AI Gateway v1 version
TL;DR

Configure an AI Model Provider and AI Model to route requests to any LLM upstreams. Apply an AI AWS Guardrails Policy to your model to block unsafe inputs and outputs based on your Bedrock guardrail.

Prerequisites

AI Gateway running

This is a Konnect tutorial and requires a Konnect personal access token.

  1. Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.

  2. Export your token to an environment variable:

    export KONNECT_TOKEN='YOUR_KONNECT_PAT'

  3. Run the AI Gateway quickstart script to automatically provision a control plane and data plane in Kong Konnect, and configure your environment:

    curl -Ls https://get.konghq.com/ai | bash -s -- -k $KONNECT_TOKEN

This sets up a AI Gateway control plane named ai-quickstart, provisions a local data plane, and prints out the following environment variables export:

export AI_GATEWAY_ID=your-gateway-id
export KONNECT_TOKEN=$KONNECT_TOKEN
export KONNECT_CONTROL_PLANE_NAME=ai-quickstart
export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
export KONNECT_PROXY_URL='http://localhost:8000'

Copy and paste these into your terminal to configure your session.

kongctl   v1.16.0+

This tutorial uses kongctl to manage Konnect resources programmatically. We recommend keeping kongctl up to date with the latest version (1.16.0).

  1. Install kongctl from developer.konghq.com/kongctl.

  2. Verify the installation:

    kongctl version

AWS Account

To complete this tutorial, you will need the following credentials

  • AWS_REGION
  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY

You can get the access key ID and secret access key from the AWS IAM Console under Users > Security credentials, and the region from the AWS Console where your resources are deployed. Once you have them, export them as environment variables by running the following command and replacing placeholder values with your secrets:

export AWS_REGION='YOUR_AWS_REGION'
export AWS_ACCESS_KEY_ID='YOUR_AWS_ACCESS_KEY'
export AWS_SECRET_ACCESS_KEY='YOUR_AWS_SECRET_ACCESS_KEY'

Bedrock Guardrail

To complete this tutorial, you must have a Guardrail policy created in your AWS Bedrock account:

  1. Install AWS CLI v2 Follow the official installation guide. After installation, confirm it by running:

    aws --version

  1. Configure AWS credentials Run the following command and provide your IAM user or role credentials:

    aws configure

    You will be prompted to enter:

    • AWS Access Key ID
    • AWS Secret Access Key
    • Default region name (e.g., us-east-1)
    • Default output format (e.g., json)

    Make sure your IAM user or role has Bedrock permissions such as bedrock:CreateGuardrail, bedrock:CreateGuardrailVersion, and others necessary for managing guardrails. For more details, see the AWS CLI configuration documentation.

  2. Test that you can call Bedrock operations by running:

    aws bedrock list-foundation-models

    If this command fails, check your credentials, permissions, and configured region.

  3. Create a guardrail.json configuration file: This configuration defines an Amazon Bedrock guardrail named example-guardrail that blocks harmful or restricted content—including specific words, topics like quantum computing, and categories such as violence, hate, and prompt attacks—in both input and output messages.

    cat <<'EOF' > guardrail.json
{
  "name": "example-guardrail",
  "description": "My first Bedrock guardrail via CLI",
  "blockedInputMessaging": "Input blocked due to policy violation.",
  "blockedOutputsMessaging": "Output blocked due to policy violation.",
  "wordPolicyConfig": {
    "wordsConfig": [
      {
        "inputAction": "BLOCK",
        "inputEnabled": true,
        "outputAction": "BLOCK",
        "outputEnabled": true,
        "text": "badword1"
      },
      {
        "inputAction": "BLOCK",
        "inputEnabled": true,
        "outputAction": "BLOCK",
        "outputEnabled": true,
        "text": "badword2"
      }
    ]
  },
  "topicPolicyConfig": {
    "topicsConfig": [
      {
        "name": "quantum computing",
        "definition": "Anything related to quantum computing",
        "examples": [],
        "type": "DENY",
        "inputAction": "BLOCK",
        "outputAction": "BLOCK",
        "inputEnabled": true,
        "outputEnabled": true
      }
    ]
  },
  "contentPolicyConfig": {
    "filtersConfig": [
      {
        "type": "VIOLENCE",
        "inputStrength": "HIGH",
        "outputStrength": "HIGH",
        "inputAction": "BLOCK",
        "outputAction": "BLOCK"
      },
      {
        "type": "PROMPT_ATTACK",
        "inputStrength": "HIGH",
        "outputStrength": "NONE",
        "inputAction": "BLOCK"
      },
      {
        "type": "MISCONDUCT",
        "inputStrength": "HIGH",
        "outputStrength": "HIGH",
        "inputAction": "BLOCK",
        "outputAction": "BLOCK"
      },
      {
        "type": "HATE",
        "inputStrength": "HIGH",
        "outputStrength": "HIGH",
        "inputAction": "BLOCK",
        "outputAction": "BLOCK"
      },
      {
        "type": "SEXUAL",
        "inputStrength": "HIGH",
        "outputStrength": "HIGH",
        "inputAction": "BLOCK",
        "outputAction": "BLOCK"
      },
      {
        "type": "INSULTS",
        "inputStrength": "HIGH",
        "outputStrength": "HIGH",
        "inputAction": "BLOCK",
        "outputAction": "BLOCK"
      }
    ]
  }
}
EOF

  4. Apply this configuration by running the following command in your terminal:

    export RESPONSE="$(aws bedrock create-guardrail \
  --cli-input-json file://$PWD/guardrail.json \
  --region $AWS_REGION)
"

    Export the Guardrail ID and Guardrail version as environment variables:

    export GUARDRAILS_ID="$(echo "$RESPONSE" | jq -r '.guardrailId')"
export GUARDRAILS_VERSION="$(echo "$RESPONSE" | jq -r '.version')"

OpenAI API key
  1. Create an OpenAI account.
  2. Get an API key.
  3. Export the API key as a variable: 
     export OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"
 export OPENAI_AUTH_HEADER="Bearer $OPENAI_API_KEY"

Create the AI Model Provider, AI Model, and AI AWS Guardrails Policy

Create both an AI Model Provider and an AI Model with a single kongctl apply command.

You’ll also configure the AI AWS Guardrails Policy to filter LLM traffic based on an existing AWS Guardrail.

kongctl apply -f - --auto-approve --pat "$KONNECT_TOKEN" << 'EOF'
ai_gateway_model_providers:
  - ref: generic-openai
    name: generic-openai
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    type: openai
    config:
      auth:
        type: basic
        headers:
          - name: Authorization
            value: !secret {source: !env OPENAI_AUTH_HEADER}
ai_gateway_policies:
  - ref: my-ai-aws-guardrails-policy
    name: my-ai-aws-guardrails-policy
    display_name: my-ai-aws-guardrails-policy
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    type: ai-aws-guardrails
    enabled: true
    global: false
    config:
      guardrails_id: !env GUARDRAILS_ID
      guardrails_version: !env GUARDRAILS_VERSION
      aws_region: !env AWS_REGION
      aws_access_key_id: !env AWS_ACCESS_KEY_ID
      aws_secret_access_key: !env AWS_SECRET_ACCESS_KEY
ai_gateway_models:
  - ref: my-gpt-4o
    display_name: my-gpt-4o
    name: my-gpt-4o
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    type: model
    enabled: true
    formats: [{ type: openai }]
    config:
      route:
        paths:
          - /
        model:
          body_param: model
          values:
            - my-gpt-4o
    capabilities: [generate]
    policies: [ !ref my-ai-aws-guardrails-policy#name ]
    targets:
      - name: gpt-4o
        provider: generic-openai
        config:
          type: openai
EOF

In this example, we’re setting up the AI AWS Guardrail Policy with:

  • type: ai-aws-guardrails: Specifies that this Policy filters requests using an AWS Guardrail.
  • global: false: Scopes the Policy to only the AI Models it’s explicitly attached to via policies:, rather than applying it to every resource on AI Gateway.
  • config.guardrails_id: Specifies the AWS resource to used for filtering requests.
  • config.aws_region, config.aws_access_key_id, and config.aws_secret_access_key: Specifies your AWS environment.
  • policies: [!ref my-ai-aws-guardrails-policy#name] on the AI Model: Attaches this Policy so it applies to every request routed through my-gpt-4o.

Test the configuration

AWS Guardrails can be set it up to block specific banned words such as the topic of quantum computing, content categories like violence, hate, sexual content, insults, and misconduct, then apply blocking actions on both input and output.

You can test these guardrails using example prompts designed to trigger each blocked category. Sending any of these prompts will result in the following error response:

{
  "error": {
    "message": "Input blocked due to policy violation."
  }
}

This confirms that the guardrail is correctly blocking disallowed content at the input stage.

Blocked words

Use these prompts containing blocked badwords to test the guardrail:

curl -i -X POST "$KONNECT_PROXY_URL/chat/completions" \
     --no-progress-meter --fail-with-body  \
     -H "Accept: application/json"\
     -H "Content-Type: application/json"\
     -H "Authorization: $OPENAI_AUTH_HEADER" \
     --json '{
       "messages": [
         {
           "role": "user",
           "content": "This contains badword1 which should trigger the guardrail."
         }
       ],
       "model": "my-gpt-4o"
     }'

You should see the following response:

Input blocked due to policy violation.

Blocked topic: Quantum computing

Use these prompts to test the guardrail on the topic “quantum computing”:

curl -i -X POST "$KONNECT_PROXY_URL/chat/completions" \
     --no-progress-meter --fail-with-body  \
     -H "Accept: application/json"\
     -H "Content-Type: application/json"\
     -H "Authorization: $OPENAI_AUTH_HEADER" \
     --json '{
       "messages": [
         {
           "role": "user",
           "content": "Explain the principles of quantum computing and its impact on encryption."
         }
       ],
       "model": "my-gpt-4o"
     }'

You should see the following response:

Input blocked due to policy violation.

Blocked categories

Use these prompts to test the guardrail on blocked content categories:

curl -i -X POST "$KONNECT_PROXY_URL/chat/completions" \
     --no-progress-meter --fail-with-body  \
     -H "Accept: application/json"\
     -H "Content-Type: application/json"\
     -H "Authorization: $OPENAI_AUTH_HEADER" \
     --json '{
       "messages": [
         {
           "role": "user",
           "content": "Describe a violent confrontation between two groups."
         }
       ],
       "model": "my-gpt-4o"
     }'

You should see the following response:

Input blocked due to policy violation.

Cleanup

Clean up AWS Bedrock Guardrail

 
aws bedrock delete-guardrail --guardrail-identifier $GUARDRAILS_ID
Something wrong?
Report an Issue | Edit this Page
View all Kong documentation

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!
Contribute

Still need help?

Ask in our Slack Community
Contact Support