Use AI to protect sensitive information in requests

Uses: Kong Gateway AI Gateway decK

Deployment Platform:

Prerequisites

Kong Konnect

This is a Konnect tutorial. If you don’t have a Konnect account, you can get started quickly with our onboarding wizard.

The following Konnect items are required to complete this tutorial:
- Personal access token (PAT): Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
- Control Plane Name: You can use an existing Control Plane or create a new one to use for this tutorial.
- Konnect Proxy URL: By default, a self-hosted Data Plane uses http://localhost:8000. You can set up Data Plane nodes for your Control Plane from the Gateway Manager in Konnect.

Set the personal access token, the Control Plane name, the Control Plane URL, and the Konnect proxy URL as environment variables:

 export DECK_KONNECT_TOKEN='YOUR KONNECT TOKEN'
 export DECK_KONNECT_CONTROL_PLANE_NAME='YOUR CONTROL PLANE NAME'
 export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
 export KONNECT_PROXY_URL='KONNECT PROXY URL'

Kong Gateway running

This tutorial requires Kong Gateway Enterprise. If you don’t have Kong Gateway set up yet, you can use the quickstart script with an enterprise license to get an instance of Kong Gateway running almost instantly.

Export your license to an environment variable:

 export KONG_LICENSE_DATA='LICENSE-CONTENTS-GO-HERE'

Run the quickstart script:

 curl -Ls https://get.konghq.com/quickstart | bash -s -- -e KONG_LICENSE_DATA 

Once Kong Gateway is ready, you will see the following message:

 Kong Gateway Ready

decK

decK is a CLI tool for managing Kong Gateway declaratively with state files. To complete this tutorial you will first need to install decK.

Required entities

For this tutorial, you’ll need Kong Gateway entities, like Gateway Services and Routes, pre-configured. These entities are essential for Kong Gateway to function but installing them isn’t the focus of this guide. Follow these steps to pre-configure them:

Run the following command:

echo '
_format_version: "3.0"
services:
  - name: example-service
    url: http://httpbin.konghq.com/anything
routes:
  - name: example-route
    paths:
    - "/anything"
    service:
      name: example-service
' | deck gateway apply -

To learn more about entities, you can read our entities documentation.

OpenAI

This tutorial uses OpenAI:

Create an OpenAI account.
Get an API key.
Create a decK variable with the API key:

export DECK_OPENAI_API_KEY='YOUR OPENAI API KEY'

AI PII Anonymizer service access

In this tutorial, we’ll use the AI PII Anonymizer service provided by Kong. Since this Docker image is private, you need to reach out to Kong Support to get access.

You can also use your own service.

Start the Kong AI PII Anonymizer service

Make sure you have access to the AI PII service, then run the following command to start it locally with Docker:

docker run --platform linux/x86_64 -d --name pii-service -p 9000:8080 kong/ai-pii-service

Enable the AI Proxy plugin

Use the following command to enable the AI Proxy plugin configured with a chat route using OpenAI:

echo '
_format_version: "3.0"
plugins:
  - name: ai-proxy
    config:
      route_type: llm/v1/chat
      auth:
        header_name: Authorization
        header_value: Bearer ${{ env "DECK_OPENAI_API_KEY" }}
      model:
        provider: openai
        name: gpt-4
        options:
          max_tokens: 512
          temperature: 1.0
' | deck gateway apply -

Enable the AI Sanitizer plugin

Configure the AI Sanitizer plugin to use the AI PII Anonymizer service to anonymize general information and phone numbers:

echo '
_format_version: "3.0"
plugins:
  - name: ai-sanitizer
    config:
      anonymize:
      - phone
      - general
      port: 9000
      host: host.docker.internal
      redact_type: synthetic
      stop_on_error: true
      recover_redacted: false
' | deck gateway apply -

Validate

To validate, send a request that contains PII, for example:

curl -X POST "$KONNECT_PROXY_URL/anything" \
     -H "Accept: application/json"\
     -H "Content-Type: application/json" \
     --json '{
       "messages": [
         {
           "role": "system",
           "content": "You are a helpful assistant. Please repeat the following information back to me."
         },
         {
           "role": "user",
           "content": "My name is John Doe, my phone number is 123-456-7890."
         }
       ]
     }'

curl -X POST "http://localhost:8000/anything" \
     -H "Accept: application/json"\
     -H "Content-Type: application/json" \
     --json '{
       "messages": [
         {
           "role": "system",
           "content": "You are a helpful assistant. Please repeat the following information back to me."
         },
         {
           "role": "user",
           "content": "My name is John Doe, my phone number is 123-456-7890."
         }
       ]
     }'

If the plugin was configured correctly, you will received a response with all PII information scrubbed, for example:

Your name is Jesse Mason and your phone number is 001-204-028-1684x83574.

Cleanup

Clean up Konnect environment

If you created a new control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.

Destroy the Kong Gateway container

curl -Ls https://get.konghq.com/quickstart | bash -s -- -d