Transform a request body using OpenAI in Kong Gateway

Uses: Kong Gateway decK

Deployment Platform:

Prerequisites

Kong Konnect

This is a Konnect tutorial and requires a Konnect personal access token.

Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
Export your token to an environment variable:
```
 export KONNECT_TOKEN='YOUR_KONNECT_PAT'
```
Run the quickstart script to automatically provision a Control Plane and Data Plane, and configure your environment:
```
 curl -Ls https://get.konghq.com/quickstart | bash -s -- -k $KONNECT_TOKEN --deck-output
```
This sets up a Konnect Control Plane named quickstart, provisions a local Data Plane, and prints out the following environment variable exports:
```
 export DECK_KONNECT_TOKEN=$KONNECT_TOKEN
 export DECK_KONNECT_CONTROL_PLANE_NAME=quickstart
 export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
 export KONNECT_PROXY_URL='http://localhost:8000'
```
Copy and paste these into your terminal to configure your session.

Kong Gateway running

This tutorial requires Kong Gateway Enterprise. If you don’t have Kong Gateway set up yet, you can use the quickstart script with an enterprise license to get an instance of Kong Gateway running almost instantly.

Export your license to an environment variable:

 export KONG_LICENSE_DATA='LICENSE-CONTENTS-GO-HERE'

Run the quickstart script:

 curl -Ls https://get.konghq.com/quickstart | bash -s -- -e KONG_LICENSE_DATA 

Once Kong Gateway is ready, you will see the following message:

 Kong Gateway Ready

decK

decK is a CLI tool for managing Kong Gateway declaratively with state files. To complete this tutorial you will first need to install decK.

Required entities

For this tutorial, you’ll need Kong Gateway entities, like Gateway Services and Routes, pre-configured. These entities are essential for Kong Gateway to function but installing them isn’t the focus of this guide. Follow these steps to pre-configure them:

Run the following command:

echo '
_format_version: "3.0"
services:
  - name: example-service
    url: http://httpbin.konghq.com/anything
routes:
  - name: example-route
    paths:
    - "/anything"
    service:
      name: example-service
' | deck gateway apply -

To learn more about entities, you can read our entities documentation.

OpenAI

This tutorial uses OpenAI:

Create an OpenAI account.
Get an API key.
Create a decK variable with the API key:

export DECK_OPENAI_API_KEY='YOUR OPENAI API KEY'

Enable the AI Request Transformer plugin

In this example, we expect the client to send requests with a JSON body containing a city element. We want to transform this request to add the corresponding country before proxying the request to the upstream.

We also want to make sure that the LLM only returns the JSON content and doesn’t add extra text around it. There are two ways to do this:

Include this in the prompt, by adding “Return only the JSON message, no extra text” for example.
Specify a regex in the config.transformation_extract_pattern parameter to extract only the data we need. This is the option we’ll use in this example.

Configure the AI Request Transformer plugin with the required LLM details, the transformation prompt, and the expected request body pattern to extract:

echo '
_format_version: "3.0"
plugins:
  - name: ai-request-transformer
    config:
      prompt: In my JSON message, anywhere there is a JSON tag for a city, also add
        a country tag with the name of the country that city is in.
      transformation_extract_pattern: "{((.|\\n)*)}"
      llm:
        route_type: llm/v1/chat
        auth:
          header_name: Authorization
          header_value: Bearer ${{ env "DECK_OPENAI_API_KEY" }}
        model:
          provider: openai
          name: gpt-4
' | deck gateway apply -

Validate

To check that the request transformation is working, send a request with a JSON body containing a city tag:

curl -X POST "$KONNECT_PROXY_URL/anything" \
     -H "Accept: application/json"\
     -H "Content-Type: application/json" \
     --json '{
       "user": {
         "name": "Kong User",
         "city": "London"
       }
     }'

curl -X POST "http://localhost:8000/anything" \
     -H "Accept: application/json"\
     -H "Content-Type: application/json" \
     --json '{
       "user": {
         "name": "Kong User",
         "city": "London"
       }
     }'

In this example, we’re using httpbin.konghq.com/anything as the upstream. It returns anything that is passed to the request, which means the response contains the transformed request body received by the upstream:

{
   "json":{
      "user":{
         "city":"London",
         "country":"United Kingdom",
         "name":"Kong User"
      }
   }
}

Cleanup

Clean up Konnect environment

If you created a new control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.

Destroy the Kong Gateway container

curl -Ls https://get.konghq.com/quickstart | bash -s -- -d