Visualize AI Gateway metrics

Kong provides a sample stack using Elasticsearch, Logstash, and Kibana to visualize AI Gateway metrics.

The kong-ai-gateway-observability GitHub repository comes with a configured Kong Gateway instance. You can see the sample Kong Gateway configuration in kong.yaml. It includes:

• A Gateway Service
• A Route with the /gpt4o path
• A Consumer with the API key Bearer department-1-api-key
• Three plugins:
  • HTTP Log to send logs to the pre-configured Logstash server
  • Key Authentication to authenticate the Consumer
  • AI Proxy configured with OpenAI to enable a chat route

The AI Proxy plugin is pre-configured with to fetch the OpenAI key from the OPENAI_AUTH_HEADER environment variable, as defined in the prerequisites.

To use this stack, clone the repository:

git clone https://github.com/KongHQ-CX/kong-ai-gateway-observability
cd kong-ai-gateway-observability

Use the following command to start the sample stack:

docker compose up

Once the stack is running, open a new terminal and send some requests to the /gpt4o endpoint with the Consumer’s API key to generate metrics. For example:

curl -X POST "http://localhost:8000/gpt4o" \
     -H "Accept: application/json"\
     -H "Content-Type: application/json"\
     -H "Authorization: Bearer department-1-api-key" \
     --json '{
       "messages": [
         {
           "role": "system",
           "content": "You are a mathematician"
         },
         {
           "role": "user",
           "content": "What is 1+1?"
         }
       ]
     }'

Go to the following URL to visualize your metrics in Kibana:

http://localhost:5601/app/dashboards#/view/aa8e4cb0-9566-11ef-beb2-c361d8db17a8