/config-stores endpoint./vaults/ endpoint or UI./secrets endpoint or UI.{vault://mysecretvault/mistral-key}) in the AI Proxy plugin header_value.This is a Konnect tutorial and requires a Konnect personal access token.
Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
Export your token to an environment variable:
export KONNECT_TOKEN='YOUR_KONNECT_PAT'
Run the quickstart script to automatically provision a Control Plane and Data Plane, and configure your environment:
curl -Ls https://get.konghq.com/quickstart | bash -s -- -k $KONNECT_TOKEN --deck-output
This sets up a Konnect Control Plane named quickstart, provisions a local Data Plane, and prints out the following environment variable exports:
export DECK_KONNECT_TOKEN=$KONNECT_TOKEN
export DECK_KONNECT_CONTROL_PLANE_NAME=quickstart
export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
export KONNECT_PROXY_URL='http://localhost:8000'
Copy and paste these into your terminal to configure your session.
decK is a CLI tool for managing Kong Gateway declaratively with state files. To complete this tutorial, install decK version 1.43 or later.
This guide uses deck gateway apply, which directly applies entity configuration to your Gateway instance.
We recommend upgrading your decK installation to take advantage of this tool.
You can check your current decK version with deck version.
For this tutorial, you’ll need Kong Gateway entities, like Gateway Services and Routes, pre-configured. These entities are essential for Kong Gateway to function but installing them isn’t the focus of this guide. Follow these steps to pre-configure them:
Run the following command:
echo '
_format_version: "3.0"
services:
- name: example-service
url: http://httpbin.konghq.com/anything
routes:
- name: example-route
paths:
- "/anything"
service:
name: example-service
' | deck gateway apply -
To learn more about entities, you can read our entities documentation.
In this tutorial, you’ll be storing your Mistral AI API key as a secret in a Konnect Vault.
In the Mistral AI console, create an API key and copy it. You’ll add this API key as a secret to your vault.
Export the API key as an environment variable:
export MISTRAL_API_KEY='YOUR API KEY'
To use the copy, paste, and run the instructions in this how-to, you must export the additional environmental variable CONTROL_PLANE_ID:
export CONTROL_PLANE_ID='YOUR CONTROL PLANE ID'
You can find your control plane UUID by navigating to the control plane in the Konnect UI or by sending a GET request to the /control-planes endpoint.
Before you can configure a Konnect Vault, you must first create a Config Store using the Control Planes Configuration API by sending a POST request to the /config-stores endpoint:
curl -X POST "https://us.api.konghq.com/v2/control-planes/$CONTROL_PLANE_ID/config-stores" \
--no-progress-meter --fail-with-body \
-H "Authorization: Bearer $KONNECT_TOKEN" \
--json '{
"name": "my-config-store"
}'
Export your Config Store ID as an environment variable so you can use it later:
export DECK_CONFIG_STORE_ID='CONFIG STORE ID'
Note: If you’re configuring the Konnect Vault via the Konnect UI, you can skip this step as the UI creates the Config Store for you.
Enable Konnect as your vault with the Vault entity:
In this tutorial, you’ll be storing the Mistral API key you set previously and using it to generate an answer to a question using the AI Proxy plugin. By storing it as a secret in a Konnect Vault, you can reference it during plugin configuration in the next step.
To reference your stored Mistral API key, you use the prefix from your Vault config, the name of the secret, and optionally the property in the secret you want to use. Now, you’ll reference the Mistral API key as a secret in the authorization header of the AI Proxy plugin configuration.
Enable the AI Proxy plugin on your Route:
echo '
_format_version: "3.0"
plugins:
- name: ai-proxy
route: example-route
config:
route_type: llm/v1/chat
auth:
header_name: Authorization
header_value: "{vault://mysecretvault/mistral-key}"
model:
provider: mistral
name: mistral-tiny
options:
mistral_format: openai
upstream_url: https://api.mistral.ai/v1/chat/completions
' | deck gateway apply -
You can use the AI Proxy plugin to confirm that the plugin is using the correct API key when a request is made:
curl -X POST "$KONNECT_PROXY_URL/anything" \
--no-progress-meter --fail-with-body \
-H "Accept: application/json"\
-H "Content-Type: application/json" \
--json '{
"messages": [
{
"role": "system",
"content": "You are a mathematician"
},
{
"role": "user",
"content": "What is 1+1?"
}
]
}'
If you created a new control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.
How do I replace certificates used in Kong Gateway data plane nodes with a secret reference?
Set up a Konnect or any other Vault and define the certificate and key in a secret in the Vault.