Configure dynamic authentication to LLM providers using HashiCorp vault

Uses: Kong Gateway AI Gateway deck

Deployment Platform:

Prerequisites

Kong Konnect

This is a Konnect tutorial and requires a Konnect personal access token.

Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
Export your token to an environment variable:
```
 export KONNECT_TOKEN='YOUR_KONNECT_PAT'
```
Run the quickstart script to automatically provision a Control Plane and Data Plane, and configure your environment:
```
 curl -Ls https://get.konghq.com/quickstart | bash -s -- -k $KONNECT_TOKEN --deck-output
```
This sets up a Konnect Control Plane named quickstart, provisions a local Data Plane, and prints out the following environment variable exports:
```
 export DECK_KONNECT_TOKEN=$KONNECT_TOKEN
 export DECK_KONNECT_CONTROL_PLANE_NAME=quickstart
 export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
 export KONNECT_PROXY_URL='http://localhost:8000'
```
Copy and paste these into your terminal to configure your session.

Kong Gateway running

This tutorial requires Kong Gateway Enterprise. If you don’t have Kong Gateway set up yet, you can use the quickstart script with an enterprise license to get an instance of Kong Gateway running almost instantly.

Export your license to an environment variable:

 export KONG_LICENSE_DATA='LICENSE-CONTENTS-GO-HERE'

Run the quickstart script:

curl -Ls https://get.konghq.com/quickstart | bash -s -- -e KONG_LICENSE_DATA 

Once Kong Gateway is ready, you will see the following message:

 Kong Gateway Ready

decK

decK is a CLI tool for managing Kong Gateway declaratively with state files. To complete this tutorial you will first need to install decK.

HashiCorp Vault

This how-to requires you to have a dev mode or self-managed HashiCorp Vault. The following instructions will guide you through configuring a HashiCorp Vault in dev mode with the resources you need to integrate it with Kong Gateway.

Important: This tutorial uses the literal root string as your token, which should only be used in testing and development environments.

Install HashiCorp Vault.
In a terminal, start your Vault dev server with root as your token.
```
vault server -dev -dev-root-token-id root
```
In the output from the previous command, copy the VAULT_ADDR to export.
In a new terminal window, export your VAULT_ADDR as an environment variable.
Verify that your Vault is running correctly:
```
vault status
```
Authenticate with Vault:
```
vault login root
```
Verify that you are using the v2 secrets engine:
```
vault read sys/mounts/secret
```
The options key should have the map[version:2] value.

OpenAI

This tutorial uses OpenAI:

Mistral

This tutorial uses OpenAI:

Create secrets in HashiCorp Vault

Replace the placeholder with your actual OpenAI API key and run:

vault kv put secret/openai key="YOUR_OPENAI_API_KEY"

Next, replace the placeholder with your actual Mistral API key and run:

vault kv put secret/mistral key="YOUR_MISTRAL_API_KEY"

Both secrets will be stored under their respective paths (secret/openai and secret/mistral) in the key field.

Create decK environment variables

We’ll use decK environment variables for the host and token in the Kong Gateway Vault configuration. This is because these values typically vary between environments.

In this tutorial, we’re using host.docker.internal as our host instead of the localhost variable that HashiCorp Vault uses by default. This is because if you used the quick-start script Kong Gateway is running in a Docker container and uses a different localhost.

Because we are running HashiCorp Vault in dev mode, we are using root for our token value.

export DECK_HCV_HOST="host.docker.internal"
export DECK_HCV_TOKEN="root"

export DECK_HCV_HOST="host.docker.internal"
export DECK_HCV_TOKEN="root"

Create a Vault entity for HashiCorp Vault

Using decK, create a Vault entity in the kong.yaml file with the required parameters for HashiCorp Vault:

echo '
_format_version: "3.0"
vaults:
  - name: hcv
    prefix: hashicorp-vault
    description: Storing secrets in HashiCorp Vault
    config:
      host: "${{ env "DECK_HCV_HOST" }}"
      token: "${{ env "DECK_HCV_TOKEN" }}"
      kv: v2
      mount: secret
      port: 8200
      protocol: http
' | deck gateway apply -

Validate

Since Konnect Data Plane container names can vary, set your container name as an environment variable:

export KONNECT_DP_CONTAINER='your-dp-container-name'

To validate that the secret was stored correctly in HashiCorp Vault, you can call a secret from your vault using the kong vault get command within the Data Plane container.

 kong vault get {vault://hashicorp-vault/mistral/key}

 kong vault get {vault://hashicorp-vault/mistral/key}

 kong vault get {vault://hashicorp-vault/openai/key}

 kong vault get {vault://hashicorp-vault/openai/key}

If the vault was configured correctly, this command should return the value of the secrets for OpenAI and Mistral. You can use {vault://hashicorp-vault/openai/key} and {vault://hashicorp-vault/mistral/key} to reference the secret in any referenceable field.

Next → 2. Route OpenAI chat traffic using semantic balancing and Vault-stored keys