Home / Kong AI Gateway
Edit
Report

Configure dynamic authentication to LLM providers using HashiCorp vault

Uses: Kong Gateway AI Gateway deck
Deployment Platform
Related Documentation
Gateway Documentation decK Documentation
In This Series
  1. Configure dynamic authentication to LLM providers using HashiCorp vault
  2. Route OpenAI chat traffic using semantic balancing and Vault-stored keys
Tags
#secrets-management #security #hashicorp-vault
Related Resources
Secrets management
Configure HashiCorp Vault as a vault backend with certificate authentication
Configure HashiCorp Vault as a vault backend with OAuth2
Store Keyring data in a HashiCorp Vault
Configure Hashicorp Vault with Kong Ingress Controller
Minimum Version
Kong Gateway - 3.4
TL;DR

Store secrets using vault kv put secret/openai key="OPENAI_API_KEY" to HashiCorp Vault. Then configure a Vault entity in Kong Gateway with the host, token, and mount path. Inside the Gateway container, run kong vault get {vault://hashicorp-vault/openai/key} to confirm access. Next Use the {vault://...} syntax in a plugin field to dynamically authenticate to LLM providers such as OpenAI and Mistral.

Prerequisites

Kong Konnect

This is a Konnect tutorial and requires a Konnect personal access token.

  1. Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.

  2. Export your token to an environment variable:

     export KONNECT_TOKEN='YOUR_KONNECT_PAT'

  3. Run the quickstart script to automatically provision a Control Plane and Data Plane, and configure your environment:

     curl -Ls https://get.konghq.com/quickstart | bash -s -- -k $KONNECT_TOKEN --deck-output

    This sets up a Konnect Control Plane named quickstart, provisions a local Data Plane, and prints out the following environment variable exports:

     export DECK_KONNECT_TOKEN=$KONNECT_TOKEN
 export DECK_KONNECT_CONTROL_PLANE_NAME=quickstart
 export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
 export KONNECT_PROXY_URL='http://localhost:8000'

    Copy and paste these into your terminal to configure your session.

Kong Gateway running

This tutorial requires Kong Gateway Enterprise. If you don’t have Kong Gateway set up yet, you can use the quickstart script with an enterprise license to get an instance of Kong Gateway running almost instantly.

  1. Export your license to an environment variable:

     export KONG_LICENSE_DATA='LICENSE-CONTENTS-GO-HERE'

  2. Run the quickstart script:

    curl -Ls https://get.konghq.com/quickstart | bash -s -- -e KONG_LICENSE_DATA

    Once Kong Gateway is ready, you will see the following message:

     Kong Gateway Ready

decK   v1.43+

decK is a CLI tool for managing Kong Gateway declaratively with state files. To complete this tutorial, install decK version 1.43 or later.

This guide uses deck gateway apply, which directly applies entity configuration to your Gateway instance. We recommend upgrading your decK installation to take advantage of this tool.

You can check your current decK version with deck version.

HashiCorp Vault

This how-to requires you to have a dev mode or self-managed HashiCorp Vault. The following instructions will guide you through configuring a HashiCorp Vault in dev mode with the resources you need to integrate it with Kong Gateway.

Important: This tutorial uses the literal root string as your token, which should only be used in testing and development environments.

  1. In a terminal, start your Vault dev server with root as your token. 
    docker run -d --name vault -p 8200:8200 -e 'VAULT_DEV_ROOT_TOKEN_ID=root' hashicorp/vault
  2. Export the VAULT_ADDR and VAULT_TOKEN: 
    export VAULT_ADDR="http://host.docker.internal:8200"
export VAULT_TOKEN="root"
export VAULT_HOST="host.docker.internal"

OpenAI

This tutorial uses OpenAI:

  1. Create an OpenAI account.
  2. Get an API key.

  3. Create a decK variable with the API key:

    export DECK_OPENAI_API_KEY='YOUR OPENAI API KEY'

Mistral

This tutorial uses Mistral:

  1. Create a Mistral account.
  2. Get your API key.
  3. Export a decK environment variable with the Mistral API key:
export DECK_MISTRAL_API_KEY='YOUR MISTRAL API KEY'

Create secrets in HashiCorp Vault

Replace the placeholder with your OpenAI API key and run:

curl -X POST http://localhost:8200/v1/secret/data/openai \
     -H "X-Vault-Token: $VAULT_TOKEN" \
     -H "Content-Type: application/json" \
     --data '{"data": {"key": "'$DECK_OPENAI_API_KEY'" }}'

Next, replace the placeholder with your Mistral API key and run:

curl -X POST http://localhost:8200/v1/secret/data/mistral \
     -H "X-Vault-Token: $VAULT_TOKEN" \
     -H "Content-Type: application/json" \
     --data '{"data": {"key": "'$DECK_MISTRAL_API_KEY'" }}'

Both secrets will be stored under their respective paths (secret/openai and secret/mistral) in the key field.

Create decK environment variables

We’ll use decK environment variables for the host and token in the Kong Gateway Vault configuration. This is because these values typically vary between environments.

In this tutorial, we’re using host.docker.internal as our host instead of the localhost variable that HashiCorp Vault uses by default. This is because if you used the quick-start script Kong Gateway is running in a Docker container and uses a different localhost.

Because we are running HashiCorp Vault in dev mode, we are using root for our token value.

export DECK_HCV_HOST='host.docker.internal'
export DECK_HCV_TOKEN='root'

Create a Vault entity for HashiCorp Vault

Using decK, create a Vault entity in the kong.yaml file with the required parameters for HashiCorp Vault:

echo '
_format_version: "3.0"
vaults:
  - name: hcv
    prefix: hashicorp-vault
    description: Storing secrets in HashiCorp Vault
    config:
      host: "${{ env "DECK_HCV_HOST" }}"
      token: "${{ env "DECK_HCV_TOKEN" }}"
      kv: v2
      mount: secret
      port: 8200
      protocol: http
' | deck gateway apply -

Validate

Since Konnect Data Plane container names can vary, set your container name as an environment variable:

export KONNECT_DP_CONTAINER='your-dp-container-name'

To validate that the secret was stored correctly in HashiCorp Vault, you can call a secret from your vault using the kong vault get command within the Data Plane container.

docker exec $KONNECT_DP_CONTAINER kong vault get {vault://hashicorp-vault/mistral/key}

docker exec kong-quickstart-gateway kong vault get {vault://hashicorp-vault/mistral/key}

docker exec $KONNECT_DP_CONTAINER kong vault get {vault://hashicorp-vault/openai/key}

docker exec kong-quickstart-gateway kong vault get {vault://hashicorp-vault/openai/key}

If the vault was configured correctly, this command should return the value of the secrets for OpenAI and Mistral. You can use {vault://hashicorp-vault/openai/key} and {vault://hashicorp-vault/mistral/key} to reference the secret in any referenceable field.

Next → 2. Route OpenAI chat traffic using semantic balancing and Vault-stored keys
Something wrong?
Report an Issue | Edit this Page

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!
Contribute

Still need help

Ask in our Forum
Contact Support