docker exec $KONNECT_DP_CONTAINER kong vault get {vault://gcp-sm-vault/test-secret}Store and rotate Mistral API keys as secrets in Google Cloud
You are browsing documentation for an older version of AI Gateway. See the latest documentation here.
Create a secret in Google Cloud Secret Manager and create a service account with the Secret Manager Secret Accessor role. Export your service account key JSON as an environment variable (GCP_SERVICE_ACCOUNT). Then configure a Vault entity with your Secret Manager configuration and ttl set to how many seconds Kong Gateway should wait before picking up the rotated secret. Reference secrets from your Secret Manager vault like the following in a referenceable field: {vault://gcp-sm-vault/test-secret}. Rotate your secret by creating a new secret version in Google Cloud.
Prerequisites
Google Cloud Secret Manager
To add Secret Manager as a Vault backend to Kong Gateway, you must create a project, service account key, and grant IAM permissions. This tutorial also uses gcloud, so you need to install and configure that.
- In the Google Cloud console, create a project and name it
test-gateway-vault. - In the Service Account settings, click the
test-gateway-vaultproject and then click the email address of the service account that you want to create a key for. - From the Keys tab, create a new key from the add key menu and select JSON for the key type.
- Save the JSON file you downloaded.
- From the IAM & Admin settings, click the edit icon next to the service account to grant access to the
Secret Manager Secret Accessorrole for your service account. - Install gcloud.
- Authenticate with gcloud and set your project to
test-gateway-vault:gcloud auth login gcloud config set project test-gateway-vault
Mistral AI API key
In this tutorial, you’ll be storing your Mistral AI API key as a secret in a Konnect Vault.
In the Mistral AI console, create an API key and copy it. You’ll add this API key as a secret to your vault.
Environment variables
Set the environment variables needed to authenticate to Google Cloud:
export GCP_SERVICE_ACCOUNT=$(cat /path/to/file/service-account.json)
export MISTRAL_API_KEY="Bearer YOUR-MISTRAL-API-KEY"Note that the GCP_SERVICE_ACCOUNT variables must be passed when creating your data plane container.
Kong Konnect
This is a Konnect tutorial and requires a Konnect personal access token.
-
Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
-
Export your token to an environment variable:
export KONNECT_TOKEN='YOUR_KONNECT_PAT' -
Run the quickstart script to automatically provision a Control Plane and Data Plane, and configure your environment:
curl -Ls https://get.konghq.com/quickstart | bash -s -- -k $KONNECT_TOKEN \ -e GCP_SERVICE_ACCOUNT --deck-outputThis sets up a Konnect Control Plane named
quickstart, provisions a local Data Plane, and prints out the following environment variable exports:export DECK_KONNECT_TOKEN=$KONNECT_TOKEN export DECK_KONNECT_CONTROL_PLANE_NAME=quickstart export DECK_KONNECT_ADDR=https://us.api.konghq.com export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com export KONNECT_PROXY_URL='http://localhost:8000'Copy and paste these into your terminal to configure your session.
Kong Gateway running
This tutorial requires Kong Gateway Enterprise. If you don’t have Kong Gateway set up yet, you can use the quickstart script with an enterprise license to get an instance of Kong Gateway running almost instantly.
-
Export your license to an environment variable:
export KONG_LICENSE_DATA='LICENSE-CONTENTS-GO-HERE' -
Run the quickstart script:
curl -Ls https://get.konghq.com/quickstart | bash -s -- -e KONG_LICENSE_DATA \ -e GCP_SERVICE_ACCOUNTOnce Kong Gateway is ready, you will see the following message:
Kong Gateway Ready
decK v1.66.1+
To complete this tutorial, install decK. We recommend keeping decK up to date with the latest version (1.66.1).
decK is a CLI tool for managing Kong Gateway declaratively with state files.
This guide uses deck gateway apply, which directly applies entity configuration to your Gateway instance.
You can check your current decK version with deck version.
Required entities
For this tutorial, you’ll need Kong Gateway entities, like Gateway Services and Routes, pre-configured. These entities are essential for Kong Gateway to function but installing them isn’t the focus of this guide. Follow these steps to pre-configure them:
-
Run the following command:
echo ' _format_version: "3.0" services: - name: example-service url: http://httpbin.konghq.com/anything routes: - name: example-route paths: - "/anything" service: name: example-service protocols: - http - https ' | deck gateway apply -
To learn more about entities, you can read our entities documentation.
Add an invalid API key as a secret in Google Cloud Secret Manager
In this tutorial, first we’ll create a secret with an invalid API key in Google Cloud Secret Manager. Later, we’ll add the correct API key as another secret version, but this allows us to test if Kong Gateway picks up the rotated secret correctly.
Create a secret called test-secret and then create a new secret version with the secret value of Bearer invalid:
gcloud secrets create test-secret \
--replication-policy="automatic"
echo -n "Bearer invalid" | \
gcloud secrets versions add test-secret --data-file=-The first command is supported on Linux, macOS, and Cloud Shell. For other distributions, see Create a secret in Google Cloud documentation.
Configure Secret Manager as a vault with the Vault entity
To enable Secret Manager as your vault in Kong Gateway, you can use the Vault entity.
In this tutorial, we are configuring the time-to-live (ttl) as 60 seconds/1 minute. This tells Kong Gateway to check every minute with Google Cloud to get the rotated secret. We’ve configured a low value so that we can quickly validate that the secret rotation is functioning as expected.
echo '
_format_version: "3.0"
vaults:
- name: gcp
description: Stored secrets in Secret Manager
prefix: gcp-sm-vault
config:
project_id: test-gateway-vault
ttl: 60
' | deck gateway apply -Enable the AI Proxy plugin
In this tutorial, you’ll use the Mistral API key you stored as a secret to generate an answer to a question using the AI Proxy plugin.
echo '
_format_version: "3.0"
plugins:
- name: ai-proxy
route: example-route
config:
route_type: llm/v1/chat
auth:
header_name: Authorization
header_value: "{vault://gcp-sm-vault/test-secret}"
model:
provider: mistral
name: mistral-tiny
options:
mistral_format: openai
upstream_url: https://api.mistral.ai/v1/chat/completions
' | deck gateway apply -Validate that Kong Gateway uses the invalid API key from the secret
First, let’s validate that the secret was stored correctly in Google Cloud by calling a secret from your vault using the kong vault get command within the Data Plane container.
docker exec kong-quickstart-gateway kong vault get {vault://gcp-sm-vault/test-secret}If the vault was configured correctly, this command should return Bearer invalid.
Now, let’s validate that when we make a call to the Route associated with the AI Proxy plugin, that it is using this invalid API key stored in our secret:
curl -X POST "$KONNECT_PROXY_URL/anything" \
--no-progress-meter --fail-with-body \
-H "Accept: application/json"\
-H "Content-Type: application/json" \
--json '{
"messages": [
{
"role": "system",
"content": "You are a mathematician"
},
{
"role": "user",
"content": "What is 1+1?"
}
]
}'You should see the following response:
Unauthorizedcurl -X POST "http://localhost:8000/anything" \
--no-progress-meter --fail-with-body \
-H "Accept: application/json"\
-H "Content-Type: application/json" \
--json '{
"messages": [
{
"role": "system",
"content": "You are a mathematician"
},
{
"role": "user",
"content": "What is 1+1?"
}
]
}'You should see the following response:
UnauthorizedYou should get a 401 error with the message Unauthorized because we’re currently using an invalid API key.
Rotate the secret in Secret Manager
We can now rotate the secret with the correct API key from Mistral. You can rotate a secret by creating a new secret version with the new secret value. Kong Gateway will fetch the new secret value based on the ttl setting we configured in the Vault entity.
Rotate the secret with the valid Mistral API key:
echo -n "$MISTRAL_API_KEY" | \
gcloud secrets versions add test-secret --data-file=-Validate that Kong Gateway uses the valid API key from the rotated secret
Now we can validate that Kong Gateway picks up the valid Mistral API key from the rotated secret. Since Kong Gateway is configured to pick up any rotated secrets every 60 seconds, the following command waits a minute before sending a request:
sleep 60 && curl -X POST "$KONNECT_PROXY_URL/anything" \
--no-progress-meter --fail-with-body \
-H "Accept: application/json"\
-H "Content-Type: application/json" \
--json '{
"messages": [
{
"role": "system",
"content": "You are a mathematician"
},
{
"role": "user",
"content": "What is 1+1?"
}
]
}'sleep 60 && curl -X POST "http://localhost:8000/anything" \
--no-progress-meter --fail-with-body \
-H "Accept: application/json"\
-H "Content-Type: application/json" \
--json '{
"messages": [
{
"role": "system",
"content": "You are a mathematician"
},
{
"role": "user",
"content": "What is 1+1?"
}
]
}'You should get a 200 error with an answer to the chat response because Kong Gateway picked up the rotated secret with the valid API key.
Cleanup
Destroy the Kong Gateway container
curl -Ls https://get.konghq.com/quickstart | bash -s -- -dClean up Konnect environment
If you created a new control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.
FAQs
How do I fix the Error: could not get value from external vault (no value found (unable to retrieve secret from gcp secret manager (code : 403, status: PERMISSION_DENIED))) error when I try to use my secret from the Google Cloud vault?
Verify that your Google Cloud service account has the Secret Manager Secret Accessor role. This role is required for Kong Gateway to access secrets in the vault.
I’m using Google Workload Identity, how do I configure a Vault?
To use GCP Secret Manager with
Workload Identity
on a GKE cluster, update your pod spec so that the service account (GCP_SERVICE_ACCOUNT) is
attached to the pod. For configuration information, read the Workload
Identity configuration
documentation.
Notes:
- With Workload Identity, setting the
GCP_SERVICE_ACCOUNTisn’t necessary.- When using GCP Vault as a backend, make sure you have configured
systemas part of thelua_ssl_trusted_certificateconfiguration directive so that the SSL certificates used by the official GCP API can be trusted by Kong Gateway.