Create a secret in Google Cloud Secret Manager and create a service account with the Secret Manager Secret Accessor role. Export your service account key JSON as an environment variable (GCP_SERVICE_ACCOUNT). Then configure a Vault entity with your Secret Manager configuration and ttl set to how many seconds Kong Gateway should wait before picking up the rotated secret. Reference secrets from your Secret Manager vault like the following in a referenceable field: {vault://gcp-sm-vault/test-secret}. Rotate your secret by creating a new secret version in Google Cloud.
To add Secret Manager as a Vault backend to Kong Gateway, you must create a project, service account key, and grant IAM permissions. This tutorial also uses gcloud, so you need to install and configure that.
test-gateway-vault.test-gateway-vault project and then click the email address of the service account that you want to create a key for.Secret Manager Secret Accessor role for your service account.test-gateway-vault:
gcloud auth login
gcloud config set project test-gateway-vault
In this tutorial, you’ll be storing your Mistral AI API key as a secret in a Konnect Vault.
In the Mistral AI console, create an API key and copy it. You’ll add this API key as a secret to your vault.
Set the environment variables needed to authenticate to Google Cloud:
export GCP_SERVICE_ACCOUNT=$(cat /path/to/file/service-account.json)
export MISTRAL_API_KEY="Bearer YOUR-MISTRAL-API-KEY"
Note that the GCP_SERVICE_ACCOUNT variables must be passed when creating your data plane container.
This is a Konnect tutorial and requires a Konnect personal access token.
Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
Export your token to an environment variable:
export KONNECT_TOKEN='YOUR_KONNECT_PAT'
Run the quickstart script to automatically provision a Control Plane and Data Plane, and configure your environment:
curl -Ls https://get.konghq.com/quickstart | bash -s -- -k $KONNECT_TOKEN -e GCP_SERVICE_ACCOUNT --deck-output
This sets up a Konnect Control Plane named quickstart, provisions a local Data Plane, and prints out the following environment variable exports:
export DECK_KONNECT_TOKEN=$KONNECT_TOKEN
export DECK_KONNECT_CONTROL_PLANE_NAME=quickstart
export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
export KONNECT_PROXY_URL='http://localhost:8000'
Copy and paste these into your terminal to configure your session.
This tutorial requires Kong Gateway Enterprise. If you don’t have Kong Gateway set up yet, you can use the quickstart script with an enterprise license to get an instance of Kong Gateway running almost instantly.
Export your license to an environment variable:
export KONG_LICENSE_DATA='LICENSE-CONTENTS-GO-HERE'
Run the quickstart script:
curl -Ls https://get.konghq.com/quickstart | bash -s -- -e KONG_LICENSE_DATA \
-e GCP_SERVICE_ACCOUNT
Once Kong Gateway is ready, you will see the following message:
Kong Gateway Ready
decK is a CLI tool for managing Kong Gateway declaratively with state files. To complete this tutorial, install decK version 1.43 or later.
This guide uses deck gateway apply, which directly applies entity configuration to your Gateway instance.
We recommend upgrading your decK installation to take advantage of this tool.
You can check your current decK version with deck version.
For this tutorial, you’ll need Kong Gateway entities, like Gateway Services and Routes, pre-configured. These entities are essential for Kong Gateway to function but installing them isn’t the focus of this guide. Follow these steps to pre-configure them:
Run the following command:
echo '
_format_version: "3.0"
services:
- name: example-service
url: http://httpbin.konghq.com/anything
routes:
- name: example-route
paths:
- "/anything"
service:
name: example-service
' | deck gateway apply -
To learn more about entities, you can read our entities documentation.
In this tutorial, first we’ll create a secret with an invalid API key in Google Cloud Secret Manager. Later, we’ll add the correct API key as another secret version, but this allows us to test if Kong Gateway picks up the rotated secret correctly.
Create a secret called test-secret and then create a new secret version with the secret value of Bearer invalid:
gcloud secrets create test-secret \
--replication-policy="automatic"
echo -n "Bearer invalid" | \
gcloud secrets versions add test-secret --data-file=-
The first command is supported on Linux, macOS, and Cloud Shell. For other distributions, see Create a secret in Google Cloud documentation.
To enable Secret Manager as your vault in Kong Gateway, you can use the Vault entity.
In this tutorial, we are configuring the time-to-live (ttl) as 60 seconds/1 minute. This tells Kong Gateway to check every minute with Google Cloud to get the rotated secret. We’ve configured a low value so that we can quickly validate that the secret rotation is functioning as expected.
echo '
_format_version: "3.0"
vaults:
- name: gcp
description: Stored secrets in Secret Manager
prefix: gcp-sm-vault
config:
project_id: test-gateway-vault
ttl: 60
' | deck gateway apply -
In this tutorial, you’ll use the Mistral API key you stored as a secret to generate an answer to a question using the AI Proxy plugin.
echo '
_format_version: "3.0"
plugins:
- name: ai-proxy
route: example-route
config:
route_type: llm/v1/chat
auth:
header_name: Authorization
header_value: "{vault://gcp-sm-vault/test-secret}"
model:
provider: mistral
name: mistral-tiny
options:
mistral_format: openai
upstream_url: https://api.mistral.ai/v1/chat/completions
' | deck gateway apply -
First, let’s validate that the secret was stored correctly in Google Cloud by calling a secret from your vault using the kong vault get command within the Data Plane container.
docker exec $KONNECT_DP_CONTAINER kong vault get {vault://gcp-sm-vault/test-secret}
docker exec kong-quickstart-gateway kong vault get {vault://gcp-sm-vault/test-secret}
If the vault was configured correctly, this command should return Bearer invalid.
Now, let’s validate that when we make a call to the Route associated with the AI Proxy plugin, that it is using this invalid API key stored in our secret:
curl -X POST "$KONNECT_PROXY_URL/anything" \
--no-progress-meter --fail-with-body \
-H "Accept: application/json"\
-H "Content-Type: application/json" \
--json '{
"messages": [
{
"role": "system",
"content": "You are a mathematician"
},
{
"role": "user",
"content": "What is 1+1?"
}
]
}'
You should see the following response:
Unauthorized
curl -X POST "http://localhost:8000/anything" \
--no-progress-meter --fail-with-body \
-H "Accept: application/json"\
-H "Content-Type: application/json" \
--json '{
"messages": [
{
"role": "system",
"content": "You are a mathematician"
},
{
"role": "user",
"content": "What is 1+1?"
}
]
}'
You should see the following response:
Unauthorized
You should get a 401 error with the message Unauthorized because we’re currently using an invalid API key.
We can now rotate the secret with the correct API key from Mistral. You can rotate a secret by creating a new secret version with the new secret value. Kong Gateway will fetch the new secret value based on the ttl setting we configured in the Vault entity.
Rotate the secret with the valid Mistral API key:
echo -n "$MISTRAL_API_KEY" | \
gcloud secrets versions add test-secret --data-file=-
Now we can validate that Kong Gateway picks up the valid Mistral API key from the rotated secret. Since Kong Gateway is configured to pick up any rotated secrets every 60 seconds, the following command waits a minute before sending a request:
sleep 60 && curl -X POST "$KONNECT_PROXY_URL/anything" \
--no-progress-meter --fail-with-body \
-H "Accept: application/json"\
-H "Content-Type: application/json" \
--json '{
"messages": [
{
"role": "system",
"content": "You are a mathematician"
},
{
"role": "user",
"content": "What is 1+1?"
}
]
}'
sleep 60 && curl -X POST "http://localhost:8000/anything" \
--no-progress-meter --fail-with-body \
-H "Accept: application/json"\
-H "Content-Type: application/json" \
--json '{
"messages": [
{
"role": "system",
"content": "You are a mathematician"
},
{
"role": "user",
"content": "What is 1+1?"
}
]
}'
You should get a 200 error with an answer to the chat response because Kong Gateway picked up the rotated secret with the valid API key.
curl -Ls https://get.konghq.com/quickstart | bash -s -- -d
If you created a new control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.
How do I fix the Error: could not get value from external vault (no value found (unable to retrieve secret from gcp secret manager (code : 403, status: PERMISSION_DENIED))) error when I try to use my secret from the Google Cloud vault?
Verify that your Google Cloud service account has the Secret Manager Secret Accessor role. This role is required for Kong Gateway to access secrets in the vault.
I’m using Google Workload Identity, how do I configure a Vault?
To use GCP Secret Manager with
Workload Identity
on a GKE cluster, update your pod spec so that the service account (GCP_SERVICE_ACCOUNT) is
attached to the pod. For configuration information, read the Workload
Identity configuration
documentation.
Notes:
- With Workload Identity, setting the
GCP_SERVICE_ACCOUNTisn’t necessary.- When using GCP Vault as a backend, make sure you have configured
systemas part of thelua_ssl_trusted_certificateconfiguration directive so that the SSL certificates used by the official GCP API can be trusted by Kong Gateway.