Store and rotate Mistral API keys as secrets in Google Cloud

Uses: Kong Gateway AI Gateway decK

Deployment Platform:

Prerequisites

Google Cloud Secret Manager

To add Secret Manager as a Vault backend to Kong Gateway, you must create a project, service account key, and grant IAM permissions. This tutorial also uses gcloud, so you need to install and configure that.

In the Google Cloud console, create a project and name it test-gateway-vault.
In the Service Account settings, click the test-gateway-vault project and then click the email address of the service account that you want to create a key for.
From the Keys tab, create a new key from the add key menu and select JSON for the key type.
Save the JSON file you downloaded.
From the IAM & Admin settings, click the edit icon next to the service account to grant access to the Secret Manager Secret Accessor role for your service account.
Install gcloud.
Authenticate with gcloud and set your project to test-gateway-vault:
```
gcloud auth login
gcloud config set project test-gateway-vault
```

Mistral AI API key

In this tutorial, you’ll be storing your Mistral AI API key as a secret in a Konnect Vault.

In the Mistral AI console, create an API key and copy it. You’ll add this API key as a secret to your vault.

Environment variables

Set the environment variables needed to authenticate to Google Cloud:

export GCP_SERVICE_ACCOUNT=$(cat /path/to/file/service-account.json)
export MISTRAL_API_KEY="Bearer <Mistral-API-key>"

Note that the GCP_SERVICE_ACCOUNT variables must be passed when creating your data plane container.

Kong Konnect

This is a Konnect tutorial. If you don’t have a Konnect account, you can get started quickly with our onboarding wizard.

The following Konnect items are required to complete this tutorial:
- Personal access token (PAT): Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
- Control Plane Name: You can use an existing Control Plane or create a new one to use for this tutorial.
- Konnect Proxy URL: By default, a self-hosted Data Plane uses http://localhost:8000. You can set up Data Plane nodes for your Control Plane from the Gateway Manager in Konnect.

Set the personal access token, the Control Plane name, the Control Plane URL, and the Konnect proxy URL as environment variables:

 export DECK_KONNECT_TOKEN='YOUR KONNECT TOKEN'
 export DECK_KONNECT_CONTROL_PLANE_NAME='YOUR CONTROL PLANE NAME'
 export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
 export KONNECT_PROXY_URL='KONNECT PROXY URL'

Kong Gateway running

This tutorial requires Kong Gateway Enterprise. If you don’t have Kong Gateway set up yet, you can use the quickstart script with an enterprise license to get an instance of Kong Gateway running almost instantly.

Export your license to an environment variable:

 export KONG_LICENSE_DATA='LICENSE-CONTENTS-GO-HERE'

Run the quickstart script:

 curl -Ls https://get.konghq.com/quickstart | bash -s -- -e KONG_LICENSE_DATA \
      -e GCP_SERVICE_ACCOUNT

Once Kong Gateway is ready, you will see the following message:

 Kong Gateway Ready

decK

decK is a CLI tool for managing Kong Gateway declaratively with state files. To complete this tutorial you will first need to install decK.

Required entities

For this tutorial, you’ll need Kong Gateway entities, like Gateway Services and Routes, pre-configured. These entities are essential for Kong Gateway to function but installing them isn’t the focus of this guide. Follow these steps to pre-configure them:

Run the following command:

echo '
_format_version: "3.0"
services:
  - name: example-service
    url: http://httpbin.konghq.com/anything
routes:
  - name: example-route
    paths:
    - "/anything"
    service:
      name: example-service
' | deck gateway apply -

To learn more about entities, you can read our entities documentation.

Add an invalid API key as a secret in Google Cloud Secret Manager

In this tutorial, first we’ll create a secret with an invalid API key in Google Cloud Secret Manager. Later, we’ll add the correct API key as another secret version, but this allows us to test if Kong Gateway picks up the rotated secret correctly.

Create a secret called test-secret and then create a new secret version with the secret value of Bearer invalid:

gcloud secrets create test-secret \
    --replication-policy="automatic"

echo -n "Bearer invalid" | \
      gcloud secrets versions add test-secret --data-file=-

The first command is supported on Linux, macOS, and Cloud Shell. For other distributions, see Create a secret in Google Cloud documentation.

Configure Secret Manager as a vault with the Vault entity

To enable Secret Manager as your vault in Kong Gateway, you can use the Vault entity.

In this tutorial, we are configuring the time-to-live (ttl) as 60 seconds/1 minute. This tells Kong Gateway to check every minute with Google Cloud to get the rotated secret. We’ve configured a low value so that we can quickly validate that the secret rotation is functioning as expected.

echo '
_format_version: "3.0"
vaults:
  - name: gcp
    description: Stored secrets in Secret Manager
    prefix: gcp-sm-vault
    config:
      project_id: test-gateway-vault
      ttl: 60
' | deck gateway apply -

Enable the AI Proxy plugin

In this tutorial, you’ll use the Mistral API key you stored as a secret to generate an answer to a question using the AI Proxy plugin.

echo '
_format_version: "3.0"
plugins:
  - name: ai-proxy
    route: example-route
    config:
      route_type: llm/v1/chat
      auth:
        header_name: Authorization
        header_value: "{vault://gcp-sm-vault/test-secret}"
      model:
        provider: mistral
        name: mistral-tiny
        options:
          mistral_format: openai
          upstream_url: https://api.mistral.ai/v1/chat/completions
' | deck gateway apply -

Validate that Kong Gateway uses the invalid API key from the secret

First, let’s validate that the secret was stored correctly in Google Cloud by calling a secret from your vault using the kong vault get command within the Data Plane container.

 kong vault get {vault://gcp-sm-vault/test-secret}

 kong vault get {vault://gcp-sm-vault/test-secret}

If the vault was configured correctly, this command should return Bearer invalid.

Now, let’s validate that when we make a call to the Route associated with the AI Proxy plugin, that it is using this invalid API key stored in our secret:

curl -X POST "$KONNECT_PROXY_URL/anything" \
     -H "Accept: application/json"\
     -H "Content-Type: application/json" \
     --json '{
       "messages": [
         {
           "role": "system",
           "content": "You are a mathematician"
         },
         {
           "role": "user",
           "content": "What is 1+1?"
         }
       ]
     }'

You should see the following response:

Unauthorized

curl -X POST "http://localhost:8000/anything" \
     -H "Accept: application/json"\
     -H "Content-Type: application/json" \
     --json '{
       "messages": [
         {
           "role": "system",
           "content": "You are a mathematician"
         },
         {
           "role": "user",
           "content": "What is 1+1?"
         }
       ]
     }'

You should see the following response:

Unauthorized

You should get a 401 error with the message Unauthorized because we’re currently using an invalid API key.

Rotate the secret in Secret Manager

We can now rotate the secret with the correct API key from Mistral. You can rotate a secret by creating a new secret version with the new secret value. Kong Gateway will fetch the new secret value based on the ttl setting we configured in the Vault entity.

Rotate the secret with the valid Mistral API key:

echo -n "$MISTRAL_API_KEY" | \
    gcloud secrets versions add test-secret --data-file=-

Validate that Kong Gateway uses the valid API key from the rotated secret

Now we can validate that Kong Gateway picks up the valid Mistral API key from the rotated secret. Since Kong Gateway is configured to pick up any rotated secrets every 60 seconds, the following command waits a minute before sending a request:

sleep 60 && curl -X POST "$KONNECT_PROXY_URL/anything" \
     -H "Accept: application/json"\
     -H "Content-Type: application/json" \
     --json '{
       "messages": [
         {
           "role": "system",
           "content": "You are a mathematician"
         },
         {
           "role": "user",
           "content": "What is 1+1?"
         }
       ]
     }'

sleep 60 && curl -X POST "http://localhost:8000/anything" \
     -H "Accept: application/json"\
     -H "Content-Type: application/json" \
     --json '{
       "messages": [
         {
           "role": "system",
           "content": "You are a mathematician"
         },
         {
           "role": "user",
           "content": "What is 1+1?"
         }
       ]
     }'

You should get a 200 error with an answer to the chat response because Kong Gateway picked up the rotated secret with the valid API key.

Cleanup

Destroy the Kong Gateway container

curl -Ls https://get.konghq.com/quickstart | bash -s -- -d

Clean up Konnect environment

If you created a new control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.

FAQs

How do I fix the Error: could not get value from external vault (no value found (unable to retrieve secret from gcp secret manager (code : 403, status: PERMISSION_DENIED))) error when I try to use my secret from the Google Cloud vault?

Verify that your Google Cloud service account has the Secret Manager Secret Accessor role. This role is required for Kong Gateway to access secrets in the vault.

I’m using Google Workload Identity, how do I configure a Vault?

To use GCP Secret Manager with Workload Identity on a GKE cluster, update your pod spec so that the service account (GCP_SERVICE_ACCOUNT) is attached to the pod. For configuration information, read the Workload Identity configuration documentation.

Notes:

With Workload Identity, setting the GCP_SERVICE_ACCOUNT isn’t necessary.

When using GCP Vault as a backend, make sure you have configured system as part of the lua_ssl_trusted_certificate configuration directive so that the SSL certificates used by the official GCP API can be trusted by Kong Gateway.