Configure Google Cloud Secret Manager as a vault backend

Uses: Kong Gateway decK

Deployment Platform:

Prerequisites

Google Cloud configuration

To add Secret Manager as a Vault backend to Kong Gateway, you must configure the following:

In the Google Cloud console, create a project and name it test-gateway-vault.
On the Secret Manager page, create a secret called test-secret with the following JSON content:
```
 secret
```
Create a service account key and grant IAM permissions:
1. In the Service Account settings, click the test-gateway-vault project and then click the email address of the service account that you want to create a key for.
2. From the Keys tab, create a new key from the add key menu and select JSON for the key type.
3. Save the JSON file you downloaded.
4. From the IAM & Admin settings, click the edit icon next to the service account to grant access to the Secret Manager Secret Accessor role for your service account. icon_url: /assets/icons/google-cloud.svg

Set the environment variables needed to authenticate to Google Cloud:

export GCP_SERVICE_ACCOUNT=$(cat /path/to/file/service-account.json | jq -c)
export KONG_LUA_SSL_TRUSTED_CERTIFICATE='system'

Note that these variables need to be passed when creating your Data Plane container.

Kong Konnect

This is a Konnect tutorial. If you don’t have a Konnect account, you can get started quickly with our onboarding wizard.

The following Konnect items are required to complete this tutorial:
- Personal access token (PAT): Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
- Control Plane Name: You can use an existing Control Plane or create a new one to use for this tutorial.
- Konnect Proxy URL: By default, a self-hosted Data Plane uses http://localhost:8000. You can set up Data Plane nodes for your Control Plane from the Gateway Manager in Konnect.

Set the personal access token, the Control Plane name, the Control Plane URL, and the Konnect proxy URL as environment variables:

 export DECK_KONNECT_TOKEN='YOUR KONNECT TOKEN'
 export DECK_KONNECT_CONTROL_PLANE_NAME='YOUR CONTROL PLANE NAME'
 export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
 export KONNECT_PROXY_URL='KONNECT PROXY URL'

Kong Gateway running

This tutorial requires Kong Gateway Enterprise. If you don’t have Kong Gateway set up yet, you can use the quickstart script with an enterprise license to get an instance of Kong Gateway running almost instantly.

Export your license to an environment variable:

 export KONG_LICENSE_DATA='LICENSE-CONTENTS-GO-HERE'

Run the quickstart script:

 curl -Ls https://get.konghq.com/quickstart | bash -s -- -e KONG_LICENSE_DATA \
      -e GCP_SERVICE_ACCOUNT \
      -e KONG_LUA_SSL_TRUSTED_CERTIFICATE

Once Kong Gateway is ready, you will see the following message:

 Kong Gateway Ready

decK

decK is a CLI tool for managing Kong Gateway declaratively with state files. To complete this tutorial you will first need to install decK.

Configure Secret Manager as a vault with the Vault entity

To enable Secret Manager as your vault in Kong Gateway, you can use the Vault entity.

echo '
_format_version: "3.0"
vaults:
  - name: gcp
    description: Stored secrets in Secret Manager
    prefix: gcp-sm-vault
    config:
      project_id: test-gateway-vault
' | deck gateway apply -

Validate

To validate that the secret was stored correctly in Google Cloud, you can call a secret from your vault using the kong vault get command within the Data Plane container.

 kong vault get {vault://gcp-sm-vault/test-secret}

 kong vault get {vault://gcp-sm-vault/test-secret}

If the vault was configured correctly, this command should return the value of the secret. You can use {vault://gcp-sm-vault/test-secret} to reference the secret in any referenceable field.

Cleanup

Cleanup Google Cloud Resources

If you created new Google Cloud resources for this tutorial, make sure to delete them to avoid unnecessary charges.

Destroy the Kong Gateway container

curl -Ls https://get.konghq.com/quickstart | bash -s -- -d

Clean up Konnect environment

If you created a new control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.

FAQs

How do I fix the Error: could not get value from external vault (no value found (unable to retrieve secret from gcp secret manager (code : 403, status: PERMISSION_DENIED))) error when I try to use my secret from the Google Cloud vault?

Verify that your Google Cloud service account has the Secret Manager Secret Accessor role. This role is required for Kong Gateway to access secrets in the vault.

How do I rotate my secrets in Google Cloud and how does Kong Gateway pick up the new secret values?

You can rotate your secret in Google Cloud by creating a new secret version with the updated value. You’ll also want to configure the ttl settings in your Kong Gateway Vault entity so that Kong Gateway pulls the rotated secret periodically. For more information, see Store and rotate Mistral API keys as secrets in Google Cloud with Kong Gateway and the AI Proxy plugin.

I’m using Google Workload Identity, how do I configure a Vault?

To use GCP Secret Manager with Workload Identity on a GKE cluster, update your pod spec so that the service account (GCP_SERVICE_ACCOUNT) is attached to the pod. For configuration information, read the Workload Identity configuration documentation.

Notes:

With Workload Identity, setting the GCP_SERVICE_ACCOUNT isn’t necessary.

When using GCP Vault as a backend, make sure you have configured system as part of the lua_ssl_trusted_certificate configuration directive so that the SSL certificates used by the official GCP API can be trusted by Kong Gateway.

Next Steps

Review the Vaults entity

Rotate secrets in Google Cloud Secret with Kong Gateway