Secrets management
What is secrets management?
A secret is any sensitive piece of information required for API gateway operations.
Secrets can be information like the following:
- TLS certificates
- Credentials and API keys (to access databases, identity providers, and other services)
- Cryptographic keys (for digital signing and encryption)
By storing these values as secrets, you ensure that they aren’t visible in plain text in logs, the UI, kong.conf, or decK files.
Once a secret is stored, you can use the secret reference instead of the sensitive information. For example, a secret stored in an environment variable called MY_SECRET_POSTGRES_PASSWORD would be referenced with {vault://env/my-secret-postgres-password}.
You can reference secrets in the following Kong Gateway configuration:
- Any field in the
kong.conf - Referenceable fields in Kong plugins
- Configuration associated with APIs serviced by Kong Gateway
How can I manage secrets in Kong Gateway?
Secret management options vary depending on which Kong Gateway tier you have. Kong Gateway OSS users can only manage secrets by storing them in environment variables. Kong Gateway Enterprise users can use environment variables as well as Vaults. The Vaults entity allows you to store and reference secrets from an external, third-party vault or Konnect Config Store.
Environment variables
Store secrets as environment variables and reference them in any field that supports Vault references.
Konnect Config Store
Use Konnect as a Vault backend using a Config Store.
Azure Key Vaults
Connect your Azure Key Vaults to Kong Gateway to reference secrets.
HashiCorp Vault
Connect HashiCorp Vault to Kong Gateway to reference secrets.
AWS Secrets Manager
Connect your AWS Secrets Manager to Kong Gateway to reference secrets.
Google Cloud Platform Secret Manager
Connect Google Cloud Platform Secret Manager to Kong Gateway to reference secrets.
Secrets rotation
Secret rotation is a process that periodically updates secrets. Secret rotation is important for the following reasons:
- Reducing the impact of compromised secrets
- Enhancing resilience against brute-force attacks
- Complying with security regulations
- Maintaining separation of duties
- Adapting to evolving threats
- Mitigating the effects of insider threats
How can I rotate secrets in Kong Gateway?
If you have secrets rotation configured in your third-party vault settings, Kong Gateway can retrieve rotated secrets on a schedule. Kong Gateway can be configured to check for rotated secrets in the following ways:
- Check periodically using TTLs (for example, check for new TLS certificates once per day). This can be configured in the Vault entity configuration.
- Check on failure (for example, on database authentication failure, check if the secrets were updated, and try again). You must write code in Kong Gateway to configure this, so support is limited to PostgreSQL credentials for now.
Frequently Asked Questions
I have a secret with multiple versions, how do I specify an earlier version when I’m referencing the secret?
If you have a secret with multiple versions, you can access the current version or any previous version of the secret by specifying a version in the reference.
In the following AWS example, AWSCURRENT refers to the latest secret version and AWSPREVIOUS refers to an older version:
# For AWSCURRENT, not specifying version
{vault://aws/secret-name/foo}
# For AWSCURRENT, specifying version == 1
{vault://aws/secret-name/foo#1}
# For AWSPREVIOUS, specifying version == 2
{vault://aws/secret-name/foo#2}
This applies to all providers with versioned secrets.
My secret in AWS Secret Manager has a / backslash in the secret name. How do I reference this secret in Kong Gateway?
The slash symbol (/) is a valid character for the secret name in AWS Secrets Manager. If you want to reference a secret name that starts with a slash or has two consecutive slashes, transform one of the slashes in the name into URL-encoded format. For example:
- A secret named
/secret/keyshould be referenced as{vault://aws/%2Fsecret/key} - A secret named
secret/path//aaa/keyshould be referenced as{vault://aws/secret/path/%2Faaa/key}
Since Kong Gateway tries to resolve the secret reference as a valid URL, using a slash instead of a URL-encoded slash will result in unexpected secret name fetching.
I have secrets stored in multiple AWS Secret Manager regions, how do I reference those secrets in Kong Gateway?
You can create multiple Vault entities, one per region with the config.region parameter. You’d then reference the secret by the name of the Vault:
{vault://aws-eu-central-vault/secret-name/foo}
{vault://aws-us-west-vault/secret-name/snip}
I’m using Google Workload Identity, how do I configure a Vault?
To use GCP Secret Manager with
Workload Identity
on a GKE cluster, update your pod spec so that the service account (GCP_SERVICE_ACCOUNT) is
attached to the pod. For configuration information, read the Workload
Identity configuration
documentation.
Notes:
- With Workload Identity, setting the
GCP_SERVICE_ACCOUNTisn’t necessary.- When using GCP Vault as a backend, make sure you have configured
systemas part of thelua_ssl_trusted_certificateconfiguration directive so that the SSL certificates used by the official GCP API can be trusted by Kong Gateway.
How does Kong Gateway retrieve secrets from HashiCorp Vault?
Kong Gateway retrieves secrets from HashiCorp Vault’s HTTP API through a two-step process: authentication and secret retrieval.
Step 1: Authentication
Depending on the authentication method defined in config.auth_method, Kong Gateway authenticates to HashiCorp Vault using one of the following methods:
- If you’re using the
tokenauth method, Kong Gateway uses theconfig.tokenas the client token. - If you’re using the
kubernetesauth method, Kong Gateway uses the service account JWT token mounted in the pod (path defined in theconfig.kube_api_token_file) to call the login API for the Kubernetes auth path on the HashiCorp Vault server and retrieve a client token. -
v3.4+ If you’re using the
approleauth method, Kong Gateway uses the AppRole credentials to retrieve a client token. The AppRole role ID is configured by fieldconfig.approle_role_id, and the secret ID is configured by fieldconfig.approle_secret_idorconfig.approle_secret_id_file.- If you set
config.approle_response_wrappingtotrue, then the secret ID configured byconfig.approle_secret_idorconfig.approle_secret_id_filewill be a response wrapping token, and Kong Gateway will call the unwrap API/v1/sys/wrapping/unwrapto unwrap the response wrapping token to fetch the real secret ID. Kong Gateway will use the AppRole role ID and secret ID to call the login API for the AppRole auth path on the HashiCorp Vault server and retrieve a client token.
- If you set
By calling the login API, Kong Gateway will retrieve a client token and then use it in the next step as the value of X-Vault-Token header to retrieve a secret.
Step 2: Retrieving the secret
Kong Gateway uses the client token retrieved in the authentication step to call the Read Secret API and retrieve the secret value. The request varies depending on the secrets engine version you’re using. Kong Gateway will parse the response of the read secret API automatically and return the secret value.