The Proxy Cache Advanced Policy builds a cache key for each request that matches the policy configuration and looks the key up in the storage backend.
If an entry exists, AI Gateway returns it without calling the upstream.
If not, AI Gateway sends the request upstream and stores the response when it meets the policy configuration.
The policy stores entries for the number of seconds set in config.cache_ttl.
After an entry expires, the next request goes to the upstream and the response replaces the entry.
The Proxy Cache Advanced Policy caches a response when the request method, the response status code, and the response content type all match the policy configuration.
Set these conditions in config.request_method, config.response_code, and config.content_type.
The response content type must match one of the config.content_type values exactly.
For a configuration that caches successful GET responses, see Cache responses to successful GET requests.
The config.strategy setting chooses where AI Gateway stores cache entries:
memory: AI Gateway stores entries in the memory of the data plane node.
redis: AI Gateway stores each entry as a hash in a Redis database.
Set the connection in config.redis, for example config.redis.host and config.redis.port.
The Redis entry expires after config.cache_ttl seconds, or after config.storage_ttl seconds when you set it.
For a Redis configuration, see Cache responses with Redis.
If Redis fails, for example when the database is down, the result depends on config.bypass_on_err:
The config.storage_ttl setting keeps entries in the Redis database for longer than config.cache_ttl.
When an entry is older than config.cache_ttl and config.cache_control is false, the next request goes to the upstream.
AI Gateway replaces the entry with the response and sets the X-Cache-Status header to Refresh.
AI Gateway builds the cache key from the following parts of the request:
AI Gateway returns the key of each request in the X-Cache-Key response header.
AI Gateway reports the cache result in the X-Cache-Status response header:
Miss: The request matches the policy configuration, but an entry doesn’t exist. AI Gateway sent the request upstream and stored the response.
Hit: AI Gateway served the response from the cache. The age header shows how many seconds ago the entry was stored.
Refresh: The entry was older than config.cache_ttl. AI Gateway sent the request upstream and replaced the entry.
Bypass: The policy didn’t use the cache, for example because the request method, response status code, or response content type isn’t in the policy configuration.