When a request matches the policy, AI Gateway builds a cache key and looks it up in memory.
If an entry exists, AI Gateway returns it without calling the upstream.
If not, AI Gateway sends the request upstream and stores the response when it meets the policy configuration.
The policy stores entries for the number of seconds set in config.cache_ttl.
After an entry expires, the next request goes to the upstream and the response replaces the entry.
By default, the policy caches responses that meet all of the following conditions:
The content type must match one of the listed values exactly.
To cache a streamed response, add text/event-stream to config.content_type.
Requests to AI MCP Servers use POST.
To cache them, add POST to config.request_method.
Because the cache key excludes the request body, add POST only for endpoints where the body doesn’t change the response.
See Limitations.
AI Gateway builds the cache key from the following parts of the request:
Use these settings to control which requests share a cache entry:
- By default, all query parameters are part of the key.
Set
config.vary_query_params to use only the listed parameters.
- By default, the path is case-sensitive.
Set
config.ignore_uri_case to true to treat /Items and /items as the same path.
- By default, no request headers are part of the key.
List the headers that change the response in
config.vary_headers.
AI Gateway returns the key of each request in the X-Cache-Key response header.
AI Gateway reports the cache result in the X-Cache-Status response header:
Miss: The request matches the policy configuration, but no entry exists. AI Gateway sent the request upstream and stored the response.
Hit: AI Gateway served the response from the cache. The age header shows how many seconds ago the entry was stored.
Bypass: The policy didn’t use the cache, for example because the request method, response status code, or response content type isn’t in the policy configuration.
To remove a header from responses, set X-Cache-Key, X-Cache-Status, or age in config.response_headers to false.