Before you use Headroom with the AI Prompt Compressor Policy, you need a Headroom instance accessible to your AI Gateway.
You can do this with one of the following:
To configure an AI Prompt Compressor Policy with Headroom as the compressor service, do the following:
curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/policies \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $KONNECT_TOKEN" \
--data '
{
"display_name": "AI Prompt Compressor with Headroom",
"name": "my-ai-prompt-compressor",
"type": "ai-prompt-compressor",
"config": {
"provider": "headroom",
"compressor_url": "http://headroom-service:8787/v1/compress",
"timeout": 45000,
"keepalive_timeout": 60000,
"log_text_data": false,
"stop_on_error": true,
"headroom": {
"proxy_token": "'$HEADROOM_PROXY_TOKEN'",
"ssl_verify": true,
"session_id_headers": [
"x-claude-code-session-id",
"x-claude-code-agent-id",
"thread-id",
"session-id",
"x-session-id"
]
}
}
}
'
ai_gateway_policies:
- ref: my-ai-prompt-compressor
ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
display_name: AI Prompt Compressor with Headroom
name: my-ai-prompt-compressor
type: ai-prompt-compressor
config:
provider: headroom
compressor_url: http://headroom-service:8787/v1/compress
timeout: 45000
keepalive_timeout: 60000
log_text_data: false
stop_on_error: true
headroom:
proxy_token: !env HEADROOM_PROXY_TOKEN
ssl_verify: true
session_id_headers:
- x-claude-code-session-id
- x-claude-code-agent-id
- thread-id
- session-id
- x-session-id
Make sure to replace the following placeholders with your own values:
Set timeout higher than the sum of two Headroom waits.
The first is the per-session turn lock, which makes concurrent turns of one conversation wait for each other (10 seconds).
The second is the compression work itself, which HEADROOM_COMPRESSION_TIMEOUT_SECONDS limits (30 seconds by default).
The default timeout of 10000 milliseconds is lower than that sum, so requests can time out before Headroom finishes.
For more details, see the configuration reference.
The compressor service exposes a /v1/compress endpoint that compresses messages and returns them. This endpoint accepts OpenAI and Anthropic’s message formats. Requests in unsupported formats are forwarded unchanged. You can use this interface to compress prompts or integrate the service with the AI Prompt Compressor Policy.
Headroom uses loopback-trust by default. It answers unauthenticated calls on 127.0.0.1, and returns 404 to non-loopback callers unless it’s started with HEADROOM_COMPRESS_ALLOW_REMOTE=1. You can run Headroom as a co-located sidecar reachable from the AI Gateway data plane without authentication. Alternatively, you can point the Policy at a remote instance and configure a bearer token, sent as both the X-Headroom-Proxy-Token and Authorization: Bearer headers, set with the headroom.proxy_token configuration field.
Configuration validation requires headroom.proxy_token unless compressor_url points to a loopback address (localhost, 127.0.0.0/8, or ::1).
The AI Prompt Compressor Policy uses Headroom in a stateful mode and derives a session identifier for each conversation. Headroom can recognize the turns it has already compressed for that conversation. Recognized turns are replayed unchanged instead of compressed again. This ensures the upstream LLM provider’s cache hits on repeated turns.
The Policy derives the session identifier as follows:
- If any header listed in
config.headroom.session_id_headers is present on the request, the identifier is built from all of the present headers together.
Configuring several headers narrows the session instead of replacing it.
- If none of those headers is present, the identifier is built from the model, the leading system message, and the first user message.
The model name is always part of the identifier.
Every identifier is also scoped to the Policy instance and to the AI Consumer, or to the credential when the route authenticates without resolving a consumer.
We recommend headers whose values stay the same across turns, because a header that changes on every request stops the upstream cache from hitting.
A header that identifies the caller instead of the conversation, such as an installation or machine ID, places all of that caller’s conversations in one session.
Two requests share one session state on the Headroom side when they have the same Policy instance, the same AI Consumer or credential, and the same model.
They must also have the same values for the configured headers or, when no header is present, the same system message and first user message.
If sessions collide, then only one will hit the upstream cache.
- AI Gateway sends the user or agent’s request to the AI Prompt Compressor.
- The AI Prompt Compressor takes the request body, which contains the whole conversation in a
messages or input array.
- The AI Prompt Compressor sends a
POST request to Headroom’s /v1/compress endpoint with the request body, the model name, and a session identifier for the conversation.
The model that AI Gateway selects for the route takes precedence over the model in the request body.
- If Headroom recognizes the session identifier, it replays the turns it already compressed unchanged and compresses only the new messages. Otherwise, it compresses the whole conversation.
- Headroom returns
200 with metadata and the rewritten request body, which includes system and tools.
If the call to Headroom fails, the AI Prompt Compressor rejects the request by default. For details, see Failure behavior.
The following diagram illustrates how the AI Prompt Compressor Policy processes and compresses incoming prompts using Headroom:
sequenceDiagram
actor User as User/Agent
participant KongAICompressor as AI Prompt Compressor Policy
participant Headroom
participant LLM as LLM Provider
User->>KongAICompressor: Sends request
activate KongAICompressor
KongAICompressor->>Headroom: POST /v1/compress with request body and session ID
alt Session recognized
Headroom->>Headroom: Replay cached turns, compress only new messages
else New session
Headroom->>Headroom: Compress whole conversation
end
alt Compression succeeds
Headroom-->>KongAICompressor: Return 200 with rewritten request body and metadata
KongAICompressor->>LLM: Send compressed request to upstream provider
LLM-->>KongAICompressor: Return response
KongAICompressor-->>User: Return response
else Compression fails
Headroom-->>KongAICompressor: Return failure response
KongAICompressor--xUser: Return HTTP 500, request not forwarded
end
deactivate KongAICompressorBy default, if the call to Headroom fails, times out, or returns an error, the Policy returns an HTTP 500 to the client rather than forwarding the request uncompressed.
The Policy retries a 503 response that indicates a compression timeout, for up to three attempts in total, with a wait of 0.2 seconds and then 0.5 seconds between attempts.
Headroom can return this response when a concurrent turn of the same conversation holds the per-session turn lock.
After the last attempt, the Policy returns an HTTP 500.
Every other non-200 response (for example a 400 from a bad configuration, or a 401 from a missing or incorrect bearer token) fails immediately.
A call to Headroom that reuses a pooled connection Headroom has already closed is sent again on a new connection instead of failing the request.
- Deterministic compression requires sessions which is only available in Headroom v0.37.0 or newer. Older images silently ignore the session id and run stateless.
- By default, Headroom keeps sessions in memory in one Headroom process, and Kong must send all turns of a session to that process.
Point each data plane node at its own sidecar, or point all data plane nodes at one shared instance.
Do not put a load balancer in front of Headroom instances that keep sessions in memory.
- To run several Headroom instances behind a load balancer, set
HEADROOM_COMPRESS_SESSION_STORE to a Redis URL (redis://...) on every instance.
The instances then share the session state.
Shared sessions require Headroom v0.42.0 or later, installed with the redis extra.
The default Headroom image doesn’t include the redis extra, so build an image that installs headroom-ai[proxy,redis].
The lock that keeps the turns of one session in order works only inside one instance.
If a client sends turns of one session in parallel, route each session to one instance.
For more information, see the Headroom repository.
- No MCP tool-response compression: only the LLM request path is supported.
- Only requests with a
messages or input conversation array are sent to Headroom. Other formats are forwarded uncompressed.
- A native Bedrock Converse request is skipped, with a warning. Headroom can’t read Converse’s content blocks.
- The Policy forwards a request uncompressed, regardless of
stop_on_error, in the following cases:
- The request body isn’t JSON.
- No model name is found in the request or in the model that AI Gateway selects for the route.
- Compressing a Responses API request (
input) requires a Headroom image with input support.