Dashboards that show traffic and latency per entity help you find slow upstreams, error spikes, and unexpected usage before users report them. The StatsD Policy is an AI Policy that sends request metrics from AI Gateway, such as request count, request and response size, and latency, to a StatsD server. The Policy sends the metrics over UDP by default, or over TCP, so the server must be reachable from the data plane at the address set in config.host and config.port.

AI Consumer Group scope: AI Gateway accepts a StatsD Policy attached to an AI Consumer Group, but the data plane fails to apply the change. The data plane stays on its previous configuration until you detach the Policy from the AI Consumer Group.

Metrics

When you don’t set config.metrics, the StatsD Policy sends the following metrics for each request. Each metric name starts with the configured prefix. In the names, <entity> is the name of the AI Agent or AI MCP Server that handled the request. For AI Models, see Limitations.

Metric

Name

Description

Default type

request_count kong.service.<entity>.request.count Counts the request. counter
latency kong.service.<entity>.latency Tracks the time in milliseconds between the request and the response. timer
request_size kong.service.<entity>.request.size Tracks the size of the request in bytes. counter
response_size kong.service.<entity>.response.size Tracks the size of the response in bytes. counter
status_count kong.service.<entity>.status.<status> Counts the responses for each HTTP status code. counter
upstream_latency kong.service.<entity>.upstream_latency Tracks the time in milliseconds the upstream took to process the request. timer
kong_latency kong.service.<entity>.kong_latency Tracks the time in milliseconds AI Gateway spent processing the request itself. timer
unique_users kong.service.<entity>.user.uniques Tracks the unique AI Consumers that made requests. set
request_per_user kong.service.<entity>.user.<consumer_identifier>.request.count Counts the requests for each AI Consumer. counter
status_count_per_user kong.service.<entity>.user.<consumer_identifier>.status.<status> Counts the responses for each HTTP status code and AI Consumer. counter
status_count_per_workspace kong.service.<entity>.workspace.<workspace_id>.status.<status> Counts the responses for each HTTP status code, with a workspace ID in the name. counter
shdict_usage kong.node.<node_hostname>.shdict.<dictionary>.free_space
kong.node.<node_hostname>.shdict.<dictionary>.capacity
Tracks the free space and capacity of each shared memory dictionary on the data plane node, about once a minute. gauge

When you set config.metrics, the Policy sends only the metrics in the list.

Metric types

The stat_type field of each metric in config.metrics sets the metric type. The Policy sends each type to the StatsD server as a type code:

Type

Code

counter c
timer ms
gauge g
histogram h
meter m
set s

Metrics of type counter or gauge require a sample_rate. AI Gateway rejects the Policy when a counter or gauge metric doesn’t set one. The Policy reports every request and appends the rate to the metric, for example kong.service.<entity>.request.count:1|c|@0.5, so your StatsD server can scale the counts.

For unique_users and status_count, the Policy ignores stat_type and sends a set and a counter.

Per-consumer metrics

The request_per_user, status_count_per_user, and unique_users metrics report a value for each AI Consumer. For an authenticated request, the Policy doesn’t send a per-consumer metric without a consumer_identifier. Since api-key AI Consumers don’t have a custom ID, you’ll need to configure this identifier. oauth AI Consumers set their own custom_id, for example to the sub claim. For requests that authentication rejects, the default metrics include per-consumer metrics with the user name anonymous.

For a configuration, see Count requests per AI Consumer.

Tags

By default, the Policy puts the entity in the metric name. Set config.tag_style to send the metrics with tags instead. The metric names then drop the entity, for example kong.request.count, and each metric carries the node, and status tags, plus a consumer tag when the request is authenticated. The shdict_usage metric carries only the node and shdict tags. The route tag holds a route name that AI Gateway generated for the entity, and service holds the entity name.

The Policy supports four tag formats:

tag_style

Example

dogstatsd kong.request.count:1|c|#route:<route>,status:200
influxdb kong.request.count,route=<route>,status=200:1|c
librato kong.request.count#route=<route>,status=200:1|c
signalfx kong.request.count[route=<route>,status=200]:1|c

Choose the format that matches your backend, for example dogstatsd for Datadog, influxdb for InfluxDB, librato for Librato, and signalfx for SignalFx. When you set tag_style, the Policy doesn’t send the status_count metric. For a configuration, see Send metrics with DogStatsD tags.

Metric names

Use the following settings to change what the metric names contain after the prefix. service_identifier is a field of each metric in config.metrics. Set config.service_identifier_default to apply one value to every metric.

Setting

Effect on the metric name

config.hostname_in_prefix: true Adds node.<hostname> after the prefix, for example kong.node.<hostname>.service.<entity>.request.count.
service_identifier: service_name Uses the entity name as <entity>.
service_identifier: service_id Uses an ID that AI Gateway generates for the entity.
service_identifier: service_host Uses the host of the upstream URL, with each dot replaced by an underscore.
service_identifier: service_name_or_host Uses the entity name. This is the default.

Which requests the Policy reports

The StatsD Policy reports every request sent to the entity it’s attached to, including requests that authentication rejects. For example, the Policy reports a request that authentication rejects with the 401 status code.

To report only some responses, set config.allow_status_codes to a list of status code ranges, for example 200-205 and 400-499. The Policy then skips every response with another status code. For a configuration, see Report requests in a status range.

Packet size and transport

By default, the Policy sends each metric in its own UDP datagram. Set config.udp_packet_size to the largest datagram size, in bytes, that the Policy may fill with several metrics. The Policy separates the metrics in a datagram with newlines. The value can’t be larger than 65507, so consider the MTU of your network when you set it.

Set config.use_tcp to true to send the metrics over TCP.

Queuing

The StatsD Policy uses internal queues to decouple the production of log entries from their transmission to the upstream server.

With queuing, entries are placed in a configurable queue before being sent in batches to the upstream server. This has the following benefits:

  • Reduces concurrency on the upstream server
  • Helps deal with temporary outages of the upstream server due to network or administrative changes
  • Can reduce resource usage both in AI Gateway and on the upstream server by collecting multiple entries from the queue in one request

Note: Because queues are structural elements for components in AI Gateway, they only live in the main memory of each worker process and aren’t shared between workers. Therefore, queued content isn’t preserved under abnormal operational situations, like power loss or unexpected worker process shutdown due to memory shortage or program errors.

You can configure several parameters for queuing:

Parameters

Description

Queue capacity limits:

config.queue.max_entries
config.queue.max_bytes
config.queue.max_batch_size
Configure sizes for various aspects of the queue: maximum number of entries, batch size, and queue size in bytes.

When a queue reaches the maximum number of entries and another entry is enqueued, the oldest entry in the queue is deleted to make space for the new entry. The queue code logs a warning when it reaches a capacity threshold of 80% and when it starts to delete entries from the queue, and logs again when the situation normalizes.
Timer usage:

config.queue.concurrency_limit
Only one timer is used to start queue processing in the background by default. Once the queue is empty, the timer handler terminates, and a new timer is created as soon as a new entry is pushed onto the queue.
Retry logic:

config.queue.initial_retry_delay
config.queue.max_coalescing_delay
config.queue.max_retry_delay
config.queue.max_retry_time
If a queue fails to process, it can automatically retry if the failure is temporary, for example due to network problems or upstream unavailability.

The Policy sends the metrics of a request up to config.queue.max_coalescing_delay seconds late. Keep this value low for near-real-time dashboards.

Scopes

You can apply the StatsD Policy globally, or attach it to an AI Model, an AI Agent, an AI MCP Server, or an AI Consumer. For details about how scopes work, see AI Policy scopes.

  • At AI Agent and AI MCP Server scope, the Policy reports requests to that entity.
  • At AI Model scope, the Policy reports requests to that AI Model. See Limitations for the metric names.
  • At AI Consumer scope, the Policy reports only requests authenticated as that AI Consumer.
  • At global scope, the Policy reports requests to every entity on the gateway. To report only some traffic, set a condition on the Policy. For example, the following condition reports only requests that include the header x-rt: on:

    condition: 'http.headers.x_rt == "on"'

Use AI Consumer scope to count requests per caller, and use global scope when every entity needs the same metrics.

Limitations

The StatsD Policy has the following limitations:

  • AI Model names: At AI Model scope, and at global scope for AI Model traffic, every metric name contains ai-gateway as the entity, regardless of which AI Model handled the request. As a result, you can’t tell AI Models apart in your StatsD server by the entity part of the name.

Looking for on-prem configuration? Convert the kongctl configuration to a decK yaml file and use deck file ai2kong. See Configure Kong AI Gateway on-prem for more information.

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!