Gen AI OpenTelemetry metrics reference

AI Gateway can export OpenTelemetry (OTLP) metrics for generative AI, MCP, and A2A traffic through an OpenTelemetry AI Policy. These metrics are aggregated time-series data points (counters, histograms) pushed to a configured OTLP metrics endpoint on a regular interval. They are separate from the per-request Gen AI span attributes emitted on traces.

You can use these metrics to:

• Track LLM request latency and upstream provider processing time
• Monitor token consumption across AI Model Providers, AI Models, and AI Consumers
• Measure time-to-first-token (TTFT) and inter-token latency (TPOT) for streaming responses
• Calculate AI request costs
• Observe MCP tool-call latency, error rates, and ACL decisions
• Monitor A2A agent request volume, duration, and task state transitions

Metrics are natively supported by the OpenTelemetry Policy. To send metrics, set the following config.metrics parameters:

Some metrics have additional requirements:

• gen_ai.server.request.duration and mcp.client.operation.duration require config.metrics.enable_latency_metrics set to true in the OpenTelemetry AI Policy.
• The error.type attribute on duration metrics requires config.metrics.enable_request_metrics set to true in the OpenTelemetry AI Policy.

These metrics follow the OpenTelemetry Gen AI semantic conventions. They capture request duration, upstream latency, token usage, and streaming performance.

Metric reference

gen_ai.client.operation.duration

Total time Kong spends processing a Gen AI operation, such as an LLM request. Requires enable_request_metrics to populate the error.type attribute.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  gen_ai.provider.name	Name of the Gen AI provider.
  gen_ai.request.model	Model name targeted by the request.
  gen_ai.response.model	Model name reported by the provider in the response.
  gen_ai.operation.name	Operation requested, such as chat or embeddings.
  kong.workspace.name	Name of the Workspace.
  kong.auth.consumer.name	Name of the authenticated AI Consumer.
  kong.gen_ai.request.mode	Request mode: oneshot, stream, or realtime.
  error.type	Type of error that occurred.

gen_ai.server.request.duration

Time the LLM provider spends processing the request. Requires enable_latency_metrics set to true. Requires enable_request_metrics to populate the error.type attribute.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  gen_ai.provider.name	Name of the Gen AI provider.
  gen_ai.request.model	Model name targeted by the request.
  gen_ai.response.model	Model name reported by the provider in the response.
  gen_ai.operation.name	Operation requested, such as chat or embeddings.
  kong.workspace.name	Name of the Workspace.
  kong.auth.consumer.name	Name of the authenticated AI Consumer.
  kong.gen_ai.request.mode	Request mode: oneshot, stream, or realtime.
  error.type	Type of error that occurred.

gen_ai.client.token.usage

Number of tokens consumed by the Gen AI operation.

• Instrument unit: {token}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  gen_ai.provider.name	Name of the Gen AI provider.
  gen_ai.request.model	Model name targeted by the request.
  gen_ai.response.model	Model name reported by the provider in the response.
  gen_ai.token.type	Token category: input, output, or total.
  gen_ai.operation.name	Operation requested, such as chat or embeddings.
  kong.workspace.name	Name of the Workspace.
  kong.auth.consumer.name	Name of the authenticated AI Consumer.
  kong.gen_ai.request.mode	Request mode: oneshot, stream, or realtime.

gen_ai.server.time_to_first_token

Time from when the model server receives the request until the first output token is generated.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  gen_ai.provider.name	Name of the Gen AI provider.
  gen_ai.request.model	Model name targeted by the request.
  gen_ai.response.model	Model name reported by the provider in the response.
  gen_ai.operation.name	Operation requested, such as chat or embeddings.
  kong.workspace.name	Name of the Workspace.
  kong.auth.consumer.name	Name of the authenticated AI Consumer.
  kong.gen_ai.request.mode	Request mode: oneshot, stream, or realtime.

gen_ai.server.time_per_output_token

Time between successive output tokens generated by the model server after the first token.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  gen_ai.provider.name	Name of the Gen AI provider.
  gen_ai.request.model	Model name targeted by the request.
  gen_ai.response.model	Model name reported by the provider in the response.
  gen_ai.operation.name	Operation requested, such as chat or embeddings.
  kong.workspace.name	Name of the Workspace.
  kong.auth.consumer.name	Name of the authenticated AI Consumer.
  kong.gen_ai.request.mode	Request mode: oneshot, stream, or realtime.

These metrics use the kong.gen_ai.* namespace and capture Kong-specific AI observability data, including cost tracking, cache and RAG latency, and AWS Guardrails processing time.

To populate kong.gen_ai.llm.cost, define targets[].config.input_cost and targets[].config.output_cost in your AI Model configuration.

kong.gen_ai.llm.cost

Cost of AI requests.

• Instrument unit: {cost}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  gen_ai.provider.name	Name of the Gen AI provider.
  gen_ai.request.model	Model name targeted by the request.
  gen_ai.response.model	Model name reported by the provider in the response.
  gen_ai.operation.name	Operation requested, such as chat or embeddings.
  kong.gen_ai.cache.status	Cache status: hit or empty if not cached.
  kong.gen_ai.vector_db	Vector database used for caching, such as redis.
  kong.gen_ai.embeddings.provider	Embeddings provider used for caching.
  kong.gen_ai.embeddings.model	Embeddings model used for caching.
  kong.workspace.name	Name of the Workspace.
  kong.auth.consumer.name	Name of the authenticated AI Consumer.
  kong.gen_ai.request.mode	Request mode: oneshot, stream, or realtime.

kong.gen_ai.cache.fetch.latency

Time to fetch a response from the semantic cache.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  gen_ai.provider.name	Name of the Gen AI provider.
  gen_ai.request.model	Model name targeted by the request.
  gen_ai.response.model	Model name reported by the provider in the response.
  gen_ai.operation.name	Operation requested, such as chat or embeddings.
  kong.gen_ai.cache.status	Cache status: hit or empty if not cached.
  kong.gen_ai.vector_db	Vector database used for caching, such as redis.
  kong.gen_ai.embeddings.provider	Embeddings provider used for caching.
  kong.gen_ai.embeddings.model	Embeddings model used for caching.
  kong.workspace.name	Name of the Workspace.
  kong.auth.consumer.name	Name of the authenticated AI Consumer.
  kong.gen_ai.request.mode	Request mode: oneshot, stream, or realtime.

kong.gen_ai.cache.embeddings.latency

Time to generate embeddings during cache operations.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  gen_ai.provider.name	Name of the Gen AI provider.
  gen_ai.request.model	Model name targeted by the request.
  gen_ai.response.model	Model name reported by the provider in the response.
  gen_ai.operation.name	Operation requested, such as chat or embeddings.
  kong.gen_ai.cache.status	Cache status: hit or empty if not cached.
  kong.gen_ai.vector_db	Vector database used for caching, such as redis.
  kong.gen_ai.embeddings.provider	Embeddings provider used for caching.
  kong.gen_ai.embeddings.model	Embeddings model used for caching.
  kong.workspace.name	Name of the Workspace.
  kong.auth.consumer.name	Name of the authenticated AI Consumer.
  kong.gen_ai.request.mode	Request mode: oneshot, stream, or realtime.

kong.gen_ai.rag.fetch.latency

Time to fetch data from a RAG source.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  gen_ai.provider.name	Name of the Gen AI provider.
  gen_ai.request.model	Model name targeted by the request.
  gen_ai.response.model	Model name reported by the provider in the response.
  gen_ai.operation.name	Operation requested, such as chat or embeddings.
  kong.gen_ai.cache.status	Cache status: hit or empty if not cached.
  kong.gen_ai.vector_db	Vector database used for caching, such as redis.
  kong.gen_ai.embeddings.provider	Embeddings provider used for caching.
  kong.gen_ai.embeddings.model	Embeddings model used for caching.
  kong.workspace.name	Name of the Workspace.
  kong.auth.consumer.name	Name of the authenticated AI Consumer.
  kong.gen_ai.request.mode	Request mode: oneshot, stream, or realtime.

kong.gen_ai.rag.embeddings.latency

Time to generate embeddings for RAG operations.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  gen_ai.provider.name	Name of the Gen AI provider.
  gen_ai.request.model	Model name targeted by the request.
  gen_ai.response.model	Model name reported by the provider in the response.
  gen_ai.operation.name	Operation requested, such as chat or embeddings.
  kong.gen_ai.cache.status	Cache status: hit or empty if not cached.
  kong.gen_ai.vector_db	Vector database used for caching, such as redis.
  kong.gen_ai.embeddings.provider	Embeddings provider used for caching.
  kong.gen_ai.embeddings.model	Embeddings model used for caching.
  kong.workspace.name	Name of the Workspace.
  kong.auth.consumer.name	Name of the authenticated AI Consumer.
  kong.gen_ai.request.mode	Request mode: oneshot, stream, or realtime.

kong.gen_ai.aws.guardrails.latency

Time for AWS Guardrails to process a request.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  kong.gen_ai.aws.guardrails.id	ID of the AWS Guardrails configuration.
  kong.gen_ai.aws.guardrails.version	Version of the AWS Guardrails configuration.
  kong.gen_ai.aws.guardrails.mode	Mode of the AWS Guardrails evaluation.
  kong.gen_ai.aws.guardrails.region	AWS region of the Guardrails service.
  kong.workspace.name	Name of the Workspace.
  kong.auth.consumer.name	Name of the authenticated AI Consumer.

kong.gen_ai.mcp.response.size

Size of the MCP response body in bytes.

• Instrument unit: By
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  mcp.method.name	MCP method name, such as tools/call.
  gen_ai.tool.name	Name of the MCP tool invoked.

kong.gen_ai.mcp.request.error.count

Number of MCP request errors.

• Instrument unit: {error}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  mcp.method.name	MCP method name, such as tools/call.
  gen_ai.tool.name	Name of the MCP tool invoked.
  error.type	Type of error that occurred.

kong.gen_ai.mcp.acl.allowed

Number of MCP requests allowed by ACL rules.

• Instrument unit: {request}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.mcp.primitive	MCP primitive type, such as tool.
  kong.gen_ai.mcp.primitive_name	Name of the MCP primitive.

kong.gen_ai.mcp.acl.denied

Number of MCP requests denied by ACL rules.

• Instrument unit: {request}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.mcp.primitive	MCP primitive type, such as tool.
  kong.gen_ai.mcp.primitive_name	Name of the MCP primitive.

kong.gen_ai.a2a.request.count

Total number of A2A requests.

• Instrument unit: {request}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.a2a.method	A2A method name.
  kong.gen_ai.a2a.binding	A2A binding type.

kong.gen_ai.a2a.request.duration

Duration of an A2A request in seconds.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.a2a.method	A2A method name.
  kong.gen_ai.a2a.binding	A2A binding type.

kong.gen_ai.a2a.response.size

Size of the A2A response body in bytes.

• Instrument unit: By
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.a2a.method	A2A method name.
  kong.gen_ai.a2a.binding	A2A binding type.

kong.gen_ai.a2a.ttfb

Time to first byte for A2A streaming responses in seconds.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.a2a.method	A2A method name.
  kong.gen_ai.a2a.binding	A2A binding type.

kong.gen_ai.a2a.request.error.count

Number of A2A request errors.

• Instrument unit: {error}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.a2a.method	A2A method name.
  kong.gen_ai.a2a.binding	A2A binding type.
  kong.gen_ai.a2a.error.type	Type of the A2A error.

kong.gen_ai.a2a.task.state.count

Number of A2A task state transitions.

• Instrument unit: {state}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.a2a.task.state	Task state, such as completed, failed, or in_progress.

These metrics provide observability into MCP (Model Context Protocol) server interactions, including latency, response sizes, errors, and ACL decisions.

kong.gen_ai.mcp.response.size

Size of the MCP response body in bytes.

• Instrument unit: By
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  mcp.method.name	MCP method name, such as tools/call.
  gen_ai.tool.name	Name of the MCP tool invoked.

kong.gen_ai.mcp.request.error.count

Number of MCP request errors.

• Instrument unit: {error}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  mcp.method.name	MCP method name, such as tools/call.
  gen_ai.tool.name	Name of the MCP tool invoked.
  error.type	Type of error that occurred.

mcp.client.operation.duration

Duration of the MCP request as observed by the sender. Only available when the MCP entity is in passthrough-listener mode. Requires enable_latency_metrics set to true.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  mcp.method.name	MCP method name, such as tools/call.
  gen_ai.tool.name	Name of the MCP tool invoked.
  error.type	Type of error that occurred.
  gen_ai.operation.name	Operation requested, such as chat or embeddings.

mcp.server.operation.duration

Duration of the MCP request as observed by the receiver.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  mcp.method.name	MCP method name, such as tools/call.
  gen_ai.tool.name	Name of the MCP tool invoked.
  error.type	Type of error that occurred.
  gen_ai.operation.name	Operation requested, such as chat or embeddings.

kong.gen_ai.mcp.acl.allowed

Number of MCP requests allowed by ACL rules.

• Instrument unit: {request}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.mcp.primitive	MCP primitive type, such as tool.
  kong.gen_ai.mcp.primitive_name	Name of the MCP primitive.

kong.gen_ai.mcp.acl.denied

Number of MCP requests denied by ACL rules.

• Instrument unit: {request}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.mcp.primitive	MCP primitive type, such as tool.
  kong.gen_ai.mcp.primitive_name	Name of the MCP primitive.

These metrics provide observability into A2A (Agent-to-Agent) traffic, including request volume, latency, response sizes, and task state transitions.

kong.gen_ai.a2a.request.count

Total number of A2A requests.

• Instrument unit: {request}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.a2a.method	A2A method name.
  kong.gen_ai.a2a.binding	A2A binding type.

kong.gen_ai.a2a.request.duration

Duration of an A2A request in seconds.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.a2a.method	A2A method name.
  kong.gen_ai.a2a.binding	A2A binding type.

kong.gen_ai.a2a.response.size

Size of the A2A response body in bytes.

• Instrument unit: By
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.a2a.method	A2A method name.
  kong.gen_ai.a2a.binding	A2A binding type.

kong.gen_ai.a2a.ttfb

Time to first byte for A2A streaming responses in seconds.

• Instrument unit: s
• Instrument type: Histogram

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.a2a.method	A2A method name.
  kong.gen_ai.a2a.binding	A2A binding type.

kong.gen_ai.a2a.request.error.count

Number of A2A request errors.

• Instrument unit: {error}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.a2a.method	A2A method name.
  kong.gen_ai.a2a.binding	A2A binding type.
  kong.gen_ai.a2a.error.type	Type of the A2A error.

kong.gen_ai.a2a.task.state.count

Number of A2A task state transitions.

• Instrument unit: {state}
• Instrument type: Sum

• Attributes:

  Attribute	Attribute description
  kong.service.name	Name of the Gateway Service.
  kong.route.name	Name of the Route.
  kong.workspace.name	Name of the Workspace.
  kong.gen_ai.a2a.task.state	Task state, such as completed, failed, or in_progress.