AI Gateway Documentation

Overview of AI Gateway capabilities

Launch a AI Gateway control plane and data plane.

Set up your first AI Gateway instance and start proxying requests to LLM providers.

Install AI Gateway 2.x on self-managed Kong Gateway using Docker Compose.

Run a fully-hosted AI Gateway control plane and data planes on Dedicated Cloud Gateways.

Understand the architecture of AI Gateway, including its control plane and data plane, how AI entities translate into data plane configuration, and its deployment and multi-tenancy topologies.

Control who can use models and MCP servers in AI Gateway with users, groups, identity providers, Token Vault, and virtual keys.

Learn about the core capabilities of AI Gateway.

Learn about the various providers supported by AI Gateway.

Browse Policies available for use with AI Gateway.

Understand how to manage and govern AI usage effectively.

Explore how AI Gateway helps enforce data governance Policies.

This page provides an overview of AI Gateway safety, security and compliance features.

Best practices and tools for designing effective prompts.

Implement safeguards to ensure safe and compliant AI outputs.

Customize and transform AI requests with Gateway features.

Learn how AI proxy streaming works.

Proxy AI traffic that AI Gateway can't parse, forwarding unchanged request and response bodies while keeping upstream authentication, rate limiting, and logging.

Review AI Gateway recommended resource allocation sizing guidelines for AI Gateway based on configuration and traffic patterns.

Configure AI Gateway to proxy requests from AI command-line tools to LLM providers.

Route outbound traffic from AI Gateway Policies through a forward proxy to operate in network-isolated environments without breaking load balancing, streaming, WebSocket, or HTTP/2.

This reference explains how AI Gateway uses embedding-based similarity to compare prompts with various inputs, such as cached entries, target model descriptions, document chunks, or allow/deny lists.

Reference for the `condition` field, which lets an AI Policy run only when a CEL expression matches the request.

Changelog for supported Kong AI Gateway versions.

The AI Gateway version support policy outlines the AI Gateway versioning scheme and version lifecycle, from release to sunset support.

Reference for how an AI Gateway control plane determines which data planes are compatible with its configuration. Covers the minimum supported data plane version, feature gating, and automatic raising of the minimum.

This page lists the entities that make up AI Gateway.

AI Model Provider credentials and configuration used by AI Gateway.

AI Models registered with the AI Gateway.

AI Agent entity used by AI Gateway for A2A and HTTP agent configurations.

AI MCP Server entity used by AI Gateway to expose tools and proxy MCP traffic.

AI Policies for AI Gateway.

AI Custom Policies let you run your own Lua plugin logic as an AI Policy type in AI Gateway.

AI Consumers for AI Gateway.

AI Consumer Groups for AI Gateway.

AI Vaults for storing and referencing secrets used by AI Gateway entities.

Configure inbound AI Consumer authentication for AI Models, AI Agents, and AI MCP Servers in AI Gateway.

Client certificates that authorize data planes to connect to an AI Gateway.

AI Management is the Konnect area where admins set up AI Gateway resources and other users request access to them.

Reference for what Org Admins, Team Managers, and Individual Users can do in AI Management. Covers permissions for AI Models, AI MCP Servers, AI Agents, virtual keys, and cost data.

Monitor traffic, latency, errors, and cost for AI Models, AI MCP Servers, and AI Agents.

Reference for supported capabilities for Amazon Bedrock provider

Reference for supported capabilities for Amazon SageMaker provider

Reference for supported capabilities for Anthropic provider

Reference for supported capabilities for Azure OpenAI provider

Reference for supported capabilities and configuration for the Baseten provider

Reference for supported capabilities for Cerebras provider

Reference for supported capabilities for Cohere provider

Reference for supported capabilities for DashScope provider

Reference for supported capabilities for Databricks provider

Reference for supported capabilities for DeepSeek provider

Reference for supported capabilities for the Gemini provider, covering both Gemini Standard and Gemini Enterprise

Reference for supported capabilities for Hugging Face provider

Reference for supported capabilities for Kimi provider

Reference for supported capabilities for Llama provider

Reference for supported capabilities for Mistral provider

Reference for supported capabilities for Ollama provider

Reference for supported capabilities for OpenAI provider

Reference for supported capabilities for TypeSafe AI provider

Reference for supported capabilities for Vercel provider

Reference for supported capabilities for vLLM

Reference for supported capabilities for xAI provider

This page is an introduction to MCP Traffic Gateway capabilities in AI Gateway.

Learn about logs available for MCP traffic via AI Gateway

Learn about metrics available for MCP traffic via AI Gateway

Get started proxying MCP server traffic through AI Gateway.

Learn how AI Gateway provides observability and control for agent-to-agent (A2A) traffic.

Attach a Request Size Limiting AI Policy to an AI Agent entity to reject oversized A2A requests

Attach an OpenTelemetry AI Policy to an AI Agent entity to export A2A traces and metrics to an OTLP collector

Apply rate limits to A2A routes proxied through AI Gateway

Create an AI Agent entity in AI Gateway to proxy Agent-to-Agent (A2A) protocol traffic

Reference an openid-connect AI Auth Strategy backed by Kong Identity on an AI Agent entity to require bearer tokens on A2A traffic

Create a Kong Identity auth server, scope, claim, and client to issue bearer tokens for authenticating AI Agent A2A traffic.

Learn how to use kongctl to create and inspect AI Gateway resources in Kong Konnect.

Learn how to manage Kong Konnect infrastructure as code using declarative YAML configuration.

Learn about AI Gateway logging capabilities.

Explore how to monitor AI LLM metrics in AI Gateway.

Access advanced analytics features in AI Gateway.

Reference for OpenTelemetry span attributes emitted by AI Gateway for generative AI requests, including model parameters, token usage, and tool-call metadata.

Reference for OpenTelemetry metrics emitted by AI Gateway for generative AI, MCP, and A2A traffic.

See where AI Gateway logs are located, the different log levels, and how to configure logs and log levels.

Overview of load balancing and retry and fallback strategies in the AI Proxy Advanced plugin.

This page describes the differences between the AI Gateway running on Kong Gateway plugin model (V1) and the new AI Gateway 2.x Policies model.

Learn how to add users, create teams and groups, and manage access to AI Gateway resources.

Reference for Kong AI Gateway configuration parameters. Set these parameters in kong.conf.

Configure Kong AI Gateway on self-hosted Kong Gateway using the 3.x data model and AI plugins, and map each core entity to its plugin.

This guide walks you through moving your configuration from the AI Gateway running on Kong Gateway plugin model to the new AI Gateway 2.x Policies model.

Migrate an on-prem Kong Gateway 3.x deployment running AI plugins to the AI Gateway 2.x entity and policy model, using the kongctl convert ai-gateway extension and deck ai sync.

Aggregate MCP tools from multiple RESTful APIs into a single MCP endpoint using AI MCP Server entities in conversion-only and listener modes

Learn how to set a Rate Limiting Advanced Policy's rate limit and rate limiting key per request, based on the authenticated Principal's metadata.

Attach a Kafka Consume AI Policy to an AI Agent to read messages from Kafka topics through an HTTP endpoint

Learn how to create an AI MCP Server entity in AI Gateway to restrict access to specific MCP tools based on AI Consumers and AI Consumer Groups. Configure default and per-tool ACLs, define user roles, and validate access behavior with the MCP Inspector CLI.

Attach an AI Auth Strategy backed by Kong Identity, per-model ACLs across a standard and a premium AI Model, and a multi-ceiling AI Rate Limiting Advanced Policy to enforce tiered AI budgets from a claim.

Replace a placeholder API key with a real oauth AI Consumer backed by Kong Identity, so AI Model traffic is authenticated and usage is tracked per consumer

Learn how to create an MCP Server entity in AI Gateway to convert any RESTful API into MCP tools.

Use the Mocking Policy to return example responses from an OpenAPI spec so you can test an AI Model without spending tokens

Learn how to meter LLM traffic from AI Gateway using Konnect Metering & Billing.

Attach an OpenTelemetry Policy to an MCP Server entity to export OTLP metrics for MCP tool traffic to a collector.

Learn how to observe MCP tool activity after you apply access controls. Attach a File Log Policy to an AI MCP Server, generate traffic with the MCP Inspector CLI, then read the audit entries to see which tools each AI Consumer reached and how the ACLs decided each call.

Attach a Kafka Upstream AI Policy to an AI Agent to publish incoming requests to a Kafka topic

Use the AI Prompt Compressor Policy with the LLMLingua compressor service to shrink prompts before they reach the LLM, and combine it with an AI Prompt Template Policy that wraps client input in LLMLINGUA tags so only that input is compressed.

Configure AI Gateway to proxy Claude CLI traffic

Configure AI Gateway to proxy Claude CLI traffic to an AWS Bedrock model

Configure AI Gateway to proxy Claude CLI traffic to a Claude model hosted on Azure AI Foundry

Configure AI Gateway to proxy Claude CLI traffic to an Alibaba Cloud DashScope model.

Configure AI Gateway to proxy Claude CLI traffic to a Gemini model

Configure AI Gateway to proxy Claude CLI traffic to a Hugging Face model

Configure AI Gateway to proxy Claude CLI traffic to an OpenAI model

Configure AI Gateway to proxy Google Gemini CLI traffic to a Gemini model

Configure AI Gateway to proxy OpenAI Codex CLI traffic through the OpenAI Responses API.

Configure an AI Model that sends OpenAI Chat Completions requests to the Amazon Bedrock Mantle endpoint without translating them.

Configure AI Gateway to proxy Qwen Code CLI traffic to an Alibaba Cloud DashScope model

Configure AI Gateway to proxy Qwen Code CLI traffic to an OpenAI model

Create a global Kafka Log AI Policy to publish request and response logs to a Kafka topic

Create a monthly budget for OpenAI spend in AI Cost Management and send email alerts as spend reaches 50, 80, and 100 percent of the limit.

Create a monthly budget for OpenAI spend in AI Cost Management and send alerts to Slack as spend reaches 50, 80, and 100 percent of the limit.

Create a Kong Identity auth server with dynamic claims and per-client labels to drive tiered AI budget enforcement.

Use the AI PII Sanitizer plugin to protect sensitive information in responses.

Use the AI Sanitizer plugin to protect sensitive information in requests.

Use the AI Prompt Guard Policy to filter LLM traffic based on regex rules that allow general IT questions and deny unsafe or off-topic content.

Learn how to use Insomnia 12.1 MCP clients to test MCP tools exposed by AI MCP Proxy in listener mode after they have been aggregated from multiple upstream APIs.

Learn how to use the AI AWS Guardrails Policy.