AI Gateway Documentation

Overview of AI Gateway capabilities

Launch a AI Gateway control plane and data plane.

Set up your first AI Gateway instance and start proxying requests to LLM providers.

Understand the architecture of AI Gateway, including its control plane and data plane, how AI entities translate into data plane configuration, and its deployment and multi-tenancy topologies.

Learn about the core capabilities of AI Gateway.

Learn about the various providers supported by AI Gateway.

Browse Policies available for use with AI Gateway.

Understand how to manage and govern AI usage effectively.

Explore how AI Gateway helps enforce data governance Policies.

This page provides an overview of AI Gateway safety, security and compliance features.

Best practices and tools for designing effective prompts.

Implement safeguards to ensure safe and compliant AI outputs.

Customize and transform AI requests with Gateway features.

Learn how AI proxy streaming works.

Review AI Gateway recommended resource allocation sizing guidelines for AI Gateway based on configuration and traffic patterns.

Configure AI Gateway to proxy requests from AI command-line tools to LLM providers.

Route outbound traffic from AI Gateway Policies through a forward proxy to operate in network-isolated environments without breaking load balancing, streaming, WebSocket, or HTTP/2.

This reference explains how AI Gateway uses embedding-based similarity to compare prompts with various inputs, such as cached entries, target model descriptions, document chunks, or allow/deny lists.

Changelog for supported Kong AI Gateway versions.

The AI Gateway version support policy outlines the AI Gateway versioning scheme and version lifecycle, from release to sunset support.

This page lists the entities that make up AI Gateway.

AI Model Provider credentials and configuration used by AI Gateway.

AI Models registered with the AI Gateway.

AI Agent entity used by AI Gateway for A2A and HTTP agent configurations.

AI MCP Server entity used by AI Gateway to expose tools and proxy MCP traffic.

AI Policies for AI Gateway.

AI Consumers for AI Gateway.

AI Consumer Groups for AI Gateway.

AI Vaults for storing and referencing secrets used by AI Gateway entities.

Configure inbound AI Consumer authentication for AI Models, AI Agents, and AI MCP Servers in AI Gateway.

Client certificates that authorize data planes to connect to an AI Gateway.

Reference for supported capabilities for Amazon Bedrock provider

Reference for supported capabilities for Amazon SageMaker provider

Reference for supported capabilities for Anthropic provider

Reference for supported capabilities for Azure OpenAI provider

Reference for supported capabilities for Cerebras provider

Reference for supported capabilities for Cohere provider

Reference for supported capabilities for DashScope provider

Reference for supported capabilities for Databricks provider

Reference for supported capabilities for DeepSeek provider

Reference for supported capabilities for the Gemini provider, covering both Gemini Standard and Gemini Enterprise

Reference for supported capabilities for Hugging Face provider

Reference for supported capabilities for Kimi provider

Reference for supported capabilities for Llama provider

Reference for supported capabilities for Mistral provider

Reference for supported capabilities for Ollama provider

Reference for supported capabilities for OpenAI provider

Reference for supported capabilities for Vercel provider

Reference for supported capabilities for vLLM

Reference for supported capabilities for xAI provider

This page is an introduction to MCP Traffic Gateway capabilities in AI Gateway.

Learn about logs available for MCP traffic via AI Gateway

Learn about metrics available for MCP traffic via AI Gateway

Get started proxying MCP server traffic through AI Gateway.

Learn how AI Gateway provides observability and control for agent-to-agent (A2A) traffic.

Attach a Request Size Limiting AI Policy to an AI Agent entity to reject oversized A2A requests

Attach an OpenTelemetry AI Policy to an AI Agent entity to export A2A traces and metrics to an OTLP collector

Apply rate limits to A2A routes proxied through AI Gateway

Create an AI Agent entity in AI Gateway to proxy Agent-to-Agent (A2A) protocol traffic

Reference an openid-connect AI Auth Strategy backed by Kong Identity on an AI Agent entity to require bearer tokens on A2A traffic

Create a Kong Identity auth server, scope, claim, and client to issue bearer tokens for authenticating AI Agent A2A traffic.

Learn how to use kongctl to create and inspect AI Gateway resources in Kong Konnect.

Learn how to manage Kong Konnect infrastructure as code using declarative YAML configuration.

Learn about AI Gateway logging capabilities.

Explore how to monitor AI LLM metrics in AI Gateway.

Access advanced analytics features in AI Gateway.

Reference for OpenTelemetry span attributes emitted by AI Gateway for generative AI requests, including model parameters, token usage, and tool-call metadata.

Reference for OpenTelemetry metrics emitted by AI Gateway for generative AI, MCP, and A2A traffic.

See where AI Gateway logs are located, the different log levels, and how to configure logs and log levels.

Overview of load balancing and retry and fallback strategies in the AI Proxy Advanced plugin.

This page describes the differences between the AI Gateway running on Kong Gateway plugin model (V1) and the new AI Gateway 2.x Policies model.

Reference for Kong AI Gateway configuration parameters. Set these parameters in kong.conf.

Configure Kong AI Gateway on self-hosted Kong Gateway using the 3.x data model and AI plugins, and map each core entity to its plugin.

This guide walks you through moving your configuration from the AI Gateway running on Kong Gateway plugin model to the new AI Gateway 2.x Policies model.

Aggregate MCP tools from multiple RESTful APIs into a single MCP endpoint using AI MCP Server entities in conversion-only and listener modes

Attach an AI Auth Strategy backed by Kong Identity, per-model ACLs across a standard and a premium AI Model, and a multi-ceiling AI Rate Limiting Advanced Policy to enforce tiered AI budgets from a claim.

Learn how to create an MCP Server entity in AI Gateway to convert any RESTful API into MCP tools.

Attach an OpenTelemetry Policy to an MCP Server entity to export OTLP metrics for MCP tool traffic to a collector.

Configure AI Gateway to proxy Claude CLI traffic

Configure AI Gateway to proxy Claude CLI traffic to an AWS Bedrock model

Configure AI Gateway to proxy Claude CLI traffic to a Claude model hosted on Azure AI Foundry

Configure AI Gateway to proxy Claude CLI traffic to an Alibaba Cloud DashScope model.

Configure AI Gateway to proxy Claude CLI traffic to a Gemini model

Configure AI Gateway to proxy Claude CLI traffic to a Hugging Face model

Configure AI Gateway to proxy Claude CLI traffic to an OpenAI model

Configure AI Gateway to proxy Google Gemini CLI traffic to a Gemini model

Configure AI Gateway to proxy OpenAI Codex CLI traffic through the OpenAI Responses API.

Configure AI Gateway to proxy Qwen Code CLI traffic to an Alibaba Cloud DashScope model

Configure AI Gateway to proxy Qwen Code CLI traffic to an OpenAI model

Create a Kong Identity auth server with dynamic claims and per-client labels to drive tiered AI budget enforcement.

Use the AI Prompt Guard Policy to filter LLM traffic based on regex rules that allow general IT questions and deny unsafe or off-topic content.

Learn how to use Insomnia 12.1 MCP clients to test MCP tools exposed by AI MCP Proxy in listener mode after they have been aggregated from multiple upstream APIs.

Learn how to use the AI AWS Guardrails Policy.