Anthropic provider

You can proxy requests to Anthropic AI models through AI Gateway using the AI Proxy and AI Proxy Advanced plugins. This reference documents all supported AI capabilities, configuration requirements, and provider-specific details needed for proper integration.

AI Gateway automatically routes requests to the appropriate Anthropic API endpoints. The following table shows the upstream paths used for each capability.

The following tables show the AI capabilities supported by Anthropic provider when used with the AI Proxy or the AI Proxy Advanced plugin.

Set the plugin’s route_type based on the capability you want to use. See the tables below for supported route types.

Text generation

Support for Anthropic basic text generation capabilities including chat, completions, and embeddings:

Advanced text generation

Support for Anthropic function calling to allow Anthropic models to use external tools and APIs:

Processing

Support for Anthropic file operations, batch operations, assistants, and response handling:

¹ Batches processing for Anthropic is supported in the native format from SDK only

The base URL is https://api.anthropic.com:443/{route_type_path}, where {route_type_path} is determined by the capability.

AI Gateway uses this URL automatically. You only need to configure a URL if you’re using a self-hosted or Anthropic-compatible endpoint, in which case set the upstream_url plugin option.

By default, the AI Proxy plugin uses OpenAI-compatible request formats. Set config.llm_format to a native format to use Anthropic-specific APIs and features.

The following native Anthropic APIs are supported:

Provider-specific limitations for native formats

• Does not support llm/v1/completions or llm/v1/embeddings

Statistics logging limitations for native formats

• No statistics logging for llm/v1/completions

To use Anthropic with AI Gateway, configure the AI Proxy or AI Proxy Advanced.

Here’s a minimal configuration for chat completions:

_format_version: "3.0"
plugins:
  - name: ai-proxy
    config:
      route_type: llm/v1/chat
      auth:
        header_name: x-api-key
        header_value: "${key}"
      model:
        provider: anthropic
        name: claude-3-5-sonnet-latest
        options:
          anthropic_version: '2023-06-01'
          max_tokens: 512
          temperature: 1.0

curl -i -X POST http://localhost:8001/plugins/ \
    --header "Accept: application/json" \
    --header "Content-Type: application/json" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "route_type": "llm/v1/chat",
        "auth": {
          "header_name": "x-api-key",
          "header_value": "${key}"
        },
        "model": {
          "provider": "anthropic",
          "name": "claude-3-5-sonnet-latest",
          "options": {
            "anthropic_version": "2023-06-01",
            "max_tokens": 512,
            "temperature": 1.0
          }
        }
      }
    }
    '

curl -X POST https://{region}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/plugins/ \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "ai-proxy",
      "config": {
        "route_type": "llm/v1/chat",
        "auth": {
          "header_name": "x-api-key",
          "header_value": "${key}"
        },
        "model": {
          "provider": "anthropic",
          "name": "claude-3-5-sonnet-latest",
          "options": {
            "anthropic_version": "2023-06-01",
            "max_tokens": 512,
            "temperature": 1.0
          }
        }
      }
    }
    '

echo "
apiVersion: configuration.konghq.com/v1
kind: KongClusterPlugin
metadata:
  name: ai-proxy
  namespace: kong
  annotations:
    kubernetes.io/ingress.class: kong
  labels:
    global: 'true'
config:
  route_type: llm/v1/chat
  auth:
    header_name: x-api-key
    header_value: '${key}'
  model:
    provider: anthropic
    name: claude-3-5-sonnet-latest
    options:
      anthropic_version: '2023-06-01'
      max_tokens: 512
      temperature: 1.0
plugin: ai-proxy
" | kubectl apply -f -

resource "konnect_gateway_plugin_ai_proxy" "my_ai_proxy" {
  enabled = true

  config = {
    route_type = "llm/v1/chat"

    auth = {
      header_name = "x-api-key"
      header_value = "${key}"
    }

    model = {
      provider = "anthropic"
      name = "claude-3-5-sonnet-latest"

      options = {
        anthropic_version = "2023-06-01"
        max_tokens = 512
        temperature = 1.0
      }
    }
  }

  control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
}

For more configuration options and examples, see:

• AI Proxy examples
• AI Proxy Advanced examples

• Set up AI Proxy Advanced with Anthropic in Kong Gateway View →
• Set up AI Proxy with Anthropic in Kong Gateway View →
• Use the AI Lakera Guard plugin View →
• Route Claude CLI traffic through AI Gateway and Anthropic View →