Amazon Bedrock provider

You can proxy requests to Amazon Bedrock AI models through AI Gateway by creating AI Model Provider and AI Model entities. This reference documents all supported AI capabilities, configuration requirements, and provider-specific details needed for proper integration.

AI Gateway supports both Bedrock Runtime and Bedrock Mantle. The following sections cover both variants together and note where their capabilities differ.

AI Gateway automatically routes requests to the appropriate Amazon Bedrock API endpoints. The following table shows the upstream paths used for each capability.

The following tables show the AI capabilities supported by the Amazon Bedrock provider when configuring AI Models.

By default, AI Gateway uses the path templates shown in the tables below (e.g., /chat/completions, /embeddings, etc.). To customize these paths, configure the config.paths field in your AI Model entity. Custom paths take the form {configured_path}/{template_path} — for example, if you set a custom path of /v2, requests to /embeddings would be routed to /v2/embeddings.

Text generation

Support for Amazon Bedrock text generation capabilities:

Embeddings

Support for Amazon Bedrock embeddings generation:

Agentic

Support for Amazon Bedrock agent and assistant capabilities:

Image

Support for Amazon Bedrock image generation and editing capabilities:

For requests with large payloads, consider increasing config.max_request_body_size on your AI Model entity to three times the raw binary size.

Supported image sizes and formats vary by model. Refer to your provider’s documentation for allowed dimensions and requirements.

Video

Support for Amazon Bedrock video generation capabilities:

For requests with large payloads (video generation), consider increasing config.max_request_body_size on your AI Model entity to three times the raw binary size.

Batches

Support for Amazon Bedrock batch processing capabilities:

¹ Bedrock Runtime: Batches processing for Bedrock is supported in the native format from SDK only

Batch jobs run on an AI Model with type: "api". Regular models handle synchronous capabilities like generate and embeddings, so a model that serves batches can’t serve those capabilities.

Files

Support for Amazon Bedrock file management capabilities:

² Bedrock Runtime: Amazon Bedrock does not have a dedicated files API. Batch input and output files are stored in Amazon S3 and referenced by S3 URI.

Files are managed through an AI Model with type: "api", the same type that serves batches. Create one dedicated AI Model for batches and files, and keep generate and embeddings on a separate regular model, because each model is either a regular model or an API model, not both.

By default, AI Gateway routes Amazon Bedrock requests to Bedrock Runtime at https://bedrock-runtime.{region}.amazonaws.com.

To route requests to Bedrock Mantle, set endpoint_type: mantle in the target config of your AI Model. AI Gateway then sends them to https://bedrock-mantle.{region}.api.aws.

You don’t need to set the URL yourself. You only need to set upstream_url in your AI Model configuration if you’re using a self-hosted or Amazon Bedrock-compatible endpoint instead.

By default, AI Gateway uses OpenAI-compatible request formats. Configure a native format in your AI Model to use Amazon Bedrock-specific APIs and features.

The following native Amazon Bedrock formats are supported:

Provider-specific limitations for native formats

Bedrock Runtime:

• None.

Bedrock Mantle:

• Mantle has no Converse or InvokeModel API, so Konnect rejects a target with endpoint_type: mantle if its AI Model declares a format with type: bedrock. Use the openai or anthropic format instead.

• Mantle supports only chat, Responses, and files requests. Embeddings and other capabilities are not supported. Use the runtime endpoint for them.

Statistics logging limitations for native formats

Bedrock Runtime:

• Statistics logging is not available for image generation or editing APIs for Amazon Bedrock

Bedrock Mantle:

• None.

To use Amazon Bedrock with AI Gateway, configure a new AI Model Provider. You can then access supported AI Models from Amazon Bedrock.

Here’s a minimal configuration for chat completions:

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/model-providers \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "display_name": "AWS Production",
      "name": "my-aws-account",
      "type": "bedrock",
      "config": {
        "auth": {
          "type": "aws",
          "access_key_id": "'$AWS_ACCESS_KEY_ID'",
          "secret_access_key": "'$AWS_SECRET_ACCESS_KEY'"
        }
      }
    }
    '

ai_gateway_model_providers:
  - ref: my-aws-account
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    display_name: AWS Production
    name: my-aws-account
    type: bedrock
    config:
      auth:
        type: aws
        access_key_id: !env AWS_ACCESS_KEY_ID
        secret_access_key: !secret {source: !env AWS_SECRET_ACCESS_KEY}

You can also use Amazon Bedrock with AWS credentials by setting auth to aws and specifying:

• access_key_id (optional): AWS access key ID for static IAM user credentials. If omitted, the default AWS credentials provider chain is used (EC2 instance profiles, environment variables, etc.).
• secret_access_key (optional): AWS secret access key paired with access_key_id. Required if access_key_id is set.
• assume_role_arn (optional): IAM role ARN to assume for temporary credentials. Useful for cross-account access.
• role_session_name (optional): Session name for the assumed role. Required if assume_role_arn is set.
• sts_endpoint_url (optional): Custom STS endpoint for role assumption. Defaults to https://sts.amazonaws.com.
• batch_role_arn (optional): Separate role ARN for Bedrock batch API calls.

To authenticate with an Amazon Bedrock API key instead of AWS credentials, set config.auth.type to basic with an Authorization header whose value is Bearer BEDROCK_API_KEY. Replace BEDROCK_API_KEY with your Amazon Bedrock API key.

Both authentication methods work with the runtime and mantle endpoints.

AI Gateway can send requests to either of the two Amazon Bedrock inference endpoints, Runtime (the default) or Mantle, depending on the endpoint_type in the target config of an AI Model. Use mantle when your clients already send OpenAI or Anthropic API requests and you want Bedrock Mantle to handle them natively, for example, to run the OpenAI Codex CLI against models on Bedrock. Keep runtime for Bedrock-native clients, for the Bedrock-specific APIs such as Rerank, async invoke, and batch, and for any model that doesn’t support Mantle.

Mantle only supports chat, Responses, and files requests, in the openai or anthropic format. For embeddings, the bedrock format, and Bedrock-specific APIs such as Rerank, async invoke, and batch, use runtime.

The format and capability of the AI Model determine which Mantle API receives the request:

The files capability is available only on an AI Model with type: api. The other capabilities are set on an AI Model with type: model.

For Anthropic requests, AI Gateway sets the anthropic-version header to 2023-06-01 unless the client sends one. If you set upstream_url on a Mantle target, AI Gateway keeps that URL’s path.

The following AI Model sends OpenAI Chat Completions requests to Bedrock Mantle:

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/models \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "my-mantle-model",
      "display_name": "My Mantle model",
      "type": "model",
      "formats": [
        {
          "type": "openai"
        }
      ],
      "capabilities": [
        "generate"
      ],
      "config": {
        "route": {
          "paths": [
            "/v1"
          ],
          "model": {
            "body_param": "model",
            "values": [
              "my-mantle-model"
            ]
          }
        }
      },
      "targets": [
        {
          "name": "'$MANTLE_MODEL_ID'",
          "provider": "my-aws-account",
          "config": {
            "type": "bedrock",
            "region": "us-east-1",
            "endpoint_type": "mantle"
          }
        }
      ]
    }
    '

ai_gateway_models:
  - ref: my-mantle-model
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    name: my-mantle-model
    display_name: My Mantle model
    type: model
    formats:
    - type: openai
    capabilities:
    - generate
    config:
      route:
        paths:
        - "/v1"
        model:
          body_param: model
          values:
          - my-mantle-model
    targets:
    - name: !env MANTLE_MODEL_ID
      provider: my-aws-account
      config:
        type: bedrock
        region: us-east-1
        endpoint_type: mantle

For a complete walkthrough, see Route OpenAI traffic to Amazon Bedrock Mantle.

Before you configure vpc_endpoint, make sure that:

• You created an interface VPC endpoint for the Bedrock Runtime service (com.amazonaws.{region}.bedrock-runtime) in the VPC where your data plane nodes run. See Use interface VPC endpoints (AWS PrivateLink) in the AWS documentation.
• Your data plane nodes can resolve the endpoint’s DNS name and reach it on port 443.
• Your IAM user or role has bedrock:Converse and bedrock:ConverseStream permissions, and the VPC endpoint policy allows them.

To connect to Amazon Bedrock through a VPC endpoint over AWS PrivateLink, set vpc_endpoint in the target config to the hostname of your interface VPC endpoint. AI Gateway then sends requests to that host. Set the value to a hostname only, for example vpce-0123456789abcdef0-abcdefgh.bedrock-runtime.us-east-1.vpce.amazonaws.com. Do not include a scheme. The rest of the request behaves as it does with the public endpoint:

• AI Gateway still builds the Bedrock path for each operation, including streaming (invoke-with-response-stream) and async invoke.
• SigV4 signs requests for the Bedrock service and the configured region, not for the VPC endpoint hostname. Set region to the region of the VPC endpoint.
• If the VPC endpoint is unreachable, the request fails. AI Gateway doesn’t fall back to the public endpoint.
• vpc_endpoint also applies to a Bedrock embeddings model, such as the one set in config.balancer.embeddings.config for semantic load balancing.

curl -X POST https://{region}.api.konghq.com/v1/ai-gateways/{AIGatewayId}/models \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $KONNECT_TOKEN" \
    --data '
    {
      "name": "my-private-bedrock-model",
      "display_name": "My private Bedrock model",
      "type": "model",
      "formats": [
        {
          "type": "openai"
        }
      ],
      "capabilities": [
        "generate"
      ],
      "config": {
        "route": {
          "paths": [
            "/v1"
          ],
          "model": {
            "body_param": "model",
            "values": [
              "my-private-bedrock-model"
            ]
          }
        }
      },
      "targets": [
        {
          "name": "us.anthropic.claude-haiku-4-5-20251001-v1:0",
          "provider": "my-aws-account",
          "config": {
            "type": "bedrock",
            "region": "us-east-1",
            "vpc_endpoint": "'$BEDROCK_VPC_ENDPOINT'"
          }
        }
      ]
    }
    '

ai_gateway_models:
  - ref: my-private-bedrock-model
    ai_gateway: !lookup {id: !env AI_GATEWAY_ID}
    name: my-private-bedrock-model
    display_name: My private Bedrock model
    type: model
    formats:
    - type: openai
    capabilities:
    - generate
    config:
      route:
        paths:
        - "/v1"
        model:
          body_param: model
          values:
          - my-private-bedrock-model
    targets:
    - name: us.anthropic.claude-haiku-4-5-20251001-v1:0
      provider: my-aws-account
      config:
        type: bedrock
        region: us-east-1
        vpc_endpoint: !env BEDROCK_VPC_ENDPOINT

vpc_endpoint overrides only the host. If you also set upstream_url on the same target, upstream_url takes precedence and AI Gateway ignores vpc_endpoint. Use upstream_url only when you need to override the full URL, because it also replaces the path that AI Gateway builds for each Bedrock operation.