AI Gateway providers

Yes, if AI Gateway is running on Azure, you can configure an AI Model Provider to detect the designated Managed Identity or User-Assigned Identity of that Azure Compute resource and use it for authentication.

In your AI Model Provider configuration, set auth.type to azure, then:

• Set auth.use_managed_identity to true to use a system-assigned Managed Identity.
• Set auth.use_managed_identity to true and auth.client_id to the client ID to use a user-assigned identity.

Then reference this AI Model Provider in your AI Model to proxy requests with the appropriate Azure credentials.

You can configure model generation parameters when calling Gemini through AI Gateway:

• Using the Gemini SDK:

  1. Create an AI Model Provider for Gemini and an AI Model that references it.
  2. Configure parameters like temperature, top_p, and top_k on the client side:

      model = genai.GenerativeModel(
          'gemini-2.5-flash',
          generation_config=genai.types.GenerationConfig(
              temperature=0.7,
              top_p=0.9,
              top_k=40,
              max_output_tokens=1024
          )
      )

     Copied!

• Using the OpenAI SDK with AI Gateway:

  1. Create an AI Model with an openai entry in its formats array, and a target that references your Gemini AI Model Provider.
  2. You can configure parameters in one of three ways:
     • Configure them in the AI Model only.
     • Configure them in the client only.
     • Configure them in both. The client-side values override the model config.

Configure an AI Model that uses a Gemini AI Model Provider, then declare the googleSearch tool in your requests.

Pass imageConfig parameters via generationConfig in your image generation requests.

Pass thinkingConfig parameters via extra_body in your requests to enable detailed reasoning traces.

For cross-region inference with Amazon Bedrock, prefix the model ID with a geographic identifier in your AI Model configuration:

{geography-prefix}.{provider}.{model-name}...

For example: us.anthropic.claude-sonnet-4-5-20250929-v1:0

For a full list of supported cross-region inference profiles, see Supported Regions and models for inference profiles in the AWS documentation.

Use the extra_body feature when sending requests to an AI Model that proxies Amazon Bedrock video generation in OpenAI format:

    curl http://localhost:8000 \
    -H "Authorization: Bearer $OPENAI_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "amazon.nova-reel-v1:0",
        "prompt": "A large red square that is rotating",
        "extra_body": {
        "fps": 24
        }
    }'

Configure an AI Model with a Bedrock AI Model Provider and set up AWS authentication using IAM credentials or assumed roles.

Add a guardrailConfig object to your request body when calling an AI Model that proxies Amazon Bedrock:

      {
          "messages": [
              {
                  "role": "system",
                  "content": "You are a scientist."
              },
              {
                  "role": "user",
                  "content": "What is the Boltzmann equation?"
              }
          ],
          "guardrailConfig": {
              "guardrailIdentifier": "$GUARDRAIL-IDENTIFIER",
              "guardrailVersion": "1",
              "trace": "enabled"
          }
      }

For more details, see Guardrails and content safety and the AWS Bedrock guardrails documentation.