For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Route agentgateway LLM traffic to models hosted on Microsoft Azure AI.

Configure Microsoft Azure AI as an LLM provider in agentgateway.

Authentication

Before you can use Azure as an LLM provider, you must authenticate by using one of the standard Azure authentication methods. In standalone mode, this authentication is configured with llm.models[] fields (for example, params.apiKey or auth.azure). In routing-based configurations, use policies.backendAuth.azure.

Configuration

Azure supports two endpoint types:

• Azure AI Foundry (foundry): Connect to Azure AI Foundry project endpoints at {resourceName}-resource.services.ai.azure.com.
• Azure OpenAI (openAI): Connect directly to Azure OpenAI Service deployments at {resourceName}.openai.azure.com.

You can add the model in the UI or in your configuration file.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models:
  - name: "*"
    provider: azure
    params:
      azureResourceName: "your-resource-name"
      azureResourceType: foundry
      azureProjectName: "your-project-name"

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models:
  - name: "gpt-4.1"
    provider: azure
    auth:
      key:
        value: "$AZURE_API_KEY"
        location:
          header:
            name: api-key
    params:
      azureResourceName: "your-resource-name"
      azureResourceType: foundry
      azureProjectName: "your-project-name"

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models:
  - name: "gpt-4.1"
    provider: azure
    params:
      azureResourceName: "your-resource-name"
      azureResourceType: openAI

Advanced configuration

For advanced Azure AI scenarios, use the traditional listener/route configuration format. The following tabs show examples for different authentication methods.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
gateways:
  default:
    port: 3000
routes:
- matches:
  - path:
      pathPrefix: /azure
  backends:
  - ai:
      name: azure
      provider:
        azure:
          resourceName: "your-resource-name"
          projectName: "your-project-name"
          resourceType: foundry
          model: gpt-4.1

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
gateways:
  default:
    port: 3000
routes:
- matches:
  - path:
      pathPrefix: /azure
  policies:
    backendAuth:
      azure:
        explicitConfig:
          clientSecret:
            tenant_id: "<your-tenant-id>"
            client_id: "<your-client-id>"
            client_secret: "<your-client-secret>"
  backends:
  - ai:
      name: azure
      provider:
        azure:
          resourceName: "your-resource-name"
          projectName: "your-project-name"
          resourceType: foundry
          model: gpt-4.1

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
gateways:
  default:
    port: 3000
routes:
- backends:
  - ai:
      name: azure
      provider:
        azure:
          resourceName: "your-resource-name"
          resourceType: openAI
          model: gpt-4.1
  policies:
    backendAuth:
      azure:
        explicitConfig:
          clientSecret:
            tenant_id: "<your-tenant-id>"
            client_id: "<your-client-id>"
            client_secret: "<your-client-secret>"

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
gateways:
  default:
    port: 3000
routes:
- backends:
  - ai:
      name: azure
      provider:
        azure:
          resourceName: "your-resource-name"
          resourceType: openAI
          model: gpt-4.1
  policies:
    backendAuth:
      azure:
        explicitConfig:
          managedIdentity: {}

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
gateways:
  default:
    port: 3000
routes:
- backends:
  - ai:
      name: azure
      provider:
        azure:
          resourceName: "your-resource-name"
          resourceType: openAI
          model: gpt-4.1
  policies:
    backendAuth:
      azure:
        explicitConfig:
          managedIdentity:
            userAssignedIdentity:
              clientId: "<your-managed-identity-client-id>"
              # OR use objectId or resourceId instead
              # objectId: "your-managed-identity-object-id"
              # resourceId: "/subscriptions/.../resourceGroups/.../providers/Microsoft.ManagedIdentity/userAssignedIdentities/..."

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
gateways:
  default:
    port: 3000
routes:
- backends:
  - ai:
      name: azure
      provider:
        azure:
          resourceName: "your-resource-name"
          resourceType: openAI
          model: gpt-4.1
  policies:
    backendAuth:
      azure:
        explicitConfig:
          workloadIdentity: {}
    backendTLS: {}

Use Claude models on Azure AI Foundry

Azure AI Foundry hosts Anthropic Claude models at native Anthropic endpoints. When you set azureResourceType: foundry and a model name that starts with claude-, agentgateway automatically routes requests to the Anthropic-native path (/anthropic/v1/messages) instead of the OpenAI-compatible path, and injects the required anthropic-version header. No extra configuration is needed beyond specifying a Claude model name.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
gateways:
  default:
    port: 3000
routes:
- name: azure
  matches:
  - path:
      pathPrefix: /azure-anthropic #prefix example
  backends:
  - ai:
      name: azure
      provider:
        azure:
          resourceName: your-foundry-resource
          projectName: your-project-name
          resourceType: foundry
          model: claude-sonnet-4-6
    policies:
      backendAuth:
        key:
          value: your-api-key

After running agentgateway with this configuration, send a request to verify:

curl -X POST http://localhost:4000/azure-anthropic \
  -H "Content-Type: application/json" \
  -d '{
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Use Microsoft Decision models on Azure AI Foundry

Azure AI Foundry hosts Microsoft Decision models, which answer classification questions about a text, such as which of several choices fits best or how likely a statement is true. agentgateway accepts two request formats for these models:

• Decisions API (/v1/decisions): The OpenAI-compatible format. agentgateway converts the request and the response to and from the native SystemOne format. For another example of this API, see the Jev guardrail guide.
• SystemOne (/v1/systemone): The native format of the model. agentgateway forwards the request unchanged.

Both formats use the Foundry resource endpoint, https://{resourceName}.cognitiveservices.azure.com/providers/microsoft/v1/systemone. In SystemOne, a question of type noul returns the probability that a statement is true. The Decisions API calls this question type predicate.

Before you begin, deploy Microsoft-Decision-1 in your Foundry resource. Then sign in with the Azure CLI or the Azure Developer CLI as an identity that has inference access to the Foundry resource.

1. Save the following configuration as config-decisions.yaml. Replace the resource name and deployment name with your own values.

   Review the following example configuration.

   # yaml-language-server: $schema=https://agentgateway.dev/schema/config
   llm:
     models:
     - auth:
         azure:
           developerImplicit: {}
       name: foundry-decision
       params:
         azureResourceName: my-foundry-resource
         azureResourceType: foundry
         model: my-decision-deployment
       provider: azure
     port: 3000
   Review the following table to understand this configuration.

   Setting	Description
   name	The model name that clients send in the model field. This name is not the deployment name. A request with any other model name returns a 404 error.
   provider	Set to azure for Azure AI Foundry.
   auth.azure.developerImplicit	Authenticate with the credentials from your Azure CLI or Azure Developer CLI sign-in.
   params.azureResourceName	The Azure AI Foundry resource name used to construct the endpoint hostname.
   params.azureResourceType	Set to foundry to use Azure AI Foundry endpoints.
   params.model	The name of your Foundry deployment. agentgateway sends this name upstream and replaces the model value in the client request.

   Both request paths are served from the same model. You do not need to configure a route for each path.

2. Validate the configuration, then start agentgateway.

   agentgateway --validate-only -f config-decisions.yaml
   agentgateway -f config-decisions.yaml

3. In another terminal, send a choice question through the Decisions API.

   curl --fail-with-body --max-time 120 http://localhost:3000/v1/decisions \
     -H 'Content-Type: application/json' \
     -d '{
       "input": "I was charged twice for my subscription.",
       "model": "foundry-decision",
       "questions": [{
         "choices": [
           {"description": "Payments and invoices", "value": "billing"},
           {"description": "Software problems", "value": "technical"}
         ],
         "instructions": "Which team should handle this support ticket?",
         "name": "department",
         "type": "choice"
       }]
     }'

   The response includes a department choice with a probability for each value, and token usage. The Decisions API also supports predicate and score questions.

4. Send the same question in the native SystemOne format. SystemOne takes the text in state and the questions as a map keyed by name, and returns an answers map. Its question types are noul, choice, and score.

   curl --fail-with-body --max-time 120 http://localhost:3000/v1/systemone \
     -H 'Content-Type: application/json' \
     -d '{
       "model": "foundry-decision",
       "questions": {
         "department": {
           "criteria": {
             "billing": "Payments and invoices",
             "technical": "Software problems"
           },
           "instructions": "Which team should handle this support ticket?",
           "type": "choice"
         }
       },
       "state": "I was charged twice for my subscription."
     }'

   Note

   Prompt guards are disabled for requests in the native SystemOne format.

5. When you finish, stop agentgateway with Ctrl+C.