diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/APIM-Connection-Objects.md b/samples/microsoft/infrastructure-setup/01-connections/apim/APIM-Connection-Objects.md new file mode 100644 index 00000000..cdf17b7e --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/APIM-Connection-Objects.md @@ -0,0 +1,414 @@ +# APIM Connection JSON Examples + +## Overview + +APIM (API Management) connections are specialized ModelGateway connections designed for Azure API Management scenarios. These connections provide intelligent defaults and follow APIM standard conventions while integrating with the broader ModelGateway ecosystem. + +### Key Features + +- **APIM Category**: Uses `"category": "ApiManagement"` for proper APIM-specific handling +- **Intelligent Defaults**: Provides standard APIM endpoints when metadata is not specified +- **Convention-Based**: Follows Azure API Management naming and routing patterns +- **Flexible Override**: Supports metadata overrides for custom APIM configurations +- **Enterprise Ready**: Designed for production APIM gateway scenarios + +### APIM-Specific Behavior + +**Default Endpoints**: When metadata is not provided, APIM connections use these defaults: +- List Deployments: `/deployments` +- Get Deployment: `/deployments/{deploymentName}` +- Provider: `AzureOpenAI` + +**Configuration Priority**: +1. Explicit metadata values (highest priority) +2. APIM standard defaults (fallback) + +### Authentication Patterns + +APIM connections support various authentication methods: +- **API Key**: Standard subscription key authentication +- **AAD (Azure Active Directory)**: Enterprise identity integration + + +## Connection Schema Definitions + +### 1. DeploymentInPath - **REQUIRED** + +Controls how deployment names are passed to the APIM gateway. **This field is required** and must be specified in all APIM connections. + +```json +{ + "deploymentInPath": "true" // or "false" +} +``` + +**When `"true"` (Path-based routing):** +``` +URL: {target}/deployments/{deploymentName}/chat/completions +``` + +**When `"false"`:** +``` +URL: {target}/chat/completions +Body: {"model": "{deploymentName}"} +``` + +**Azure Agents Behavior:** +- `"true"`: Injects deployment name into URL path for APIM routing +- `"false"`: Passes deployment name via model parameter in request body +- **If not specified**: Connection will fail validation + +### 2. InferenceAPIVersion - **OPTIONAL** + +Specifies the API version for model inference calls (chat completions, embeddings, etc.) through APIM. + +```json +{ + "inferenceAPIVersion": "2024-02-01" +} +``` + +**Usage by Azure Agents:** +- Appended as query parameter: `?api-version=2024-02-01` +- Used for all inference requests through APIM gateway +- **If not specified**: Azure Agents will use a default API version + +### 3. DeploymentAPIVersion - **OPTIONAL** + +Specifies the API version for deployment management calls through APIM. + +```json +{ + "deploymentAPIVersion": "2024-02-01" +} +``` + +**Usage by Azure Agents:** +- Used only for `modelDiscovery` endpoint calls through APIM +- Separate from `inferenceAPIVersion` to allow different versioning +- Appended as query parameter to discovery endpoints +- **If not specified**: Azure Agents will not append any query param to the deployments API. + +### 4. ModelDiscovery (Dynamic Discovery) - **OPTIONAL** + +The `modelDiscovery` object is **optional** and only needed when customers expose different endpoints than the APIM defaults (`/deployments`, `/deployments/{deploymentName}`). Azure Agents will automatically use APIM standard conventions if `modelDiscovery` is not provided in the metadata. + +**When to include `modelDiscovery`:** +- Customer APIM exposes custom endpoints (not `/deployments`) +- Different API routing or endpoint naming conventions +- Need to specify OpenAI format instead of default AzureOpenAI + +**When to omit `modelDiscovery`:** +- Standard APIM setup with default `/deployments` endpoints +- Following Azure OpenAI /deployments API conventions through APIM +- Using APIM defaults is sufficient + +```json +{ + "modelDiscovery": { + "listModelsEndpoint": "/custom/models", + "getModelEndpoint": "/custom/models/{deploymentName}", + "deploymentProvider": "OpenAI" + } +} +``` + +**Fields:** +- `listModelsEndpoint` - Endpoint to retrieve all available models (relative to target URL) +- `getModelEndpoint` - Endpoint to get specific model details with `{deploymentName}` placeholder +- `deploymentProvider` - Provider format for response parsing. **Supported values: `"OpenAI"` and `"AzureOpenAI"`** (exactly 2 formats) + +**APIM Default Values (when `modelDiscovery` is omitted):** +- `listModelsEndpoint`: `/deployments` +- `getModelEndpoint`: `/deployments/{deploymentName}` +- `deploymentProvider`: `AzureOpenAI` + +**How Azure Agents Uses It:** +1. Constructs full URL: `{target}{listModelsEndpoint}` +2. Adds authentication headers from `credentials` +3. Makes HTTP request to discover available models +4. Parses response based on `deploymentProvider` format (OpenAI or AzureOpenAI) +5. Caches model list for connection lifecycle + +**Supported DeploymentProvider Formats:** + +We support exactly **2 deployment API formats** for model discovery through APIM: + +**1. AzureOpenAI Format Responses (`deploymentProvider: "AzureOpenAI"`) - DEFAULT:** + +*List Deployments Response (`listModelsEndpoint`):`* +```json +{ + "value": [ + { + "id": "/subscriptions/.../deployments/gpt-4-deployment", + "name": "gpt-4-deployment", + "type": "Microsoft.CognitiveServices/accounts/deployments", + "properties": { + "model": { + "format": "OpenAI", + "name": "gpt-4", + "version": "0613" + }, + "provisioningState": "Succeeded" + } + }, + { + "id": "/subscriptions/.../deployments/gpt-35-turbo-deployment", + "name": "gpt-35-turbo-deployment", + "type": "Microsoft.CognitiveServices/accounts/deployments", + "properties": { + "model": { + "format": "OpenAI", + "name": "gpt-35-turbo", + "version": "0613" + }, + "provisioningState": "Succeeded" + } + } + ] +} +``` + +*Get Deployment by Name Response (`getModelEndpoint`):`* +```json +{ + "id": "/subscriptions/.../deployments/gpt-4-deployment", + "name": "gpt-4-deployment", + "type": "Microsoft.CognitiveServices/accounts/deployments", + "properties": { + "model": { + "format": "OpenAI", + "name": "gpt-4", + "version": "0613" + }, + "provisioningState": "Succeeded" + } +} +``` + +- Uses `value` array for list, single object for get-by-name +- Follows Azure ARM resource structure +- Separate deployment `name` and model details in `properties.model` +- Includes model `name`, `version`, and `format` +- Provides deployment provisioning state + +**2. OpenAI Format Responses (`deploymentProvider: "OpenAI"`):** + +*List Models Response (`listModelsEndpoint`):`* +```json +{ + "data": [ + { + "id": "gpt-4", + "object": "model", + "created": 1687882411, + "owned_by": "openai" + }, + { + "id": "gpt-3.5-turbo", + "object": "model", + "created": 1677610602, + "owned_by": "openai" + } + ] +} +``` + +*Get Model by Name Response (`getModelEndpoint`):`* +```json +{ + "id": "gpt-4", + "object": "model", + "created": 1687882411, + "owned_by": "openai" +} +``` + +- Uses `data` array for list, single object for get-by-name +- `id` serves as both deployment name and model name +- No version information provided in API responses + +### 5. Static Discovery - **OPTIONAL** + +Static discovery uses a predefined `models` array in metadata. Models are defined using the `ModelInfo` structure. **Either static discovery OR dynamic discovery can be used, not both.** + +```json +{ + "models": [ + { + "name": "deployment-name", + "properties": { + "model": { + "name": "model-name", + "version": "model-version", + "format": "provider-format" + } + } + } + ] +} +``` + +**Structure:** +- `name` - Deployment name (how you reference the model in APIM API calls) +- `properties.model.name` - Actual model name from provider +- `properties.model.version` - Model version identifier +- `properties.model.format` - Provider format (OpenAI, DeepSeek, etc.) + +**When to use static discovery:** +- Fixed set of known models through APIM +- No need for runtime model discovery +- Predefined model configurations + +**If not specified**: Azure Agents will attempt dynamic discovery using `modelDiscovery` settings or APIM defaults + +## Example 1: All Defaults with Required Fields Only + +Uses all APIM defaults, but provides fields: `deploymentInPath`, `inferenceAPIVersion`. + +```json +{ + "id": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/connections/apim-defaults", + "name": "apim-defaults", + "type": "Microsoft.MachineLearningServices/workspaces/connections", + "properties": { + "category": "ApiManagement", + "target": "https://your-apim-gateway.azure-api.net/myapi", + "authType": "ApiKey", + "credentials": { + "key": "{api-key-reference}" + }, + "metadata": { + "deploymentInPath": "true", + "inferenceAPIVersion": "2024-02-01" + } + } +} +``` + +## Example 2: APIM with Deployment API Version + +Configuration with required `deploymentInPath` and inference version, deployment API Version. + +```json +{ + "id": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/connections/apim-minimal", + "name": "apim-minimal", + "type": "Microsoft.MachineLearningServices/workspaces/connections", + "properties": { + "category": "ApiManagement", + "target": "https://minimal-apim.azure-api.net/api", + "authType": "AAD", + "credentials": {}, + "metadata": { + "deploymentInPath": "true", + "inferenceAPIVersion": "2024-02-01", + "deploymentAPIVersion": "2025-01-01" + } + } +} +``` + +## Example 3: APIM with Dynamic Discovery + +Dynamic model discovery using `/models` and `/models/{deploymentName}` endpoints with AzureOpenAI format deployments. + +```json +{ + "id": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/connections/apim-dynamic-azure", + "name": "apim-dynamic-azure", + "type": "Microsoft.MachineLearningServices/workspaces/connections", + "properties": { + "category": "ApiManagement", + "target": "https://dynamic-apim.azure-api.net/api", + "authType": "ApiKey", + "credentials": { + "key": "{api-key-reference}" + }, + "metadata": { + "modelDiscovery": { + "listModelsEndpoint": "v1/models", + "getModelEndpoint": "/models/{deploymentName}", + "deploymentProvider": "AzureOpenAI" + }, + "deploymentInPath": "true", + "inferenceAPIVersion": "2024-02-01", + "deploymentAPIVersion": "2024-02-01" + } + } +} +``` + +## Example 4: APIM with Dynamic Discovery - OpenAI + +Same as Example 3 but using OpenAI format for model discovery. + +```json +{ + "id": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/connections/apim-dynamic-openai", + "name": "apim-dynamic-openai", + "type": "Microsoft.MachineLearningServices/workspaces/connections", + "properties": { + "category": "ApiManagement", + "target": "https://openai-apim.azure-api.net/api/", + "authType": "ApiKey", + "credentials": { + "key": "{api-key-reference}" + }, + "metadata": { + "modelDiscovery": { + "listModelsEndpoint": "/models", + "getModelEndpoint": "/models/{deploymentName}", + "deploymentProvider": "OpenAI" + }, + "deploymentInPath": "false", + "inferenceAPIVersion": "2024-02-01" + } + } +} +``` + +## Example 5: APIM with Static Model List + +Predefined static list of models without dynamic discovery. + +```json +{ + "id": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/connections/apim-static-models", + "name": "apim-static-models", + "type": "Microsoft.MachineLearningServices/workspaces/connections", + "properties": { + "category": "ApiManagement", + "target": "https://static-apim.azure-api.net/api", + "authType": "AAD", + "credentials": {}, + "metadata": { + "models": [ + { + "name": "gpt-4-deployment", + "properties": { + "model": { + "name": "gpt-4", + "version": "0613", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-35-turbo-deployment", + "properties": { + "model": { + "name": "gpt-35-turbo", + "version": "0613", + "format": "OpenAI" + } + } + } + ], + "deploymentInPath": "false", + "inferenceAPIVersion": "2024-02-01" + } + } +} +``` \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/README.md b/samples/microsoft/infrastructure-setup/01-connections/apim/README.md new file mode 100644 index 00000000..9ca91d35 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/README.md @@ -0,0 +1,60 @@ +# APIM Connection Examples + +This folder contains Azure Bicep templates for creating APIM (API Management) connections to Azure AI Foundry projects. + +## Prerequisites + +1. **Azure CLI** installed and configured +2. **Existing APIM service** with APIs configured +3. **AI Foundry account and project** already created + +## How to Deploy + +### Basic APIM Connection +```bash +# 1. Edit parameters-basic.json with your resource IDs +# 2. Deploy using the parameters file +az deployment group create \ + --resource-group \ + --template-file connection-apim-basic.bicep \ + --parameters @parameters-basic.json +``` + +### Deployment API Version APIM Connection +```bash +# 1. Edit parameters-deployment-api.json with your resource IDs +# 2. Deploy using the parameters file +az deployment group create \ + --resource-group \ + --template-file connection-apim-deployment-api-version.bicep \ + --parameters @parameters-deployment-api.json +``` + +### Dynamic Discovery APIM Connection +```bash +# 1. Edit parameters-dynamic.json with your resource IDs +# 2. Deploy using the parameters file +az deployment group create \ + --resource-group \ + --template-file connection-apim-dynamic-discovery.bicep \ + --parameters @parameters-dynamic.json +``` + +### Static Models APIM Connection +```bash +# 1. Edit parameters-static.json with your resource IDs +# 2. Deploy using the parameters file +az deployment group create \ + --resource-group \ + --template-file connection-apim-static-models.bicep \ + --parameters @parameters-static.json +``` + +## Parameter Files + +- `parameters-basic.json`: For basic APIM connections with minimal configuration +- `parameters-deployment-api.json`: For APIM connections with API versioning (includes inferenceAPIVersion and deploymentAPIVersion) +- `parameters-dynamic.json`: For APIM connections with dynamic model discovery (includes OpenAI endpoint configurations) +- `parameters-static.json`: For APIM connections with static model lists (includes customizable staticModels array) + +Edit these files to update the resource IDs for your environment. \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-basic.bicep b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-basic.bicep new file mode 100644 index 00000000..115d0d1a --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-basic.bicep @@ -0,0 +1,78 @@ +/* +Connections enable your AI applications to access tools and objects managed elsewhere in or outside of Azure. + +This example demonstrates how to add an Azure API Management connection for a specific API. +This implements Example 1 from the APIM Connection documentation: "All Defaults with Required Fields Only" + +Uses all APIM defaults with only the required fields: +- deploymentInPath: Controls how deployment names are passed to APIM gateway +- inferenceAPIVersion: API version for model inference calls + +This uses APIM default endpoints: +- List Deployments: /deployments +- Get Deployment: /deployments/{deploymentName} +- Provider: AzureOpenAI + +IMPORTANT: Make sure you are logged into the subscription where the AI Foundry resource exists before deploying. +The connection will be created in the AI Foundry project, so you need to be in that subscription context. +Use: az account set --subscription +*/ + +param projectResourceId string = '/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project' +param apimResourceId string = '/subscriptions/87654321-4321-4321-4321-cba987654321/resourceGroups/rg-sample-apim/providers/Microsoft.ApiManagement/service/sample-apim-service' +param apiName string = 'foundry' +param apimSubscriptionName string = 'master' // Default subscription name in APIM, update it to your subscription name for apikey auth + +// Connection naming - can be overridden via parameter +param connectionName string = '' // Optional: specify custom connection name + +// Generate connection name if not provided +var apimServiceName = split(apimResourceId, '/')[8] +var generatedConnectionName = 'apim-${apimServiceName}-${apiName}' +var finalConnectionName = connectionName != '' ? connectionName : generatedConnectionName + +// Connection configuration +@allowed([ + 'ApiKey' + 'AAD' +]) +param authType string = 'ApiKey' // Authentication type for the connection + +param isSharedToAll bool = false // Whether the connection should be shared to all users in the project + +// APIM-specific configuration parameters +@allowed([ + 'true' + 'false' +]) +param deploymentInPath string = 'true' // Controls how deployment names are passed to APIM gateway + +param inferenceAPIVersion string = '2024-02-01' // API version for inference calls (chat completions, embeddings, etc.) + +// Build the metadata object for Example 1: All Defaults with Required Fields Only +var example1Metadata = { + deploymentInPath: deploymentInPath + inferenceAPIVersion: inferenceAPIVersion +} + +// Use the common module to create the APIM connection +module apimConnection 'modules/apim-connection-common.bicep' = { + name: 'apim-connection-example1' + params: { + projectResourceId: projectResourceId + connectionName: finalConnectionName + apimResourceId: apimResourceId + apiName: apiName + apimSubscriptionName: apimSubscriptionName + authType: authType + isSharedToAll: isSharedToAll + metadata: example1Metadata + } +} + +// Output information from the connection +output connectionName string = apimConnection.outputs.connectionName +output connectionId string = apimConnection.outputs.connectionId +output targetUrl string = apimConnection.outputs.targetUrl +output authType string = apimConnection.outputs.authType +output metadata object = apimConnection.outputs.metadata diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-basic.json b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-basic.json new file mode 100644 index 00000000..a48a43c5 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-basic.json @@ -0,0 +1,214 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "10056086952953764903" + } + }, + "parameters": { + "projectResourceId": { + "type": "string", + "defaultValue": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project" + }, + "apimResourceId": { + "type": "string", + "defaultValue": "/subscriptions/87654321-4321-4321-4321-cba987654321/resourceGroups/rg-sample-apim/providers/Microsoft.ApiManagement/service/sample-apim-service" + }, + "apiName": { + "type": "string", + "defaultValue": "foundry" + }, + "apimSubscriptionName": { + "type": "string", + "defaultValue": "master" + }, + "connectionName": { + "type": "string", + "defaultValue": "" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey", + "allowedValues": [ + "ApiKey", + "AAD" + ] + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "deploymentInPath": { + "type": "string", + "defaultValue": "true", + "allowedValues": [ + "true", + "false" + ] + }, + "inferenceAPIVersion": { + "type": "string", + "defaultValue": "2024-02-01" + } + }, + "variables": { + "apimServiceName": "[split(parameters('apimResourceId'), '/')[8]]", + "generatedConnectionName": "[format('apim-{0}-{1}', variables('apimServiceName'), parameters('apiName'))]", + "finalConnectionName": "[if(not(equals(parameters('connectionName'), '')), parameters('connectionName'), variables('generatedConnectionName'))]", + "example1Metadata": { + "deploymentInPath": "[parameters('deploymentInPath')]", + "inferenceAPIVersion": "[parameters('inferenceAPIVersion')]" + } + }, + "resources": [ + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "apim-connection-example1", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "projectResourceId": { + "value": "[parameters('projectResourceId')]" + }, + "connectionName": { + "value": "[variables('finalConnectionName')]" + }, + "apimResourceId": { + "value": "[parameters('apimResourceId')]" + }, + "apiName": { + "value": "[parameters('apiName')]" + }, + "apimSubscriptionName": { + "value": "[parameters('apimSubscriptionName')]" + }, + "authType": { + "value": "[parameters('authType')]" + }, + "isSharedToAll": { + "value": "[parameters('isSharedToAll')]" + }, + "metadata": { + "value": "[variables('example1Metadata')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "5161752358692660897" + } + }, + "parameters": { + "projectResourceId": { + "type": "string" + }, + "connectionName": { + "type": "string" + }, + "apimResourceId": { + "type": "string" + }, + "apiName": { + "type": "string" + }, + "apimSubscriptionName": { + "type": "string", + "defaultValue": "master" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey" + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "metadata": { + "type": "object" + } + }, + "variables": { + "aiFoundryName": "[split(parameters('projectResourceId'), '/')[8]]", + "projectName": "[split(parameters('projectResourceId'), '/')[10]]", + "apimSubscriptionId": "[split(parameters('apimResourceId'), '/')[2]]", + "apimResourceGroupName": "[split(parameters('apimResourceId'), '/')[4]]", + "apimServiceName": "[split(parameters('apimResourceId'), '/')[8]]" + }, + "resources": [ + { + "condition": "[equals(parameters('authType'), 'ApiKey')]", + "type": "Microsoft.CognitiveServices/accounts/projects/connections", + "apiVersion": "2025-04-01-preview", + "name": "[format('{0}/{1}/{2}', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]", + "properties": { + "category": "ApiManagement", + "target": "[format('{0}/{1}', reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service', variables('apimServiceName')), '2021-08-01').gatewayUrl, reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service/apis', variables('apimServiceName'), parameters('apiName')), '2021-08-01').path)]", + "authType": "ApiKey", + "isSharedToAll": "[parameters('isSharedToAll')]", + "credentials": { + "key": "[listSecrets(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service/subscriptions', variables('apimServiceName'), parameters('apimSubscriptionName')), '2021-08-01').primaryKey]" + }, + "metadata": "[parameters('metadata')]" + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[if(equals(parameters('authType'), 'ApiKey'), parameters('connectionName'), '')]" + }, + "connectionId": { + "type": "string", + "value": "[if(equals(parameters('authType'), 'ApiKey'), resourceId('Microsoft.CognitiveServices/accounts/projects/connections', variables('aiFoundryName'), variables('projectName'), parameters('connectionName')), '')]" + }, + "targetUrl": { + "type": "string", + "value": "[format('{0}/{1}', reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service', variables('apimServiceName')), '2021-08-01').gatewayUrl, reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service/apis', variables('apimServiceName'), parameters('apiName')), '2021-08-01').path)]" + }, + "authType": { + "type": "string", + "value": "[parameters('authType')]" + }, + "metadata": { + "type": "object", + "value": "[parameters('metadata')]" + } + } + } + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example1'), '2022-09-01').outputs.connectionName.value]" + }, + "connectionId": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example1'), '2022-09-01').outputs.connectionId.value]" + }, + "targetUrl": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example1'), '2022-09-01').outputs.targetUrl.value]" + }, + "authType": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example1'), '2022-09-01').outputs.authType.value]" + }, + "metadata": { + "type": "object", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example1'), '2022-09-01').outputs.metadata.value]" + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-deployment-api-version.bicep b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-deployment-api-version.bicep new file mode 100644 index 00000000..f1e11aa7 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-deployment-api-version.bicep @@ -0,0 +1,82 @@ +/* +Connections enable your AI applications to access tools and objects managed elsewhere in or outside of Azure. + +This example demonstrates how to add an Azure API Management connection for a specific API. +This implements Example 2 from the APIM Connection documentation: "APIM with Deployment API Version" +Uses ApiKey authentication with both inference and deployment API versions specified. + +Configuration includes: +- deploymentInPath: Controls how deployment names are passed to APIM gateway +- inferenceAPIVersion: API version for model inference calls (chat completions, embeddings, etc.) +- deploymentAPIVersion: API version for deployment management calls (model discovery) + +This uses APIM default endpoints: +- List Deployments: /deployments +- Get Deployment: /deployments/{deploymentName} +- Provider: AzureOpenAI + +IMPORTANT: Make sure you are logged into the subscription where the AI Foundry resource exists before deploying. +The connection will be created in the AI Foundry project, so you need to be in that subscription context. +Use: az account set --subscription +*/ + +param projectResourceId string = '/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project' +param apimResourceId string = '/subscriptions/87654321-4321-4321-4321-cba987654321/resourceGroups/rg-sample-apim/providers/Microsoft.ApiManagement/service/sample-apim-service' +param apiName string = 'foundry' +param apimSubscriptionName string = 'master' // Default subscription name in APIM, update it to your subscription name for apikey auth + +// Connection naming - can be overridden via parameter +param connectionName string = '' // Optional: specify custom connection name + +// Generate connection name if not provided +var apimServiceName = split(apimResourceId, '/')[8] +var generatedConnectionName = 'apim-${apimServiceName}-${apiName}-v2' +var finalConnectionName = connectionName != '' ? connectionName : generatedConnectionName + +// Connection configuration +@allowed([ + 'ApiKey' + 'AAD' +]) +param authType string = 'ApiKey' // Authentication type for the connection + +param isSharedToAll bool = false // Whether the connection should be shared to all users in the project + +// APIM-specific configuration parameters +@allowed([ + 'true' + 'false' +]) +param deploymentInPath string = 'true' // Controls how deployment names are passed to APIM gateway + +param inferenceAPIVersion string = '2024-02-01' // API version for inference calls (chat completions, embeddings, etc.) +param deploymentAPIVersion string = '2025-01-01' // API version for deployment management calls + +// Build the metadata object for Example 2: APIM with Deployment API Version +var example2Metadata = { + deploymentInPath: deploymentInPath + inferenceAPIVersion: inferenceAPIVersion + deploymentAPIVersion: deploymentAPIVersion +} + +// Use the common module to create the APIM connection +module apimConnection 'modules/apim-connection-common.bicep' = { + name: 'apim-connection-example2' + params: { + projectResourceId: projectResourceId + connectionName: finalConnectionName + apimResourceId: apimResourceId + apiName: apiName + apimSubscriptionName: apimSubscriptionName + authType: authType + isSharedToAll: isSharedToAll + metadata: example2Metadata + } +} + +// Output information from the connection +output connectionName string = apimConnection.outputs.connectionName +output connectionId string = apimConnection.outputs.connectionId +output targetUrl string = apimConnection.outputs.targetUrl +output authType string = apimConnection.outputs.authType +output metadata object = apimConnection.outputs.metadata diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-deployment-api-version.json b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-deployment-api-version.json new file mode 100644 index 00000000..43a6b9ac --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-deployment-api-version.json @@ -0,0 +1,219 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "16136308199175131090" + } + }, + "parameters": { + "projectResourceId": { + "type": "string", + "defaultValue": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project" + }, + "apimResourceId": { + "type": "string", + "defaultValue": "/subscriptions/87654321-4321-4321-4321-cba987654321/resourceGroups/rg-sample-apim/providers/Microsoft.ApiManagement/service/sample-apim-service" + }, + "apiName": { + "type": "string", + "defaultValue": "foundry" + }, + "apimSubscriptionName": { + "type": "string", + "defaultValue": "master" + }, + "connectionName": { + "type": "string", + "defaultValue": "" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey", + "allowedValues": [ + "ApiKey", + "AAD" + ] + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "deploymentInPath": { + "type": "string", + "defaultValue": "true", + "allowedValues": [ + "true", + "false" + ] + }, + "inferenceAPIVersion": { + "type": "string", + "defaultValue": "2024-02-01" + }, + "deploymentAPIVersion": { + "type": "string", + "defaultValue": "2025-01-01" + } + }, + "variables": { + "apimServiceName": "[split(parameters('apimResourceId'), '/')[8]]", + "generatedConnectionName": "[format('apim-{0}-{1}-v2', variables('apimServiceName'), parameters('apiName'))]", + "finalConnectionName": "[if(not(equals(parameters('connectionName'), '')), parameters('connectionName'), variables('generatedConnectionName'))]", + "example2Metadata": { + "deploymentInPath": "[parameters('deploymentInPath')]", + "inferenceAPIVersion": "[parameters('inferenceAPIVersion')]", + "deploymentAPIVersion": "[parameters('deploymentAPIVersion')]" + } + }, + "resources": [ + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "apim-connection-example2", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "projectResourceId": { + "value": "[parameters('projectResourceId')]" + }, + "connectionName": { + "value": "[variables('finalConnectionName')]" + }, + "apimResourceId": { + "value": "[parameters('apimResourceId')]" + }, + "apiName": { + "value": "[parameters('apiName')]" + }, + "apimSubscriptionName": { + "value": "[parameters('apimSubscriptionName')]" + }, + "authType": { + "value": "[parameters('authType')]" + }, + "isSharedToAll": { + "value": "[parameters('isSharedToAll')]" + }, + "metadata": { + "value": "[variables('example2Metadata')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "5161752358692660897" + } + }, + "parameters": { + "projectResourceId": { + "type": "string" + }, + "connectionName": { + "type": "string" + }, + "apimResourceId": { + "type": "string" + }, + "apiName": { + "type": "string" + }, + "apimSubscriptionName": { + "type": "string", + "defaultValue": "master" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey" + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "metadata": { + "type": "object" + } + }, + "variables": { + "aiFoundryName": "[split(parameters('projectResourceId'), '/')[8]]", + "projectName": "[split(parameters('projectResourceId'), '/')[10]]", + "apimSubscriptionId": "[split(parameters('apimResourceId'), '/')[2]]", + "apimResourceGroupName": "[split(parameters('apimResourceId'), '/')[4]]", + "apimServiceName": "[split(parameters('apimResourceId'), '/')[8]]" + }, + "resources": [ + { + "condition": "[equals(parameters('authType'), 'ApiKey')]", + "type": "Microsoft.CognitiveServices/accounts/projects/connections", + "apiVersion": "2025-04-01-preview", + "name": "[format('{0}/{1}/{2}', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]", + "properties": { + "category": "ApiManagement", + "target": "[format('{0}/{1}', reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service', variables('apimServiceName')), '2021-08-01').gatewayUrl, reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service/apis', variables('apimServiceName'), parameters('apiName')), '2021-08-01').path)]", + "authType": "ApiKey", + "isSharedToAll": "[parameters('isSharedToAll')]", + "credentials": { + "key": "[listSecrets(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service/subscriptions', variables('apimServiceName'), parameters('apimSubscriptionName')), '2021-08-01').primaryKey]" + }, + "metadata": "[parameters('metadata')]" + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[if(equals(parameters('authType'), 'ApiKey'), parameters('connectionName'), '')]" + }, + "connectionId": { + "type": "string", + "value": "[if(equals(parameters('authType'), 'ApiKey'), resourceId('Microsoft.CognitiveServices/accounts/projects/connections', variables('aiFoundryName'), variables('projectName'), parameters('connectionName')), '')]" + }, + "targetUrl": { + "type": "string", + "value": "[format('{0}/{1}', reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service', variables('apimServiceName')), '2021-08-01').gatewayUrl, reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service/apis', variables('apimServiceName'), parameters('apiName')), '2021-08-01').path)]" + }, + "authType": { + "type": "string", + "value": "[parameters('authType')]" + }, + "metadata": { + "type": "object", + "value": "[parameters('metadata')]" + } + } + } + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example2'), '2022-09-01').outputs.connectionName.value]" + }, + "connectionId": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example2'), '2022-09-01').outputs.connectionId.value]" + }, + "targetUrl": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example2'), '2022-09-01').outputs.targetUrl.value]" + }, + "authType": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example2'), '2022-09-01').outputs.authType.value]" + }, + "metadata": { + "type": "object", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example2'), '2022-09-01').outputs.metadata.value]" + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-dynamic-discovery.bicep b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-dynamic-discovery.bicep new file mode 100644 index 00000000..1227d626 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-dynamic-discovery.bicep @@ -0,0 +1,90 @@ +/* +Connections enable your AI applications to access tools and objects managed elsewhere in or outside of Azure. + +This example demonstrates how to add an Azure API Management connection for a specific API. +This implements Example 4 from the APIM Connection documentation: "APIM with Dynamic Discovery with OpenAI like deployments API" +Uses ApiKey authentication with dynamic model discovery using custom endpoints. + +Configuration includes: +- deploymentInPath: Controls how deployment names are passed to APIM gateway +- modelDiscovery: Custom endpoints for dynamic model discovery with OpenAI format + +Dynamic model discovery endpoints: +- List Models: v1/models +- Get Model: v1/models/{deploymentName} +- Provider: OpenAI format responses for deployments API + +IMPORTANT: Make sure you are logged into the subscription where the AI Foundry resource exists before deploying. +The connection will be created in the AI Foundry project, so you need to be in that subscription context. +Use: az account set --subscription +*/ + +param projectResourceId string = '/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project' +param apimResourceId string = '/subscriptions/87654321-4321-4321-4321-cba987654321/resourceGroups/rg-sample-apim/providers/Microsoft.ApiManagement/service/sample-apim-service' +param apiName string = 'foundry' +param apimSubscriptionName string = 'master' // Default subscription name in APIM, update it to your subscription name for apikey auth + +// Connection naming - can be overridden via parameter +param connectionName string = '' // Optional: specify custom connection name + +// Generate connection name if not provided +var apimServiceName = split(apimResourceId, '/')[8] +var generatedConnectionName = 'apim-${apimServiceName}-${apiName}-v4' +var finalConnectionName = connectionName != '' ? connectionName : generatedConnectionName + +// Connection configuration +@allowed([ + 'ApiKey' + 'AAD' +]) +param authType string = 'ApiKey' // Authentication type for the connection + +param isSharedToAll bool = false // Whether the connection should be shared to all users in the project + +// APIM-specific configuration parameters +@allowed([ + 'true' + 'false' +]) +param deploymentInPath string = 'false' // Controls how deployment names are passed to APIM gateway + +// Model discovery configuration (custom endpoints) +param listModelsEndpoint string = 'v1/models' // Custom endpoint for listing models +param getModelEndpoint string = 'v1/models/{deploymentName}' // Custom endpoint for getting specific model +param deploymentProvider string = 'OpenAI' // Provider format for response parsing + +// Build the modelDiscovery object and serialize it as JSON string +var modelDiscoveryObject = { + listModelsEndpoint: listModelsEndpoint + getModelEndpoint: getModelEndpoint + deploymentProvider: deploymentProvider +} + +// Build the metadata object for Example 3: APIM with Dynamic Discovery +// All values must be strings, including serialized JSON objects +var example3Metadata = { + deploymentInPath: deploymentInPath + modelDiscovery: string(modelDiscoveryObject) // Serialize as JSON string +} + +// Use the common module to create the APIM connection +module apimConnection 'modules/apim-connection-common.bicep' = { + name: 'apim-connection-example4' + params: { + projectResourceId: projectResourceId + connectionName: finalConnectionName + apimResourceId: apimResourceId + apiName: apiName + apimSubscriptionName: apimSubscriptionName + authType: authType + isSharedToAll: isSharedToAll + metadata: example3Metadata + } +} + +// Output information from the connection +output connectionName string = apimConnection.outputs.connectionName +output connectionId string = apimConnection.outputs.connectionId +output targetUrl string = apimConnection.outputs.targetUrl +output authType string = apimConnection.outputs.authType +output metadata object = apimConnection.outputs.metadata diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-dynamic-discovery.json b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-dynamic-discovery.json new file mode 100644 index 00000000..c07e26e6 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-dynamic-discovery.json @@ -0,0 +1,227 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "15998365188909223298" + } + }, + "parameters": { + "projectResourceId": { + "type": "string", + "defaultValue": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project" + }, + "apimResourceId": { + "type": "string", + "defaultValue": "/subscriptions/87654321-4321-4321-4321-cba987654321/resourceGroups/rg-sample-apim/providers/Microsoft.ApiManagement/service/sample-apim-service" + }, + "apiName": { + "type": "string", + "defaultValue": "foundry" + }, + "apimSubscriptionName": { + "type": "string", + "defaultValue": "master" + }, + "connectionName": { + "type": "string", + "defaultValue": "" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey", + "allowedValues": [ + "ApiKey", + "AAD" + ] + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "deploymentInPath": { + "type": "string", + "defaultValue": "false", + "allowedValues": [ + "true", + "false" + ] + }, + "listModelsEndpoint": { + "type": "string", + "defaultValue": "v1/models" + }, + "getModelEndpoint": { + "type": "string", + "defaultValue": "v1/models/{deploymentName}" + }, + "deploymentProvider": { + "type": "string", + "defaultValue": "OpenAI" + } + }, + "variables": { + "apimServiceName": "[split(parameters('apimResourceId'), '/')[8]]", + "generatedConnectionName": "[format('apim-{0}-{1}-v4', variables('apimServiceName'), parameters('apiName'))]", + "finalConnectionName": "[if(not(equals(parameters('connectionName'), '')), parameters('connectionName'), variables('generatedConnectionName'))]", + "modelDiscoveryObject": { + "listModelsEndpoint": "[parameters('listModelsEndpoint')]", + "getModelEndpoint": "[parameters('getModelEndpoint')]", + "deploymentProvider": "[parameters('deploymentProvider')]" + }, + "example3Metadata": { + "deploymentInPath": "[parameters('deploymentInPath')]", + "modelDiscovery": "[string(variables('modelDiscoveryObject'))]" + } + }, + "resources": [ + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "apim-connection-example4", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "projectResourceId": { + "value": "[parameters('projectResourceId')]" + }, + "connectionName": { + "value": "[variables('finalConnectionName')]" + }, + "apimResourceId": { + "value": "[parameters('apimResourceId')]" + }, + "apiName": { + "value": "[parameters('apiName')]" + }, + "apimSubscriptionName": { + "value": "[parameters('apimSubscriptionName')]" + }, + "authType": { + "value": "[parameters('authType')]" + }, + "isSharedToAll": { + "value": "[parameters('isSharedToAll')]" + }, + "metadata": { + "value": "[variables('example3Metadata')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "5161752358692660897" + } + }, + "parameters": { + "projectResourceId": { + "type": "string" + }, + "connectionName": { + "type": "string" + }, + "apimResourceId": { + "type": "string" + }, + "apiName": { + "type": "string" + }, + "apimSubscriptionName": { + "type": "string", + "defaultValue": "master" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey" + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "metadata": { + "type": "object" + } + }, + "variables": { + "aiFoundryName": "[split(parameters('projectResourceId'), '/')[8]]", + "projectName": "[split(parameters('projectResourceId'), '/')[10]]", + "apimSubscriptionId": "[split(parameters('apimResourceId'), '/')[2]]", + "apimResourceGroupName": "[split(parameters('apimResourceId'), '/')[4]]", + "apimServiceName": "[split(parameters('apimResourceId'), '/')[8]]" + }, + "resources": [ + { + "condition": "[equals(parameters('authType'), 'ApiKey')]", + "type": "Microsoft.CognitiveServices/accounts/projects/connections", + "apiVersion": "2025-04-01-preview", + "name": "[format('{0}/{1}/{2}', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]", + "properties": { + "category": "ApiManagement", + "target": "[format('{0}/{1}', reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service', variables('apimServiceName')), '2021-08-01').gatewayUrl, reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service/apis', variables('apimServiceName'), parameters('apiName')), '2021-08-01').path)]", + "authType": "ApiKey", + "isSharedToAll": "[parameters('isSharedToAll')]", + "credentials": { + "key": "[listSecrets(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service/subscriptions', variables('apimServiceName'), parameters('apimSubscriptionName')), '2021-08-01').primaryKey]" + }, + "metadata": "[parameters('metadata')]" + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[if(equals(parameters('authType'), 'ApiKey'), parameters('connectionName'), '')]" + }, + "connectionId": { + "type": "string", + "value": "[if(equals(parameters('authType'), 'ApiKey'), resourceId('Microsoft.CognitiveServices/accounts/projects/connections', variables('aiFoundryName'), variables('projectName'), parameters('connectionName')), '')]" + }, + "targetUrl": { + "type": "string", + "value": "[format('{0}/{1}', reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service', variables('apimServiceName')), '2021-08-01').gatewayUrl, reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service/apis', variables('apimServiceName'), parameters('apiName')), '2021-08-01').path)]" + }, + "authType": { + "type": "string", + "value": "[parameters('authType')]" + }, + "metadata": { + "type": "object", + "value": "[parameters('metadata')]" + } + } + } + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example4'), '2022-09-01').outputs.connectionName.value]" + }, + "connectionId": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example4'), '2022-09-01').outputs.connectionId.value]" + }, + "targetUrl": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example4'), '2022-09-01').outputs.targetUrl.value]" + }, + "authType": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example4'), '2022-09-01').outputs.authType.value]" + }, + "metadata": { + "type": "object", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example4'), '2022-09-01').outputs.metadata.value]" + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-static-models.bicep b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-static-models.bicep new file mode 100644 index 00000000..aff56633 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-static-models.bicep @@ -0,0 +1,107 @@ +/* +Connections enable your AI applications to access tools and objects managed elsewhere in or outside of Azure. + +This example demonstrates how to add an Azure API Management connection for a specific API. +This implements Example 5 from the APIM Connection documentation: "APIM with Static Model List" +Uses ApiKey authentication with predefined static list of models when deployment APIs are not available. + +Configuration includes: +- deploymentInPath: Controls how deployment names are passed to APIM gateway +- inferenceAPIVersion: API version for model inference calls (chat completions, embeddings, etc.) +- models: Static predefined list of available models (when deployment discovery APIs are not available) + +Static model configuration: +- No dynamic discovery endpoints needed +- Predefined model list with deployment names and model details +- Useful when APIM doesn't expose deployment discovery APIs + +IMPORTANT: Make sure you are logged into the subscription where the AI Foundry resource exists before deploying. +The connection will be created in the AI Foundry project, so you need to be in that subscription context. +Use: az account set --subscription +*/ + +param projectResourceId string = '/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project' +param apimResourceId string = '/subscriptions/87654321-4321-4321-4321-cba987654321/resourceGroups/rg-sample-apim/providers/Microsoft.ApiManagement/service/sample-apim-service' +param apiName string = 'foundry' +param apimSubscriptionName string = 'master' // Default subscription name in APIM, update it to your subscription name for apikey auth + +// Connection naming - can be overridden via parameter +param connectionName string = '' // Optional: specify custom connection name + +// Generate connection name if not provided +var apimServiceName = split(apimResourceId, '/')[8] +var generatedConnectionName = 'apim-${apimServiceName}-${apiName}-v5' +var finalConnectionName = connectionName != '' ? connectionName : generatedConnectionName + +// Connection configuration +@allowed([ + 'ApiKey' + 'AAD' +]) +param authType string = 'ApiKey' // Authentication type for the connection + +param isSharedToAll bool = false // Whether the connection should be shared to all users in the project + +// APIM-specific configuration parameters +@allowed([ + 'true' + 'false' +]) +param deploymentInPath string = 'false' // Controls how deployment names are passed to APIM gateway + +param inferenceAPIVersion string = '2024-02-01' // API version for inference calls (chat completions, embeddings, etc.) + +// Static model list configuration (when deployment APIs are not available) +// Accept static models as parameter - array of model objects +param staticModels array = [ + { + name: 'gpt-4-deployment' + properties: { + model: { + name: 'gpt-4' + version: '0613' + format: 'OpenAI' + } + } + } + { + name: 'gpt-35-turbo-deployment' + properties: { + model: { + name: 'gpt-35-turbo' + version: '0613' + format: 'OpenAI' + } + } + } +] + +// Build the metadata object for Example 5: APIM with Static Model List +// All values must be strings, including serialized JSON objects +var example5Metadata = { + deploymentInPath: deploymentInPath + inferenceAPIVersion: inferenceAPIVersion + models: string(staticModels) // Serialize static models array as JSON string +} + +// Use the common module to create the APIM connection +module apimConnection 'modules/apim-connection-common.bicep' = { + name: 'apim-connection-example5' + params: { + projectResourceId: projectResourceId + connectionName: finalConnectionName + apimResourceId: apimResourceId + apiName: apiName + apimSubscriptionName: apimSubscriptionName + authType: authType + isSharedToAll: isSharedToAll + metadata: example5Metadata + } +} + +// Output information from the connection +output connectionName string = apimConnection.outputs.connectionName +output connectionId string = apimConnection.outputs.connectionId +output targetUrl string = apimConnection.outputs.targetUrl +output authType string = apimConnection.outputs.authType +output metadata object = apimConnection.outputs.metadata diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-static-models.json b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-static-models.json new file mode 100644 index 00000000..1bc69a74 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/connection-apim-static-models.json @@ -0,0 +1,240 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "18152794205285043328" + } + }, + "parameters": { + "projectResourceId": { + "type": "string", + "defaultValue": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project" + }, + "apimResourceId": { + "type": "string", + "defaultValue": "/subscriptions/87654321-4321-4321-4321-cba987654321/resourceGroups/rg-sample-apim/providers/Microsoft.ApiManagement/service/sample-apim-service" + }, + "apiName": { + "type": "string", + "defaultValue": "foundry" + }, + "apimSubscriptionName": { + "type": "string", + "defaultValue": "master" + }, + "connectionName": { + "type": "string", + "defaultValue": "" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey", + "allowedValues": [ + "ApiKey", + "AAD" + ] + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "deploymentInPath": { + "type": "string", + "defaultValue": "false", + "allowedValues": [ + "true", + "false" + ] + }, + "inferenceAPIVersion": { + "type": "string", + "defaultValue": "2024-02-01" + }, + "staticModels": { + "type": "array", + "defaultValue": [ + { + "name": "gpt-4-deployment", + "properties": { + "model": { + "name": "gpt-4", + "version": "0613", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-35-turbo-deployment", + "properties": { + "model": { + "name": "gpt-35-turbo", + "version": "0613", + "format": "OpenAI" + } + } + } + ] + } + }, + "variables": { + "apimServiceName": "[split(parameters('apimResourceId'), '/')[8]]", + "generatedConnectionName": "[format('apim-{0}-{1}-v5', variables('apimServiceName'), parameters('apiName'))]", + "finalConnectionName": "[if(not(equals(parameters('connectionName'), '')), parameters('connectionName'), variables('generatedConnectionName'))]", + "example5Metadata": { + "deploymentInPath": "[parameters('deploymentInPath')]", + "inferenceAPIVersion": "[parameters('inferenceAPIVersion')]", + "models": "[string(parameters('staticModels'))]" + } + }, + "resources": [ + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "apim-connection-example5", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "projectResourceId": { + "value": "[parameters('projectResourceId')]" + }, + "connectionName": { + "value": "[variables('finalConnectionName')]" + }, + "apimResourceId": { + "value": "[parameters('apimResourceId')]" + }, + "apiName": { + "value": "[parameters('apiName')]" + }, + "apimSubscriptionName": { + "value": "[parameters('apimSubscriptionName')]" + }, + "authType": { + "value": "[parameters('authType')]" + }, + "isSharedToAll": { + "value": "[parameters('isSharedToAll')]" + }, + "metadata": { + "value": "[variables('example5Metadata')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "5161752358692660897" + } + }, + "parameters": { + "projectResourceId": { + "type": "string" + }, + "connectionName": { + "type": "string" + }, + "apimResourceId": { + "type": "string" + }, + "apiName": { + "type": "string" + }, + "apimSubscriptionName": { + "type": "string", + "defaultValue": "master" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey" + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "metadata": { + "type": "object" + } + }, + "variables": { + "aiFoundryName": "[split(parameters('projectResourceId'), '/')[8]]", + "projectName": "[split(parameters('projectResourceId'), '/')[10]]", + "apimSubscriptionId": "[split(parameters('apimResourceId'), '/')[2]]", + "apimResourceGroupName": "[split(parameters('apimResourceId'), '/')[4]]", + "apimServiceName": "[split(parameters('apimResourceId'), '/')[8]]" + }, + "resources": [ + { + "condition": "[equals(parameters('authType'), 'ApiKey')]", + "type": "Microsoft.CognitiveServices/accounts/projects/connections", + "apiVersion": "2025-04-01-preview", + "name": "[format('{0}/{1}/{2}', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]", + "properties": { + "category": "ApiManagement", + "target": "[format('{0}/{1}', reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service', variables('apimServiceName')), '2021-08-01').gatewayUrl, reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service/apis', variables('apimServiceName'), parameters('apiName')), '2021-08-01').path)]", + "authType": "ApiKey", + "isSharedToAll": "[parameters('isSharedToAll')]", + "credentials": { + "key": "[listSecrets(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service/subscriptions', variables('apimServiceName'), parameters('apimSubscriptionName')), '2021-08-01').primaryKey]" + }, + "metadata": "[parameters('metadata')]" + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[if(equals(parameters('authType'), 'ApiKey'), parameters('connectionName'), '')]" + }, + "connectionId": { + "type": "string", + "value": "[if(equals(parameters('authType'), 'ApiKey'), resourceId('Microsoft.CognitiveServices/accounts/projects/connections', variables('aiFoundryName'), variables('projectName'), parameters('connectionName')), '')]" + }, + "targetUrl": { + "type": "string", + "value": "[format('{0}/{1}', reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service', variables('apimServiceName')), '2021-08-01').gatewayUrl, reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', variables('apimSubscriptionId'), variables('apimResourceGroupName')), 'Microsoft.ApiManagement/service/apis', variables('apimServiceName'), parameters('apiName')), '2021-08-01').path)]" + }, + "authType": { + "type": "string", + "value": "[parameters('authType')]" + }, + "metadata": { + "type": "object", + "value": "[parameters('metadata')]" + } + } + } + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example5'), '2022-09-01').outputs.connectionName.value]" + }, + "connectionId": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example5'), '2022-09-01').outputs.connectionId.value]" + }, + "targetUrl": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example5'), '2022-09-01').outputs.targetUrl.value]" + }, + "authType": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example5'), '2022-09-01').outputs.authType.value]" + }, + "metadata": { + "type": "object", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'apim-connection-example5'), '2022-09-01').outputs.metadata.value]" + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/modules/apim-connection-common.bicep b/samples/microsoft/infrastructure-setup/01-connections/apim/modules/apim-connection-common.bicep new file mode 100644 index 00000000..190988f2 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/modules/apim-connection-common.bicep @@ -0,0 +1,96 @@ +/* +Common module for creating APIM connections to Azure AI Foundry projects. +This module handles the core connection logic and can be reused across different APIM connection samples. +*/ + +// Project resource parameters +param projectResourceId string +param connectionName string + +// APIM resource parameters +param apimResourceId string +param apiName string +param apimSubscriptionName string = 'master' + +// Connection configuration +param authType string = 'ApiKey' +param isSharedToAll bool = false + +// APIM-specific metadata (passed through from parent template) +param metadata object + +// Extract project information from resource ID +var aiFoundryName = split(projectResourceId, '/')[8] +var projectName = split(projectResourceId, '/')[10] + +// Extract APIM information from resource ID +var apimSubscriptionId = split(apimResourceId, '/')[2] +var apimResourceGroupName = split(apimResourceId, '/')[4] +var apimServiceName = split(apimResourceId, '/')[8] + +// Reference the AI Foundry account +resource aiFoundry 'Microsoft.CognitiveServices/accounts@2025-04-01-preview' existing = { + name: aiFoundryName + scope: resourceGroup() +} + +// Reference the project within the AI Foundry account +resource aiProject 'Microsoft.CognitiveServices/accounts/projects@2025-04-01-preview' existing = { + name: projectName + parent: aiFoundry +} + +// Reference the APIM service (can be in different resource group/subscription) +resource existingApim 'Microsoft.ApiManagement/service@2021-08-01' existing = { + name: apimServiceName + scope: resourceGroup(apimSubscriptionId, apimResourceGroupName) +} + +// Reference the specific API within APIM +resource apimApi 'Microsoft.ApiManagement/service/apis@2021-08-01' existing = { + name: apiName + parent: existingApim +} + +// Reference the APIM subscription to get keys (only for ApiKey auth) +resource apimSubscription 'Microsoft.ApiManagement/service/subscriptions@2021-08-01' existing = { + name: apimSubscriptionName + parent: existingApim +} + +// Create the connection with ApiKey authentication +resource connectionApiKey 'Microsoft.CognitiveServices/accounts/projects/connections@2025-04-01-preview' = if (authType == 'ApiKey') { + name: connectionName + parent: aiProject + properties: { + category: 'ApiManagement' + target: '${existingApim.properties.gatewayUrl}/${apimApi.properties.path}' + authType: 'ApiKey' + isSharedToAll: isSharedToAll + credentials: { + key: apimSubscription.listSecrets(apimSubscription.apiVersion).primaryKey + } + metadata: metadata + } +} + +// TODO: Future AAD connection (when role assignments are implemented) +// resource connectionAAD 'Microsoft.CognitiveServices/accounts/projects/connections@2025-04-01-preview' = if (authType == 'AAD') { +// name: connectionName +// parent: aiProject +// properties: { +// category: 'ApiManagement' +// target: '${existingApim.properties.gatewayUrl}/${apimApi.properties.path}' +// authType: 'AAD' +// isSharedToAll: isSharedToAll +// credentials: {} +// metadata: metadata +// } +// } + +// Outputs (only from the created connection) +output connectionName string = authType == 'ApiKey' ? connectionApiKey.name : '' +output connectionId string = authType == 'ApiKey' ? connectionApiKey.id : '' +output targetUrl string = '${existingApim.properties.gatewayUrl}/${apimApi.properties.path}' +output authType string = authType +output metadata object = metadata diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/parameters-basic.json b/samples/microsoft/infrastructure-setup/01-connections/apim/parameters-basic.json new file mode 100644 index 00000000..b550bed0 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/parameters-basic.json @@ -0,0 +1,30 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "projectResourceId": { + "value": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/sample-test/providers/Microsoft.CognitiveServices/accounts/sample-foundry/projects/firstProject" + }, + "apimResourceId": { + "value": "/subscriptions/87654321-4321-4321-4321-cba987654321/resourceGroups/rg-agent-test-westus2/providers/Microsoft.ApiManagement/service/rg-agent-aigateway-westus2" + }, + "apiName": { + "value": "foundry" + }, + "connectionName": { + "value": "my-apim-basic-connection" + }, + "authType": { + "value": "ApiKey" + }, + "isSharedToAll": { + "value": false + }, + "deploymentInPath": { + "value": "true" + }, + "inferenceAPIVersion": { + "value": "2025-01-01-preview" + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/parameters-deployment-api.json b/samples/microsoft/infrastructure-setup/01-connections/apim/parameters-deployment-api.json new file mode 100644 index 00000000..1a999841 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/parameters-deployment-api.json @@ -0,0 +1,33 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "projectResourceId": { + "value": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/sample-test/providers/Microsoft.CognitiveServices/accounts/sample-foundry/projects/firstProject" + }, + "apimResourceId": { + "value": "/subscriptions/87654321-4321-4321-4321-cba987654321/resourceGroups/rg-agent-test-westus2/providers/Microsoft.ApiManagement/service/rg-agent-aigateway-westus2" + }, + "apiName": { + "value": "foundry" + }, + "connectionName": { + "value": "my-apim-deployment-api-connection" + }, + "authType": { + "value": "ApiKey" + }, + "isSharedToAll": { + "value": false + }, + "deploymentInPath": { + "value": "true" + }, + "inferenceAPIVersion": { + "value": "2025-01-01-preview" + }, + "deploymentAPIVersion": { + "value": "2023-05-15" + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/parameters-dynamic.json b/samples/microsoft/infrastructure-setup/01-connections/apim/parameters-dynamic.json new file mode 100644 index 00000000..9831fdcd --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/parameters-dynamic.json @@ -0,0 +1,36 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "projectResourceId": { + "value": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/sample-test/providers/Microsoft.CognitiveServices/accounts/sample-foundry/projects/firstProject" + }, + "apimResourceId": { + "value": "/subscriptions/87654321-4321-4321-4321-cba987654321/resourceGroups/rg-agent-test-westus2/providers/Microsoft.ApiManagement/service/rg-agent-aigateway-westus2" + }, + "apiName": { + "value": "foundry" + }, + "connectionName": { + "value": "my-apim-dynamic-discovery-connection" + }, + "authType": { + "value": "ApiKey" + }, + "isSharedToAll": { + "value": false + }, + "deploymentInPath": { + "value": "true" + }, + "listModelsEndpoint": { + "value": "v1/models" + }, + "getModelEndpoint": { + "value": "v1/models/{deploymentName}" + }, + "deploymentProvider": { + "value": "OpenAI" + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/apim/parameters-static.json b/samples/microsoft/infrastructure-setup/01-connections/apim/parameters-static.json new file mode 100644 index 00000000..29b480e8 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/apim/parameters-static.json @@ -0,0 +1,64 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "projectResourceId": { + "value": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/sample-test/providers/Microsoft.CognitiveServices/accounts/sample-foundry/projects/firstProject" + }, + "apimResourceId": { + "value": "/subscriptions/87654321-4321-4321-4321-cba987654321/resourceGroups/rg-agent-test-westus2/providers/Microsoft.ApiManagement/service/rg-agent-aigateway-westus2" + }, + "apiName": { + "value": "foundry" + }, + "connectionName": { + "value": "my-apim-static-connection" + }, + "authType": { + "value": "ApiKey" + }, + "isSharedToAll": { + "value": false + }, + "deploymentInPath": { + "value": "true" + }, + "inferenceAPIVersion": { + "value": "2025-01-01-preview" + }, + "staticModels": { + "value": [ + { + "name": "gpt-4-deployment", + "properties": { + "model": { + "name": "gpt-4", + "version": "0613", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-35-turbo-deployment", + "properties": { + "model": { + "name": "gpt-35-turbo", + "version": "0613", + "format": "OpenAI" + } + } + }, + { + "name": "text-embedding-ada-002-deployment", + "properties": { + "model": { + "name": "text-embedding-ada-002", + "version": "2", + "format": "OpenAI" + } + } + } + ] + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/ModelGateway-Connection-Objects.md b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/ModelGateway-Connection-Objects.md new file mode 100644 index 00000000..4ceb93de --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/ModelGateway-Connection-Objects.md @@ -0,0 +1,495 @@ +# ModelGateway Connection JSON Examples + +## Overview + +ModelGateway connections provide a unified interface for connecting to various AI model providers through the Azure ML workspace connection framework. These connections support both static model configuration (predefined models) and dynamic model discovery (runtime model detection). + +### Key Features + +- **Unified API**: Single connection interface for multiple AI providers (Azure AI, OpenAI, MuleSoft, etc.) +- **Authentication**: Support for API key authentication with workspace credential management +- **Discovery Patterns**: Choose between static model lists or dynamic discovery endpoints +- **Provider Abstraction**: Consistent model format regardless of underlying provider +- **Enterprise Integration**: Support for enterprise gateways like MuleSoft for multi-provider scenarios + +### Connection Categories + +All ModelGateway connections use the `"category": "ModelGateway"` to ensure proper routing through the ModelGateway service infrastructure. + +### Discovery Methods + +**Static Discovery**: Models are predefined in the connection metadata using the `models` array. Best for: +- Dynamic discovery not possible +- Fixed model deployments +- Known model configurations +- Enterprise scenarios with approved model lists + +**Dynamic Discovery**: Models are discovered at runtime using API endpoints defined in `modelDiscovery`. Best for: +- Frequently changing model deployments +- Provider-managed model catalogs +- Development and testing scenarios + +### Authentication + +All examples use `"authType": "ApiKey"` with workspace-managed credentials. The actual API keys are stored securely and referenced through the credential system. We will expand to more auth methods in upcoming releases. + +## Connection Schema Definitions + +### 1. ModelDiscovery (Dynamic Discovery) + +The `modelDiscovery` object enables runtime model detection through API endpoints. Azure Agents combines this configuration with the connection's `target` URL and `credentials` to make discovery calls. + +```json +{ + "modelDiscovery": { + "listModelsEndpoint": "/v1/models", + "getModelEndpoint": "/v1/models/{deploymentName}", + "deploymentProvider": "OpenAI" + } +} +``` + +**Fields:** +- `listModelsEndpoint` - Endpoint to retrieve all available models (relative to target URL) +- `getModelEndpoint` - Endpoint to get specific model details with `{deploymentName}` placeholder +- `deploymentProvider` - Provider format for response parsing. **Supported values: `"OpenAI"` and `"AzureOpenAI"`** (exactly 2 formats) + +**How Azure Agents Uses It:** +1. Constructs full URL: `{target}{listModelsEndpoint}` +2. Adds authentication headers from `credentials` +3. Makes HTTP request to discover available models +4. Parses response based on `deploymentProvider` format (OpenAI or AzureOpenAI) + +**Supported DeploymentProvider Formats:** + +We support exactly **2 deployment API formats** for model discovery: + +**1. OpenAI Format Responses:** + +*List Models Response (`listModelsEndpoint`):`* +```json +{ + "data": [ + { + "id": "gpt-4", + "object": "model", + "created": 1687882411, + "owned_by": "openai" + }, + { + "id": "gpt-3.5-turbo", + "object": "model", + "created": 1677610602, + "owned_by": "openai" + } + ] +} +``` + +*Get Model by Name Response (`getModelEndpoint`):`* +```json +{ + "id": "gpt-4", + "object": "model", + "created": 1687882411, + "owned_by": "openai" +} +``` + +- Uses `data` array for list, single object for get-by-name +- `id` serves as both deployment name and model name +- No version information provided in API responses + +**2. AzureOpenAI Format Responses:** + +*List Deployments Response (`listModelsEndpoint`):`* +```json +{ + "value": [ + { + "name": "gpt-4-deployment", + "properties": { + "model": { + "format": "OpenAI", + "name": "gpt-4", + "version": "0613" + } + } + }, + { + "id": "/subscriptions/.../deployments/gpt-35-turbo-deployment", + "name": "gpt-35-turbo-deployment", + "type": "Microsoft.CognitiveServices/accounts/deployments", + "properties": { + "model": { + "format": "OpenAI", + "name": "gpt-35-turbo", + "version": "0613" + }, + "provisioningState": "Succeeded" + } + } + ] +} +``` + +*Get Deployment by Name Response (`getModelEndpoint`):`* +```json +{ + "name": "gpt-4-deployment", + "properties": { + "model": { + "format": "OpenAI", + "name": "gpt-4", + "version": "0613" + } + } +} +``` + +- Uses `value` array for list, single object for get-by-name +- Follows Azure ARM resource structure +- Separate deployment `name` and model details in `properties.model` +- Includes model `name`, `version`, and `format` + +### 2. Static Discovery + +Static discovery uses a predefined `models` array in metadata. Models are defined using the `ModelInfo` structure: + +```json +{ + "models": [ + { + "name": "deployment-name", + "properties": { + "model": { + "name": "model-name", + "version": "model-version", + "format": "OpenAI" + } + } + } + ] +} +``` + +**Structure:** +- `name` - Deployment name (how you reference the model in API calls) +- `properties.model.name` - Actual model name from provider +- `properties.model.version` - Model version identifier +- `properties.model.format` - Provider format (OpenAI, DeepSeek etc) + +### 3. InferenceAPIVersion + +Specifies the API version for model inference calls (chat completions, etc.). + +```json +{ + "inferenceAPIVersion": "2025-03-01" +} +``` + +**Usage by Azure Agents:** +- Appended as query parameter: `?api-version=2025-03-01` +- Used for all inference requests (chat completions, not discovery or management calls) + +### 4. DeploymentInPath + +Controls how deployment names are passed to the provider API. + +```json +{ + "deploymentInPath": true // or false +} +``` + +**When `true` (Path-based routing):** +``` +URL: {target}/deployments/{deploymentName}/chat/completions +``` + +**When `false`:** +``` +URL: {target}/chat/completions +Body: {"model": "{deploymentName}"} +``` + +**Azure Agents Behavior:** +- `true`: Injects deployment name into URL path +- `false`: Passes deployment name via model parameter + +### 5. DeploymentAPIVersion + +Specifies the API version for deployment management calls (listing deployments, getting deployment details). + +```json +{ + "deploymentAPIVersion": "2025-03-01" +} +``` + +**Usage by Azure Agents:** +- Used only for `modelDiscovery` endpoint calls +- Separate from `inferenceAPIVersion` to allow different versioning +- Appended as query parameter to discovery endpoints + +**Complete Example with All Schema Elements:** + +```json +{ + "metadata": { + "modelDiscovery": { + "listModelsEndpoint": "/openai/deployments", + "getModelEndpoint": "/openai/deployments/{deploymentName}", + "deploymentProvider": "AzureOpenAI" + }, + "deploymentInPath": true, + "inferenceAPIVersion": "2025-03-01", + "deploymentAPIVersion": "2025-03-01" + } +} +``` + +## Basic ModelGateway Connection with Dynamic Discovery + +```json +{ + "id": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/connections/basic-gateway-connection", + "name": "basic-gateway-connection", + "type": "Microsoft.MachineLearningServices/workspaces/connections", + "properties": { + "category": "ModelGateway", + "target": "https://api.openai.com", + "authType": "ApiKey", + "credentials": { + "key": "{api-key-reference}" + }, + "metadata": { + "modelDiscovery": { + "listModelsEndpoint": "/v1/models", + "getModelEndpoint": "/v1/models/{deploymentName}", + "deploymentProvider": "OpenAI" + } + } + } +} +``` + +## OpenAI Connection with Static Models + +```json +{ + "id": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/connections/openai-static-connection", + "name": "openai-static-connection", + "type": "Microsoft.MachineLearningServices/workspaces/connections", + "properties": { + "category": "ModelGateway", + "target": "https://api.openai.com", + "authType": "ApiKey", + "credentials": { + "key": "{openai-api-key-reference}" + }, + "metadata": { + "models": [ + { + "name": "gpt-4", + "properties": { + "model": { + "name": "gpt-4", + "version": "0613", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-3.5-turbo", + "properties": { + "model": { + "name": "gpt-3.5-turbo", + "version": "0613", + "format": "OpenAI" + } + } + }, + { + "name": "text-embedding-ada-002", + "properties": { + "model": { + "name": "text-embedding-ada-002", + "version": "2", + "format": "OpenAI" + } + } + } + ] + } + } +} +``` + +## Azure OpenAI Connection with Dynamic Discovery + +```json +{ + "id": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/connections/azure-openai-dynamic", + "name": "azure-openai-dynamic", + "type": "Microsoft.MachineLearningServices/workspaces/connections", + "properties": { + "category": "ModelGateway", + "target": "https://your-resource.openai.azure.com/openai", + "authType": "ApiKey", + "credentials": { + "key": "{azure-openai-key-reference}" + }, + "metadata": { + "modelDiscovery": { + "listModelsEndpoint": "/deployments", + "getModelEndpoint": "/deployments/{deploymentName}", + "deploymentProvider": "AzureOpenAI" + }, + "deploymentInPath": true, + "inferenceAPIVersion": "2024-02-01", + "deploymentAPIVersion": "2024-02-01" + } + } +} +``` + +## MuleSoft Multi-Provider Connection with Static Models + +```json +{ + "id": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/connections/mulesoft-multi-provider", + "name": "mulesoft-multi-provider", + "type": "Microsoft.MachineLearningServices/workspaces/connections", + "properties": { + "category": "ModelGateway", + "target": "https://api.mulesoft.enterprise.com/llm-gateway", + "authType": "ApiKey", + "credentials": { + "key": "{mulesoft-api-key-reference}" + }, + "metadata": { + "models": [ + { + "name": "openai-gpt-4", + "properties": { + "model": { + "name": "gpt-4", + "version": "0613", + "format": "OpenAI" + } + } + }, + { + "name": "openai-gpt-3.5-turbo", + "properties": { + "model": { + "name": "gpt-3.5-turbo", + "version": "0613", + "format": "OpenAI" + } + } + }, + { + "name": "deepseek-v1-deploy", + "properties": { + "model": { + "name": "deepseek-v1", + "format": "Deepseek" + } + } + } + ], + "deploymentInPath": true, + "inferenceAPIVersion": "2025-03-01" + } + } +} +``` + +## MuleSoft Dynamic Discovery Connection + +```json +{ + "id": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/connections/mulesoft-dynamic-discovery", + "name": "mulesoft-dynamic-discovery", + "type": "Microsoft.MachineLearningServices/workspaces/connections", + "properties": { + "category": "ModelGateway", + "target": "https://enterprise-ai-gateway.mulesoft.com/api", + "authType": "ApiKey", + "credentials": { + "key": "{mulesoft-enterprise-key-reference}" + }, + "metadata": { + "modelDiscovery": { + "listModelsEndpoint": "/v2/models", + "getModelEndpoint": "/v2/models/{deploymentName}", + "deploymentProvider": "AzureOpenAI" + }, + "deploymentInPath": "true", + "inferenceAPIVersion": "2025-03-01", + "deploymentAPIVersion": "2025-03-01" + } + } +} +``` + +## AuthConfig (Optional) + +The `authConfig` metadata field allows customization of authentication headers sent to the model provider. This is useful for providers that require specific header formats or additional authentication information. + +### AuthConfig Fields + +- `type`: Must be "api_key" for API key authentication +- `name`: Custom header name for the API key +- `format`: Template for the header value (defaults to "{api_key}" if not provided) + +### AuthConfig Examples + +#### Custom Header Name and Template +```json +{ + "metadata": { + "authConfig": "serialized({ + "type": "api_key", + "name": "x-api-key", + "format": "Key {api_key}" + })" + } +} +``` + +#### OpenAI Bearer Format (Default) +```json +{ + "metadata": { + "authConfig": "serialized({ + "type": "api_key", + "name": "Authorization", + "format": "Bearer {api_key}" + })" + } +} +``` + +#### Custom Provider Format +```json +{ + "metadata": { + "authConfig": "serialized({ + "type": "api_key", + "name": "X-Custom-Auth", + "format": "Custom-Token {api_key}" + })" + } +} +``` + +### Important Notes + +- All complex metadata values (objects and arrays) must be stored as JSON strings. +- Simple string values like "true", "false", or API versions can remain as regular strings +- The `{api_key}` placeholder in the format field will be replaced with the actual API key at runtime +- If `authConfig` is not provided, the default api-key format will be used \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/README.md b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/README.md new file mode 100644 index 00000000..c85c32a7 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/README.md @@ -0,0 +1,52 @@ +# ModelGateway Connection Examples + +This folder contains Azure Bicep templates for creating ModelGateway connections to Azure AI Foundry projects. + +## ⚠️ Important Notice + +**ModelGateway connections are currently not supported in Azure AI Foundry.** These templates are provided as examples for future use when ModelGateway support becomes available. + +## Prerequisites + +1. **Azure CLI** installed and configured +2. **AI Foundry account and project** already created + +## How to Deploy (When Supported) + +### Basic ModelGateway Connection +```bash +# 1. Edit parameters-basic.json with your resource IDs +# 2. Deploy using the parameters file (API key will be prompted) +az deployment group create \ + --resource-group \ + --template-file connection-modelgateway-basic.bicep \ + --parameters @parameters-basic.json +``` + +### Dynamic Discovery ModelGateway Connection +```bash +# 1. Edit parameters-dynamic.json with your resource IDs +# 2. Deploy using the parameters file (API key will be prompted) +az deployment group create \ + --resource-group \ + --template-file connection-modelgateway-dynamic.bicep \ + --parameters @parameters-dynamic.json +``` + +### Static Models ModelGateway Connection +```bash +# 1. Edit parameters-static.json with your resource IDs +# 2. Deploy using the parameters file (API key will be prompted) +az deployment group create \ + --resource-group \ + --template-file connection-modelgateway-static.bicep \ + --parameters @parameters-static.json +``` + +## Parameter Files + +- `parameters-basic.json`: For basic ModelGateway connections +- `parameters-dynamic.json`: For dynamic discovery connections +- `parameters-static.json`: For static model list connections + +Edit these files to update the resource IDs and target URLs for your environment. API keys will be prompted securely during deployment. \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-basic.bicep b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-basic.bicep new file mode 100644 index 00000000..1a115548 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-basic.bicep @@ -0,0 +1,92 @@ +/* +Connections enable your AI applications to access tools and objects managed elsewhere in or outside of Azure. + +This example demonstrates how to add a ModelGateway connection for with dynamic discovery. +ModelGateway connections provide a unified interface for various AI model providers. +Uses ApiKey authentication with dynamic model discovery using Azure OpenAI format deployment endpoints. + +Configuration includes: +- deploymentInPath: Controls how deployment names are passed to the gateway +- inferenceAPIVersion: API version for model inference calls +- deploymentAPIVersion: API version for deployment management calls +- modelDiscovery: Dynamic endpoints for model discovery with AzureOpenAI deployment format + +Dynamic model discovery endpoints: +- List Deployments: /deployments +- Get Deployment: /deployments/{deploymentName} +- Provider: AzureOpenAI format responses + +IMPORTANT: Make sure you are logged into the subscription where the AI Foundry resource exists before deploying. +The connection will be created in the AI Foundry project, so you need to be in that subscription context. +Use: az account set --subscription +*/ + +param projectResourceId string = '/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project' +param targetUrl string = 'https://sample-target-url/xyz' +param gatewayName string = 'your-gateway-name' + +// Connection naming - can be overridden via parameter +param connectionName string = '' // Optional: specify custom connection name + +// Connection configuration (ModelGateway only supports ApiKey) +param authType string = 'ApiKey' +param isSharedToAll bool = false + +// API key for the Azure OpenAI resource +@secure() +param apiKey string + +// Generate connection name if not provided +var generatedConnectionName = 'modelgateway-${gatewayName}' +var finalConnectionName = connectionName != '' ? connectionName : generatedConnectionName + +// ModelGateway-specific configuration parameters for Azure OpenAI +@allowed([ + 'true' + 'false' +]) +param deploymentInPath string = 'true' // inference will include /deployments/{deploymentName} prefix in path +param inferenceAPIVersion string = '2025-03-01' // API version for inference calls +param deploymentAPIVersion string = '2025-03-01' // API version for deployment management calls + +// Model discovery configuration (deployment endpoints) +param listModelsEndpoint string = '/deployments' // list deployments endpoint +param getModelEndpoint string = '/deployments/{deploymentName}' // Get deployment endpoint +param deploymentProvider string = 'AzureOpenAI' // Provider format for response parsing + +// Build the modelDiscovery object and serialize it as JSON string +var modelDiscoveryObject = { + listModelsEndpoint: listModelsEndpoint + getModelEndpoint: getModelEndpoint + deploymentProvider: deploymentProvider +} + +// Build the metadata object for ModelGateway +// All values must be strings, including serialized JSON objects +var modelGatewayMetadata = { + deploymentInPath: deploymentInPath + inferenceAPIVersion: inferenceAPIVersion + deploymentAPIVersion: deploymentAPIVersion + modelDiscovery: string(modelDiscoveryObject) // Serialize as JSON string +} + +// Use the common module to create the ModelGateway connection +module modelGatewayConnection 'modules/modelgateway-connection-common.bicep' = { + name: 'modelgateway-connection-azure-openai' + params: { + projectResourceId: projectResourceId + connectionName: finalConnectionName + targetUrl: targetUrl + authType: authType + isSharedToAll: isSharedToAll + apiKey: apiKey + metadata: modelGatewayMetadata + } +} + +// Output information from the connection +output connectionName string = modelGatewayConnection.outputs.connectionName +output connectionId string = modelGatewayConnection.outputs.connectionId +output targetUrl string = modelGatewayConnection.outputs.targetUrl +output authType string = modelGatewayConnection.outputs.authType +output metadata object = modelGatewayConnection.outputs.metadata diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-basic.json b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-basic.json new file mode 100644 index 00000000..a0e40d00 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-basic.json @@ -0,0 +1,220 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "2263091858047082864" + } + }, + "parameters": { + "projectResourceId": { + "type": "string", + "defaultValue": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project" + }, + "targetUrl": { + "type": "string", + "defaultValue": "https://sample-target-url/xyz" + }, + "gatewayName": { + "type": "string", + "defaultValue": "your-gateway-name" + }, + "connectionName": { + "type": "string", + "defaultValue": "" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey" + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "apiKey": { + "type": "securestring" + }, + "deploymentInPath": { + "type": "string", + "defaultValue": "true", + "allowedValues": [ + "true", + "false" + ] + }, + "inferenceAPIVersion": { + "type": "string", + "defaultValue": "2025-03-01" + }, + "deploymentAPIVersion": { + "type": "string", + "defaultValue": "2025-03-01" + }, + "listModelsEndpoint": { + "type": "string", + "defaultValue": "/deployments" + }, + "getModelEndpoint": { + "type": "string", + "defaultValue": "/deployments/{deploymentName}" + }, + "deploymentProvider": { + "type": "string", + "defaultValue": "AzureOpenAI" + } + }, + "variables": { + "generatedConnectionName": "[format('modelgateway-{0}', parameters('gatewayName'))]", + "finalConnectionName": "[if(not(equals(parameters('connectionName'), '')), parameters('connectionName'), variables('generatedConnectionName'))]", + "modelDiscoveryObject": { + "listModelsEndpoint": "[parameters('listModelsEndpoint')]", + "getModelEndpoint": "[parameters('getModelEndpoint')]", + "deploymentProvider": "[parameters('deploymentProvider')]" + }, + "modelGatewayMetadata": { + "deploymentInPath": "[parameters('deploymentInPath')]", + "inferenceAPIVersion": "[parameters('inferenceAPIVersion')]", + "deploymentAPIVersion": "[parameters('deploymentAPIVersion')]", + "modelDiscovery": "[string(variables('modelDiscoveryObject'))]" + } + }, + "resources": [ + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "modelgateway-connection-azure-openai", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "projectResourceId": { + "value": "[parameters('projectResourceId')]" + }, + "connectionName": { + "value": "[variables('finalConnectionName')]" + }, + "targetUrl": { + "value": "[parameters('targetUrl')]" + }, + "authType": { + "value": "[parameters('authType')]" + }, + "isSharedToAll": { + "value": "[parameters('isSharedToAll')]" + }, + "apiKey": { + "value": "[parameters('apiKey')]" + }, + "metadata": { + "value": "[variables('modelGatewayMetadata')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "10077070729719288643" + } + }, + "parameters": { + "projectResourceId": { + "type": "string" + }, + "connectionName": { + "type": "string" + }, + "targetUrl": { + "type": "string" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey" + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "apiKey": { + "type": "securestring" + }, + "metadata": { + "type": "object" + } + }, + "variables": { + "aiFoundryName": "[split(parameters('projectResourceId'), '/')[8]]", + "projectName": "[split(parameters('projectResourceId'), '/')[10]]" + }, + "resources": [ + { + "type": "Microsoft.CognitiveServices/accounts/projects/connections", + "apiVersion": "2025-04-01-preview", + "name": "[format('{0}/{1}/{2}', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]", + "properties": { + "category": "ModelGateway", + "target": "[parameters('targetUrl')]", + "authType": "ApiKey", + "isSharedToAll": "[parameters('isSharedToAll')]", + "credentials": { + "key": "[parameters('apiKey')]" + }, + "metadata": "[parameters('metadata')]" + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[parameters('connectionName')]" + }, + "connectionId": { + "type": "string", + "value": "[resourceId('Microsoft.CognitiveServices/accounts/projects/connections', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]" + }, + "targetUrl": { + "type": "string", + "value": "[parameters('targetUrl')]" + }, + "authType": { + "type": "string", + "value": "[parameters('authType')]" + }, + "metadata": { + "type": "object", + "value": "[parameters('metadata')]" + } + } + } + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-azure-openai'), '2022-09-01').outputs.connectionName.value]" + }, + "connectionId": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-azure-openai'), '2022-09-01').outputs.connectionId.value]" + }, + "targetUrl": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-azure-openai'), '2022-09-01').outputs.targetUrl.value]" + }, + "authType": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-azure-openai'), '2022-09-01').outputs.authType.value]" + }, + "metadata": { + "type": "object", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-azure-openai'), '2022-09-01').outputs.metadata.value]" + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-custom-auth-config.bicep b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-custom-auth-config.bicep new file mode 100644 index 00000000..00ed443e --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-custom-auth-config.bicep @@ -0,0 +1,97 @@ +/* +Connections enable your AI applications to access tools and objects managed elsewhere in or outside of Azure. + +This example demonstrates how to add a ModelGateway connection with custom authentication configuration. +This shows how to configure the connection to send the API key as an Authorization header with Bearer format. +Uses ApiKey authentication with authConfig to customize authentication headers. + +Configuration includes: +- authConfig: Custom authentication configuration to send API key as "Authorization: Bearer {api_key}" +- deploymentInPath: Controls how deployment names are passed to the gateway +- inferenceAPIVersion: API version for model inference calls +- modelDiscovery: Dynamic endpoints for model discovery with OpenAI format + +Use case: When the target service expects API key authentication via Authorization Bearer header +instead of the default authentication methods. + +IMPORTANT: Make sure you are logged into the subscription where the AI Foundry resource exists before deploying. +The connection will be created in the AI Foundry project, so you need to be in that subscription context. +Use: az account set --subscription +*/ + +param projectResourceId string = '/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project' +param targetUrl string = 'https://api.openai.com' +param gatewayName string = 'custom-auth-gateway' + +// Connection configuration (ModelGateway only supports ApiKey) +param authType string = 'ApiKey' +param isSharedToAll bool = false + +// Connection naming - can be overridden via parameter +param connectionName string = '' // Optional: specify custom connection name + +// API key for the ModelGateway endpoint +@secure() +param apiKey string + +// Generate connection name if not provided +var generatedConnectionName = 'modelgateway-${gatewayName}-custom-auth' +var finalConnectionName = connectionName != '' ? connectionName : generatedConnectionName + +// ModelGateway-specific configuration parameters +@allowed([ + 'true' + 'false' +]) +param deploymentInPath string = 'false' // Controls how deployment names are passed to the gateway + +param inferenceAPIVersion string = '' // API version for inference calls + +// Model discovery configuration (dynamic endpoints) +param listModelsEndpoint string = '/v1/models' // Endpoint for listing models +param getModelEndpoint string = '/v1/models/{deploymentName}' // Endpoint for getting specific model +param deploymentProvider string = 'OpenAI' // Provider format for response parsing + +// Custom authentication configuration (Bearer token format) - from documentation +param authConfig object = { + type: 'api_key' + name: 'Authorization' + format: 'Bearer {api_key}' +} + +// Build the modelDiscovery object and serialize it as JSON string +var modelDiscoveryObject = { + listModelsEndpoint: listModelsEndpoint + getModelEndpoint: getModelEndpoint + deploymentProvider: deploymentProvider +} + +// Build the metadata object for ModelGateway Custom Auth Configuration +// All values must be strings, including serialized JSON objects +var customAuthMetadata = { + deploymentInPath: deploymentInPath + inferenceAPIVersion: inferenceAPIVersion + modelDiscovery: string(modelDiscoveryObject) // Serialize as JSON string + authConfig: string(authConfig) // Serialize as JSON string - from documentation +} + +// Use the common module to create the ModelGateway connection +module modelGatewayConnection 'modules/modelgateway-connection-common.bicep' = { + name: 'modelgateway-connection-custom-auth' + params: { + projectResourceId: projectResourceId + connectionName: finalConnectionName + targetUrl: targetUrl + authType: authType + isSharedToAll: isSharedToAll + apiKey: apiKey + metadata: customAuthMetadata + } +} + +// Output information from the connection +output connectionName string = modelGatewayConnection.outputs.connectionName +output connectionId string = modelGatewayConnection.outputs.connectionId +output targetUrl string = modelGatewayConnection.outputs.targetUrl +output authType string = modelGatewayConnection.outputs.authType +output metadata object = modelGatewayConnection.outputs.metadata diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-custom-auth-config.json b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-custom-auth-config.json new file mode 100644 index 00000000..ff8e4917 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-custom-auth-config.json @@ -0,0 +1,224 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "17175229699754795474" + } + }, + "parameters": { + "projectResourceId": { + "type": "string", + "defaultValue": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project" + }, + "targetUrl": { + "type": "string", + "defaultValue": "https://api.openai.com" + }, + "gatewayName": { + "type": "string", + "defaultValue": "custom-auth-gateway" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey" + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "connectionName": { + "type": "string", + "defaultValue": "" + }, + "apiKey": { + "type": "securestring" + }, + "deploymentInPath": { + "type": "string", + "defaultValue": "false", + "allowedValues": [ + "true", + "false" + ] + }, + "inferenceAPIVersion": { + "type": "string", + "defaultValue": "" + }, + "listModelsEndpoint": { + "type": "string", + "defaultValue": "/v1/models" + }, + "getModelEndpoint": { + "type": "string", + "defaultValue": "/v1/models/{deploymentName}" + }, + "deploymentProvider": { + "type": "string", + "defaultValue": "OpenAI" + }, + "authConfig": { + "type": "object", + "defaultValue": { + "type": "api_key", + "name": "Authorization", + "format": "Bearer {api_key}" + } + } + }, + "variables": { + "generatedConnectionName": "[format('modelgateway-{0}-custom-auth', parameters('gatewayName'))]", + "finalConnectionName": "[if(not(equals(parameters('connectionName'), '')), parameters('connectionName'), variables('generatedConnectionName'))]", + "modelDiscoveryObject": { + "listModelsEndpoint": "[parameters('listModelsEndpoint')]", + "getModelEndpoint": "[parameters('getModelEndpoint')]", + "deploymentProvider": "[parameters('deploymentProvider')]" + }, + "customAuthMetadata": { + "deploymentInPath": "[parameters('deploymentInPath')]", + "inferenceAPIVersion": "[parameters('inferenceAPIVersion')]", + "modelDiscovery": "[string(variables('modelDiscoveryObject'))]", + "authConfig": "[string(parameters('authConfig'))]" + } + }, + "resources": [ + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "modelgateway-connection-custom-auth", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "projectResourceId": { + "value": "[parameters('projectResourceId')]" + }, + "connectionName": { + "value": "[variables('finalConnectionName')]" + }, + "targetUrl": { + "value": "[parameters('targetUrl')]" + }, + "authType": { + "value": "[parameters('authType')]" + }, + "isSharedToAll": { + "value": "[parameters('isSharedToAll')]" + }, + "apiKey": { + "value": "[parameters('apiKey')]" + }, + "metadata": { + "value": "[variables('customAuthMetadata')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "10077070729719288643" + } + }, + "parameters": { + "projectResourceId": { + "type": "string" + }, + "connectionName": { + "type": "string" + }, + "targetUrl": { + "type": "string" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey" + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "apiKey": { + "type": "securestring" + }, + "metadata": { + "type": "object" + } + }, + "variables": { + "aiFoundryName": "[split(parameters('projectResourceId'), '/')[8]]", + "projectName": "[split(parameters('projectResourceId'), '/')[10]]" + }, + "resources": [ + { + "type": "Microsoft.CognitiveServices/accounts/projects/connections", + "apiVersion": "2025-04-01-preview", + "name": "[format('{0}/{1}/{2}', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]", + "properties": { + "category": "ModelGateway", + "target": "[parameters('targetUrl')]", + "authType": "ApiKey", + "isSharedToAll": "[parameters('isSharedToAll')]", + "credentials": { + "key": "[parameters('apiKey')]" + }, + "metadata": "[parameters('metadata')]" + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[parameters('connectionName')]" + }, + "connectionId": { + "type": "string", + "value": "[resourceId('Microsoft.CognitiveServices/accounts/projects/connections', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]" + }, + "targetUrl": { + "type": "string", + "value": "[parameters('targetUrl')]" + }, + "authType": { + "type": "string", + "value": "[parameters('authType')]" + }, + "metadata": { + "type": "object", + "value": "[parameters('metadata')]" + } + } + } + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-custom-auth'), '2022-09-01').outputs.connectionName.value]" + }, + "connectionId": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-custom-auth'), '2022-09-01').outputs.connectionId.value]" + }, + "targetUrl": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-custom-auth'), '2022-09-01').outputs.targetUrl.value]" + }, + "authType": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-custom-auth'), '2022-09-01').outputs.authType.value]" + }, + "metadata": { + "type": "object", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-custom-auth'), '2022-09-01').outputs.metadata.value]" + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-dynamic.bicep b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-dynamic.bicep new file mode 100644 index 00000000..66bedfd0 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-dynamic.bicep @@ -0,0 +1,93 @@ +/* +Connections enable your AI applications to access tools and objects managed elsewhere in or outside of Azure. + +This example demonstrates how to add a ModelGateway connection with dynamic model discovery. +ModelGateway connections provide a unified interface for various AI model providers. +Uses ApiKey authentication with dynamic model discovery using API endpoints. + +Configuration includes: +- deploymentInPath: Controls how deployment names are passed to the gateway in inference api calls +- inferenceAPIVersion: API version for model inference calls +- deploymentAPIVersion: API version for deployment management calls +- modelDiscovery: Dynamic endpoints for model discovery with OpenAI format + +Dynamic model discovery endpoints: +- List Models: /v1/models +- Get Model: /v1/models/{deploymentName} +- Provider: OpenAI format responses + +IMPORTANT: Make sure you are logged into the subscription where the AI Foundry resource exists before deploying. +The connection will be created in the AI Foundry project, so you need to be in that subscription context. +Use: az account set --subscription +*/ + +param projectResourceId string = '/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project' +param targetUrl string = 'https://your-model-gateway.example.com' +param gatewayName string = 'example-gateway' + +// Connection configuration (ModelGateway only supports ApiKey) +param authType string = 'ApiKey' +param isSharedToAll bool = false + +// Connection naming - can be overridden via parameter +param connectionName string = '' // Optional: specify custom connection name + +// API key for the ModelGateway endpoint +@secure() +param apiKey string + +// Generate connection name if not provided +var generatedConnectionName = 'modelgateway-${gatewayName}-dynamic' +var finalConnectionName = connectionName != '' ? connectionName : generatedConnectionName + +// ModelGateway-specific configuration parameters +@allowed([ + 'true' + 'false' +]) +param deploymentInPath string = 'false' // Controls how deployment names are passed to the gateway + +param inferenceAPIVersion string = '' // API version for inference calls +param deploymentAPIVersion string = '' // API version for deployment management calls + +// Model discovery configuration (dynamic endpoints) +param listModelsEndpoint string = '/v1/models' // Endpoint for listing models +param getModelEndpoint string = '/v1/models/{deploymentName}' // Endpoint for getting specific model +param deploymentProvider string = 'OpenAI' // Provider format for response parsing + +// Build the modelDiscovery object and serialize it as JSON string +var modelDiscoveryObject = { + listModelsEndpoint: listModelsEndpoint + getModelEndpoint: getModelEndpoint + deploymentProvider: deploymentProvider +} + +// Build the metadata object for ModelGateway Dynamic Discovery +// All values must be strings, including serialized JSON objects +var modelGatewayMetadata = { + deploymentInPath: deploymentInPath + inferenceAPIVersion: inferenceAPIVersion + deploymentAPIVersion: deploymentAPIVersion + modelDiscovery: string(modelDiscoveryObject) // Serialize as JSON string +} + +// Use the common module to create the ModelGateway connection +module modelGatewayConnection 'modules/modelgateway-connection-common.bicep' = { + name: 'modelgateway-connection-dynamic' + params: { + projectResourceId: projectResourceId + connectionName: finalConnectionName + targetUrl: targetUrl + authType: authType + isSharedToAll: isSharedToAll + apiKey: apiKey + metadata: modelGatewayMetadata + } +} + +// Output information from the connection +output connectionName string = modelGatewayConnection.outputs.connectionName +output connectionId string = modelGatewayConnection.outputs.connectionId +output targetUrl string = modelGatewayConnection.outputs.targetUrl +output authType string = modelGatewayConnection.outputs.authType +output metadata object = modelGatewayConnection.outputs.metadata diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-dynamic.json b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-dynamic.json new file mode 100644 index 00000000..5c54418c --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-dynamic.json @@ -0,0 +1,220 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "10941455783336539685" + } + }, + "parameters": { + "projectResourceId": { + "type": "string", + "defaultValue": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project" + }, + "targetUrl": { + "type": "string", + "defaultValue": "https://your-model-gateway.example.com" + }, + "gatewayName": { + "type": "string", + "defaultValue": "example-gateway" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey" + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "connectionName": { + "type": "string", + "defaultValue": "" + }, + "apiKey": { + "type": "securestring" + }, + "deploymentInPath": { + "type": "string", + "defaultValue": "false", + "allowedValues": [ + "true", + "false" + ] + }, + "inferenceAPIVersion": { + "type": "string", + "defaultValue": "" + }, + "deploymentAPIVersion": { + "type": "string", + "defaultValue": "" + }, + "listModelsEndpoint": { + "type": "string", + "defaultValue": "/v1/models" + }, + "getModelEndpoint": { + "type": "string", + "defaultValue": "/v1/models/{deploymentName}" + }, + "deploymentProvider": { + "type": "string", + "defaultValue": "OpenAI" + } + }, + "variables": { + "generatedConnectionName": "[format('modelgateway-{0}-dynamic', parameters('gatewayName'))]", + "finalConnectionName": "[if(not(equals(parameters('connectionName'), '')), parameters('connectionName'), variables('generatedConnectionName'))]", + "modelDiscoveryObject": { + "listModelsEndpoint": "[parameters('listModelsEndpoint')]", + "getModelEndpoint": "[parameters('getModelEndpoint')]", + "deploymentProvider": "[parameters('deploymentProvider')]" + }, + "modelGatewayMetadata": { + "deploymentInPath": "[parameters('deploymentInPath')]", + "inferenceAPIVersion": "[parameters('inferenceAPIVersion')]", + "deploymentAPIVersion": "[parameters('deploymentAPIVersion')]", + "modelDiscovery": "[string(variables('modelDiscoveryObject'))]" + } + }, + "resources": [ + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "modelgateway-connection-dynamic", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "projectResourceId": { + "value": "[parameters('projectResourceId')]" + }, + "connectionName": { + "value": "[variables('finalConnectionName')]" + }, + "targetUrl": { + "value": "[parameters('targetUrl')]" + }, + "authType": { + "value": "[parameters('authType')]" + }, + "isSharedToAll": { + "value": "[parameters('isSharedToAll')]" + }, + "apiKey": { + "value": "[parameters('apiKey')]" + }, + "metadata": { + "value": "[variables('modelGatewayMetadata')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "10077070729719288643" + } + }, + "parameters": { + "projectResourceId": { + "type": "string" + }, + "connectionName": { + "type": "string" + }, + "targetUrl": { + "type": "string" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey" + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "apiKey": { + "type": "securestring" + }, + "metadata": { + "type": "object" + } + }, + "variables": { + "aiFoundryName": "[split(parameters('projectResourceId'), '/')[8]]", + "projectName": "[split(parameters('projectResourceId'), '/')[10]]" + }, + "resources": [ + { + "type": "Microsoft.CognitiveServices/accounts/projects/connections", + "apiVersion": "2025-04-01-preview", + "name": "[format('{0}/{1}/{2}', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]", + "properties": { + "category": "ModelGateway", + "target": "[parameters('targetUrl')]", + "authType": "ApiKey", + "isSharedToAll": "[parameters('isSharedToAll')]", + "credentials": { + "key": "[parameters('apiKey')]" + }, + "metadata": "[parameters('metadata')]" + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[parameters('connectionName')]" + }, + "connectionId": { + "type": "string", + "value": "[resourceId('Microsoft.CognitiveServices/accounts/projects/connections', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]" + }, + "targetUrl": { + "type": "string", + "value": "[parameters('targetUrl')]" + }, + "authType": { + "type": "string", + "value": "[parameters('authType')]" + }, + "metadata": { + "type": "object", + "value": "[parameters('metadata')]" + } + } + } + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-dynamic'), '2022-09-01').outputs.connectionName.value]" + }, + "connectionId": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-dynamic'), '2022-09-01').outputs.connectionId.value]" + }, + "targetUrl": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-dynamic'), '2022-09-01').outputs.targetUrl.value]" + }, + "authType": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-dynamic'), '2022-09-01').outputs.authType.value]" + }, + "metadata": { + "type": "object", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-dynamic'), '2022-09-01').outputs.metadata.value]" + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-static.bicep b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-static.bicep new file mode 100644 index 00000000..d1ee6b70 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-static.bicep @@ -0,0 +1,112 @@ +/* +Connections enable your AI applications to access tools and objects managed elsewhere in or outside of Azure. + +This example demonstrates how to add a ModelGateway connection with static model discovery. +ModelGateway connections provide a unified interface for various AI model providers. +Uses ApiKey authentication with predefined static list of models. + +Configuration includes: +- deploymentInPath: Controls how deployment names are passed to the gateway +- inferenceAPIVersion: API version for model inference calls (chat completions, embeddings, etc.) +- models: Static predefined list of available models + +Static model configuration: +- No dynamic discovery endpoints needed +- Predefined model list with deployment names and model details +- Useful when dynamic discovery is not available or desired + +IMPORTANT: Make sure you are logged into the subscription where the AI Foundry resource exists before deploying. +The connection will be created in the AI Foundry project, so you need to be in that subscription context. +Use: az account set --subscription +*/ + +param projectResourceId string = '/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project' +param targetUrl string = 'https://your-model-gateway.example.com/v1' +param gatewayName string = 'example-gateway' + +// Connection configuration (ModelGateway only supports ApiKey) +param authType string = 'ApiKey' +param isSharedToAll bool = false + +// Connection naming - can be overridden via parameter +param connectionName string = '' // Optional: specify custom connection name + +// API key for the ModelGateway endpoint +@secure() +param apiKey string + +// Generate connection name if not provided +var generatedConnectionName = 'modelgateway-${gatewayName}-static' +var finalConnectionName = connectionName != '' ? connectionName : generatedConnectionName + +// ModelGateway-specific configuration parameters +@allowed([ + 'true' + 'false' +]) +param deploymentInPath string = 'false' // Controls how deployment names are passed to the gateway + +param inferenceAPIVersion string = '2024-02-01' // API version for inference calls + +// Static model list configuration - accept as parameter +param staticModels array = [ + { + name: 'gpt-4' + properties: { + model: { + name: 'gpt-4' + version: '0613' + format: 'OpenAI' + } + } + } + { + name: 'gpt-3.5-turbo' + properties: { + model: { + name: 'gpt-3.5-turbo' + version: '0613' + format: 'OpenAI' + } + } + } + { + name: 'text-embedding-ada-002' + properties: { + model: { + name: 'text-embedding-ada-002' + version: '2' + format: 'OpenAI' + } + } + } +] + +// Build the metadata object for ModelGateway Static Models +// All values must be strings, including serialized JSON objects +var modelGatewayMetadata = { + deploymentInPath: deploymentInPath + inferenceAPIVersion: inferenceAPIVersion + models: string(staticModels) // Serialize static models array as JSON string +} + +// Use the common module to create the ModelGateway connection +module modelGatewayConnection 'modules/modelgateway-connection-common.bicep' = { + name: 'modelgateway-connection-static' + params: { + projectResourceId: projectResourceId + connectionName: finalConnectionName + targetUrl: targetUrl + authType: authType + isSharedToAll: isSharedToAll + apiKey: apiKey + metadata: modelGatewayMetadata + } +} + +// Output information from the connection +output connectionName string = modelGatewayConnection.outputs.connectionName +output connectionId string = modelGatewayConnection.outputs.connectionId +output targetUrl string = modelGatewayConnection.outputs.targetUrl +output authType string = modelGatewayConnection.outputs.authType +output metadata object = modelGatewayConnection.outputs.metadata diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-static.json b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-static.json new file mode 100644 index 00000000..771b98bc --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/connection-modelgateway-static.json @@ -0,0 +1,233 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "12322628902215141096" + } + }, + "parameters": { + "projectResourceId": { + "type": "string", + "defaultValue": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/rg-sample/providers/Microsoft.CognitiveServices/accounts/sample-foundry-account/projects/sample-project" + }, + "targetUrl": { + "type": "string", + "defaultValue": "https://your-model-gateway.example.com/v1" + }, + "gatewayName": { + "type": "string", + "defaultValue": "example-gateway" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey" + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "connectionName": { + "type": "string", + "defaultValue": "" + }, + "apiKey": { + "type": "securestring" + }, + "deploymentInPath": { + "type": "string", + "defaultValue": "false", + "allowedValues": [ + "true", + "false" + ] + }, + "inferenceAPIVersion": { + "type": "string", + "defaultValue": "2024-02-01" + }, + "staticModels": { + "type": "array", + "defaultValue": [ + { + "name": "gpt-4", + "properties": { + "model": { + "name": "gpt-4", + "version": "0613", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-3.5-turbo", + "properties": { + "model": { + "name": "gpt-3.5-turbo", + "version": "0613", + "format": "OpenAI" + } + } + }, + { + "name": "text-embedding-ada-002", + "properties": { + "model": { + "name": "text-embedding-ada-002", + "version": "2", + "format": "OpenAI" + } + } + } + ] + } + }, + "variables": { + "generatedConnectionName": "[format('modelgateway-{0}-static', parameters('gatewayName'))]", + "finalConnectionName": "[if(not(equals(parameters('connectionName'), '')), parameters('connectionName'), variables('generatedConnectionName'))]", + "modelGatewayMetadata": { + "deploymentInPath": "[parameters('deploymentInPath')]", + "inferenceAPIVersion": "[parameters('inferenceAPIVersion')]", + "models": "[string(parameters('staticModels'))]" + } + }, + "resources": [ + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "modelgateway-connection-static", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "projectResourceId": { + "value": "[parameters('projectResourceId')]" + }, + "connectionName": { + "value": "[variables('finalConnectionName')]" + }, + "targetUrl": { + "value": "[parameters('targetUrl')]" + }, + "authType": { + "value": "[parameters('authType')]" + }, + "isSharedToAll": { + "value": "[parameters('isSharedToAll')]" + }, + "apiKey": { + "value": "[parameters('apiKey')]" + }, + "metadata": { + "value": "[variables('modelGatewayMetadata')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "10077070729719288643" + } + }, + "parameters": { + "projectResourceId": { + "type": "string" + }, + "connectionName": { + "type": "string" + }, + "targetUrl": { + "type": "string" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey" + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "apiKey": { + "type": "securestring" + }, + "metadata": { + "type": "object" + } + }, + "variables": { + "aiFoundryName": "[split(parameters('projectResourceId'), '/')[8]]", + "projectName": "[split(parameters('projectResourceId'), '/')[10]]" + }, + "resources": [ + { + "type": "Microsoft.CognitiveServices/accounts/projects/connections", + "apiVersion": "2025-04-01-preview", + "name": "[format('{0}/{1}/{2}', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]", + "properties": { + "category": "ModelGateway", + "target": "[parameters('targetUrl')]", + "authType": "ApiKey", + "isSharedToAll": "[parameters('isSharedToAll')]", + "credentials": { + "key": "[parameters('apiKey')]" + }, + "metadata": "[parameters('metadata')]" + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[parameters('connectionName')]" + }, + "connectionId": { + "type": "string", + "value": "[resourceId('Microsoft.CognitiveServices/accounts/projects/connections', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]" + }, + "targetUrl": { + "type": "string", + "value": "[parameters('targetUrl')]" + }, + "authType": { + "type": "string", + "value": "[parameters('authType')]" + }, + "metadata": { + "type": "object", + "value": "[parameters('metadata')]" + } + } + } + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-static'), '2022-09-01').outputs.connectionName.value]" + }, + "connectionId": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-static'), '2022-09-01').outputs.connectionId.value]" + }, + "targetUrl": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-static'), '2022-09-01').outputs.targetUrl.value]" + }, + "authType": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-static'), '2022-09-01').outputs.authType.value]" + }, + "metadata": { + "type": "object", + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'modelgateway-connection-static'), '2022-09-01').outputs.metadata.value]" + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/modules/modelgateway-connection-common.bicep b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/modules/modelgateway-connection-common.bicep new file mode 100644 index 00000000..0492b169 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/modules/modelgateway-connection-common.bicep @@ -0,0 +1,62 @@ +/* +Common module for creating ModelGateway connections to Azure AI Foundry projects. +This module handles the core connection logic and can be reused across different ModelGateway connection samples. +ModelGateway connections support ApiKey authentication. +*/ + +// Project resource parameters +param projectResourceId string +param connectionName string + +// ModelGateway target configuration +param targetUrl string + +// Connection configuration (ModelGateway only supports ApiKey) +param authType string = 'ApiKey' +param isSharedToAll bool = false + +// API key for the ModelGateway endpoint +@secure() +param apiKey string + +// ModelGateway-specific metadata (passed through from parent template) +param metadata object + +// Extract project information from resource ID +var aiFoundryName = split(projectResourceId, '/')[8] +var projectName = split(projectResourceId, '/')[10] + +// Reference the AI Foundry account +resource aiFoundry 'Microsoft.CognitiveServices/accounts@2025-04-01-preview' existing = { + name: aiFoundryName + scope: resourceGroup() +} + +// Reference the project within the AI Foundry account +resource aiProject 'Microsoft.CognitiveServices/accounts/projects@2025-04-01-preview' existing = { + name: projectName + parent: aiFoundry +} + +// Create the ModelGateway connection with ApiKey authentication +resource connectionApiKey 'Microsoft.CognitiveServices/accounts/projects/connections@2025-04-01-preview' = { + name: connectionName + parent: aiProject + properties: { + category: 'ModelGateway' + target: targetUrl + authType: 'ApiKey' + isSharedToAll: isSharedToAll + credentials: { + key: apiKey + } + metadata: metadata + } +} + +// Outputs +output connectionName string = connectionApiKey.name +output connectionId string = connectionApiKey.id +output targetUrl string = targetUrl +output authType string = authType +output metadata object = metadata diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/modules/modelgateway-connection-common.json b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/modules/modelgateway-connection-common.json new file mode 100644 index 00000000..774d4ca4 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/modules/modelgateway-connection-common.json @@ -0,0 +1,79 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.37.4.10188", + "templateHash": "10077070729719288643" + } + }, + "parameters": { + "projectResourceId": { + "type": "string" + }, + "connectionName": { + "type": "string" + }, + "targetUrl": { + "type": "string" + }, + "authType": { + "type": "string", + "defaultValue": "ApiKey" + }, + "isSharedToAll": { + "type": "bool", + "defaultValue": false + }, + "apiKey": { + "type": "securestring" + }, + "metadata": { + "type": "object" + } + }, + "variables": { + "aiFoundryName": "[split(parameters('projectResourceId'), '/')[8]]", + "projectName": "[split(parameters('projectResourceId'), '/')[10]]" + }, + "resources": [ + { + "type": "Microsoft.CognitiveServices/accounts/projects/connections", + "apiVersion": "2025-04-01-preview", + "name": "[format('{0}/{1}/{2}', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]", + "properties": { + "category": "ModelGateway", + "target": "[parameters('targetUrl')]", + "authType": "ApiKey", + "isSharedToAll": "[parameters('isSharedToAll')]", + "credentials": { + "key": "[parameters('apiKey')]" + }, + "metadata": "[parameters('metadata')]" + } + } + ], + "outputs": { + "connectionName": { + "type": "string", + "value": "[parameters('connectionName')]" + }, + "connectionId": { + "type": "string", + "value": "[resourceId('Microsoft.CognitiveServices/accounts/projects/connections', variables('aiFoundryName'), variables('projectName'), parameters('connectionName'))]" + }, + "targetUrl": { + "type": "string", + "value": "[parameters('targetUrl')]" + }, + "authType": { + "type": "string", + "value": "[parameters('authType')]" + }, + "metadata": { + "type": "object", + "value": "[parameters('metadata')]" + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/parameters-basic.json b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/parameters-basic.json new file mode 100644 index 00000000..9cb5cd68 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/parameters-basic.json @@ -0,0 +1,24 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "projectResourceId": { + "value": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/sample-test/providers/Microsoft.CognitiveServices/accounts/sample-foundry/projects/firstProject" + }, + "targetUrl": { + "value": "https://api.openai.com" + }, + "gatewayName": { + "value": "openai-basic" + }, + "connectionName": { + "value": "my-openai-basic-connection" + }, + "authType": { + "value": "ApiKey" + }, + "isSharedToAll": { + "value": false + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/parameters-custom-auth-config.json b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/parameters-custom-auth-config.json new file mode 100644 index 00000000..907228ac --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/parameters-custom-auth-config.json @@ -0,0 +1,46 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "projectResourceId": { + "value": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/sample-test/providers/Microsoft.CognitiveServices/accounts/sample-foundry/projects/firstProject" + }, + "targetUrl": { + "value": "https://api.openai.com" + }, + "gatewayName": { + "value": "custom-auth-gateway" + }, + "connectionName": { + "value": "my-openai-custom-auth-connection" + }, + "authType": { + "value": "ApiKey" + }, + "isSharedToAll": { + "value": false + }, + "deploymentInPath": { + "value": "false" + }, + "listModelsEndpoint": { + "value": "/v1/models" + }, + "getModelEndpoint": { + "value": "/v1/models/{deploymentName}" + }, + "deploymentProvider": { + "value": "OpenAI" + }, + "inferenceAPIVersion": { + "value": "2024-02-01" + }, + "authConfig": { + "value": { + "type": "api_key", + "name": "Authorization", + "format": "Bearer {api_key}" + } + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/parameters-dynamic.json b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/parameters-dynamic.json new file mode 100644 index 00000000..af724fc8 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/parameters-dynamic.json @@ -0,0 +1,30 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "projectResourceId": { + "value": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/sample-test/providers/Microsoft.CognitiveServices/accounts/sample-foundry/projects/firstProject" + }, + "targetUrl": { + "value": "https://api.openai.com" + }, + "gatewayName": { + "value": "openai-dynamic" + }, + "connectionName": { + "value": "my-openai-dynamic-connection" + }, + "authType": { + "value": "ApiKey" + }, + "isSharedToAll": { + "value": false + }, + "deploymentInPath": { + "value": "true" + }, + "inferenceAPIVersion": { + "value": "2024-02-01" + } + } +} \ No newline at end of file diff --git a/samples/microsoft/infrastructure-setup/01-connections/model-gateway/parameters-static.json b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/parameters-static.json new file mode 100644 index 00000000..e970bba5 --- /dev/null +++ b/samples/microsoft/infrastructure-setup/01-connections/model-gateway/parameters-static.json @@ -0,0 +1,193 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "projectResourceId": { + "value": "/subscriptions/12345678-1234-1234-1234-123456789abc/resourceGroups/sample-test/providers/Microsoft.CognitiveServices/accounts/sample-foundry/projects/firstProject" + }, + "targetUrl": { + "value": "https://api.openai.com" + }, + "gatewayName": { + "value": "openai-static" + }, + "connectionName": { + "value": "my-openai-static-connection" + }, + "authType": { + "value": "ApiKey" + }, + "isSharedToAll": { + "value": false + }, + "deploymentInPath": { + "value": "false" + }, + "inferenceAPIVersion": { + "value": "" + }, + "staticModels": { + "value": [ + { + "name": "gpt-4o-1", + "properties": { + "model": { + "name": "gpt-4o", + "version": "2024-11-20", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-2", + "properties": { + "model": { + "name": "gpt-4o", + "version": "2024-08-06", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-3", + "properties": { + "model": { + "name": "gpt-4o", + "version": "2024-05-13", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-4", + "properties": { + "model": { + "name": "gpt-4o-mini", + "version": "2024-07-18", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-5", + "properties": { + "model": { + "name": "gpt-4o-realtime-preview", + "version": "2024-10-01", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-6", + "properties": { + "model": { + "name": "gpt-4-turbo", + "version": "2024-04-09", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-7", + "properties": { + "model": { + "name": "gpt-4", + "version": "0613", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-8", + "properties": { + "model": { + "name": "gpt-35-turbo", + "version": "0125", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-9", + "properties": { + "model": { + "name": "gpt-35-turbo-16k", + "version": "0613", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-10", + "properties": { + "model": { + "name": "text-embedding-3-large", + "version": "1", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-11", + "properties": { + "model": { + "name": "text-embedding-3-small", + "version": "1", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-12", + "properties": { + "model": { + "name": "text-embedding-ada-002", + "version": "2", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-13", + "properties": { + "model": { + "name": "dall-e-3", + "version": "3.0", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-14", + "properties": { + "model": { + "name": "whisper", + "version": "001", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o-15", + "properties": { + "model": { + "name": "tts-1", + "version": "001", + "format": "OpenAI" + } + } + }, + { + "name": "gpt-4o", + "properties": { + "model": { + "name": "gpt-4o", + "format": "OpenAI" + } + } + } + ] + } + } +} \ No newline at end of file