Skip to content

azure-ai-foundry.md

Latest commit

 

History

History
260 lines (196 loc) · 8.27 KB

azure-ai-foundry.md

File metadata and controls

260 lines (196 loc) · 8.27 KB

OpenChat Playground with Azure AI Foundry

This page describes how to run OpenChat Playground (OCP) with Azure AI Foundry integration.

Get the repository root

  1. Get the repository root.

    # bash/zsh
REPOSITORY_ROOT=$(git rev-parse --show-toplevel)
    # PowerShell
$REPOSITORY_ROOT = git rev-parse --show-toplevel

Run on local machine

  1. Make sure you are at the repository root.

    cd $REPOSITORY_ROOT

  2. Add Azure AI Foundry API Key for Azure AI Foundry connection. Make sure you should replace {{AZURE_AI_FOUNDRY_API_KEY}} with your Azure AI Foundry API key.

    # bash/zsh
dotnet user-secrets --project $REPOSITORY_ROOT/src/OpenChat.PlaygroundApp \
    set AzureAIFoundry:ApiKey "{{AZURE_AI_FOUNDRY_API_KEY}}"
    # PowerShell
dotnet user-secrets --project $REPOSITORY_ROOT/src/OpenChat.PlaygroundApp `
    set AzureAIFoundry:ApiKey "{{AZURE_AI_FOUNDRY_API_KEY}}"

    To get an Azure AI Foundry instance, its API endpoint and key, refer to the doc, Get started with Azure AI Foundry.

  3. Run the app by passing the endpoint parameter {{AZURE_AI_FOUNDRY_ENDPOINT}} that was acquired from Azure AI Foundry portal. The default deployment name OCP uses is gpt-4o-mini.

    # bash/zsh
dotnet run --project $REPOSITORY_ROOT/src/OpenChat.PlaygroundApp -- \
    --connector-type AzureAIFoundry \
    --endpoint "{{AZURE_AI_FOUNDRY_ENDPOINT}}"
    # PowerShell
dotnet run --project $REPOSITORY_ROOT/src/OpenChat.PlaygroundApp -- `
    --connector-type AzureAIFoundry `
    --endpoint "{{AZURE_AI_FOUNDRY_ENDPOINT}}"

    Alternatively, if you want to run with a different deployment, say gpt-4, other than the default one, you can specify it as an argument. For more details on how to create deployments, refer to Create and deploy a model:

    # bash/zsh
dotnet run --project $REPOSITORY_ROOT/src/OpenChat.PlaygroundApp -- \
    --connector-type AzureAIFoundry \
    --endpoint "{{AZURE_AI_FOUNDRY_ENDPOINT}}" \
    --deployment-name gpt-4
    # PowerShell
dotnet run --project $REPOSITORY_ROOT/src/OpenChat.PlaygroundApp -- `
    --connector-type AzureAIFoundry `
    --endpoint "{{AZURE_AI_FOUNDRY_ENDPOINT}}" `
    --deployment-name gpt-4

  4. Open your web browser, navigate to http://localhost:5280, and enter prompts.

Run in local container

  1. Make sure that you have the Azure AI Foundry Endpoint URL.

  2. Make sure you are at the repository root.

    cd $REPOSITORY_ROOT

  3. Build a container.

    docker build -f Dockerfile -t openchat-playground:latest .

  4. Get Azure AI Foundry API Key.

    # bash/zsh
API_KEY=$(dotnet user-secrets --project ./src/OpenChat.PlaygroundApp list --json | \
          sed -n '/^\/\//d; p' | jq -r '."AzureAIFoundry:ApiKey"')
    # PowerShell
$API_KEY = (dotnet user-secrets --project ./src/OpenChat.PlaygroundApp list --json | `
            Select-String -NotMatch '^//(BEGIN|END)' | ConvertFrom-Json).'AzureAIFoundry:ApiKey'

  5. Run the app. The default deployment name OCP uses is gpt-4o-mini.

    # bash/zsh - from locally built container
docker run -i --rm -p 8080:8080 openchat-playground:latest \
    --connector-type AzureAIFoundry \
    --endpoint "{{AZURE_AI_FOUNDRY_ENDPOINT}}" \
    --api-key $API_KEY
    # PowerShell - from locally built container
docker run -i --rm -p 8080:8080 openchat-playground:latest `
    --connector-type AzureAIFoundry `
    --endpoint "{{AZURE_AI_FOUNDRY_ENDPOINT}}" `
    --api-key $API_KEY
    # bash/zsh - from GitHub Container Registry
docker run -i --rm -p 8080:8080 ghcr.io/aliencube/open-chat-playground/openchat-playground:latest \
    --connector-type AzureAIFoundry \
    --endpoint "{{AZURE_AI_FOUNDRY_ENDPOINT}}" \
    --api-key $API_KEY
    # PowerShell - from GitHub Container Registry
docker run -i --rm -p 8080:8080 ghcr.io/aliencube/open-chat-playground/openchat-playground:latest `
    --connector-type AzureAIFoundry `
    --endpoint "{{AZURE_AI_FOUNDRY_ENDPOINT}}" `
    --api-key $API_KEY `

    Alternatively, if you want to run with a different deployment, say gpt-4, other than the default one, you can specify it as an argument:

    # bash/zsh - from locally built container
docker run -i --rm -p 8080:8080 openchat-playground:latest \
    --connector-type AzureAIFoundry \
    --endpoint "{{AZURE_AI_FOUNDRY_ENDPOINT}}" \
    --api-key $API_KEY \
    --deployment-name gpt-4
    # PowerShell - from locally built container
docker run -i --rm -p 8080:8080 openchat-playground:latest `
    --connector-type AzureAIFoundry `
    --endpoint "{{AZURE_AI_FOUNDRY_ENDPOINT}}" `
    --api-key $API_KEY `
    --deployment-name gpt-4
    # bash/zsh - from GitHub Container Registry
docker run -i --rm -p 8080:8080 ghcr.io/aliencube/open-chat-playground/openchat-playground:latest \
    --connector-type AzureAIFoundry \
    --endpoint "{{AZURE_AI_FOUNDRY_ENDPOINT}}" \
    --api-key $API_KEY \
    --deployment-name gpt-4
    # PowerShell - from GitHub Container Registry
docker run -i --rm -p 8080:8080 ghcr.io/aliencube/open-chat-playground/openchat-playground:latest `
    --connector-type AzureAIFoundry `
    --endpoint "{{AZURE_AI_FOUNDRY_ENDPOINT}}" `
    --api-key $API_KEY `
    --deployment-name gpt-4

  6. Open your web browser, navigate to http://localhost:8080, and enter prompts.

Run on Azure

  1. Make sure that you have the Azure AI Foundry Endpoint URL.

  2. Make sure you are at the repository root.

    cd $REPOSITORY_ROOT

  3. Login to Azure.

    azd auth login

  4. Check login status.

    azd auth login --check-status

  5. Initialize azd template.

    azd init

    NOTE: You will be asked to provide environment name for provisioning.

  6. Get Azure AI Foundry API Key.

    # bash/zsh
API_KEY=$(dotnet user-secrets --project ./src/OpenChat.PlaygroundApp list --json | \
             sed -n '/^\/\//d; p' | jq -r '."AzureAIFoundry:ApiKey"')
    # PowerShell
$API_KEY = (dotnet user-secrets --project ./src/OpenChat.PlaygroundApp list --json | `
              Select-String -NotMatch '^//(BEGIN|END)' | ConvertFrom-Json).'AzureAIFoundry:ApiKey'

  7. Set Azure AI Foundry configuration to azd environment variables.

    azd env set AZURE_AI_FOUNDRY_ENDPOINT "{{AZURE_AI_FOUNDRY_ENDPOINT}}"
azd env set AZURE_AI_FOUNDRY_API_KEY $API_KEY

    The default deployment name OCP uses is gpt-4o-mini. If you want to run with a different deployment, say gpt-4, other than the default one, add it to azd environment variables.

    azd env set AZURE_AI_FOUNDRY_DEPLOYMENT_NAME gpt-4

  8. Set the connector type to AzureAIFoundry.

    azd env set CONNECTOR_TYPE AzureAIFoundry

  9. Run the following commands in order to provision and deploy the app.

    azd up

    NOTE: You will be asked to provide Azure subscription and location for deployment.

    Once deployed, you will be able to see the deployed OCP app URL.

  10. Open your web browser, navigate to the OCP app URL, and enter prompts.

  11. Clean up all the resources.

    azd down --force --purge