Skip to content

Support guidance for load balancing with ACA #360

Open
@jmatthiesen

Description

@jmatthiesen

This sample should support guidance for load balancing API requests using ACA, as described in the load balancing articles here:

https://learn.microsoft.com/en-us/dotnet/ai/get-started-app-chat-scaling-with-azure-container-apps?toc=%2Fazure%2Fdeveloper%2Fai%2Ftoc.json&bc=%2Fazure%2Fdeveloper%2Fai%2Fbreadcrumb%2Ftoc.json&tabs=github-codespaces%2Cinitial-deployment)

To support this work, we need to support both the OPENAI_HOST and AZURE_OPENAI_CUSTOM_URL environment variables in the application.

Environment Variable Description
OPENAI_HOST Setting to azure_custom tells the Chat app's backend to use a custom URL for the OpenAI requests.
AZURE_OPENAI_CUSTOM_URL Custom URL to use for OpenAI requests (When OPENAI_HOST variable is set to azure_custom)

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [x] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions