Skip to content

integration-guide.md

Latest commit

 

History

History
452 lines (324 loc) · 11.3 KB

integration-guide.md

File metadata and controls

452 lines (324 loc) · 11.3 KB
Raw

Chat Block — Integration Guide

Add Azure OpenAI chat completions to your existing Python FastAPI app in minutes.

Table of Contents

Prerequisites

Requirement Check
Python 3.10+ python --version
Azure CLI az --version
Azure Developer CLI (azd) azd version
Azure subscription with OpenAI access Apply here if needed
Existing FastAPI project Has main.py + requirements.txt 
# Log in to Azure (needed for keyless auth)
az login
azd auth login

Step 1: Provision Azure Resources

Option A: Using azd add (recommended)

# From your project root
azd add chat
azd provision

This provisions:

  • Azure OpenAI account (Cognitive Services, kind: OpenAI)
  • Chat model deployment (gpt-4o-mini, 10K TPM)
  • RBAC role assignment (Cognitive Services OpenAI User)

Option B: Manual Bicep integration

  1. Copy blocks/chat/infra/chat.bicep into your project's infra/ directory
  2. Add the module to your main.bicep:
module chat 'chat.bicep' = {
  name: 'chat'
  scope: resourceGroup
  params: {
    location: location
    principalId: principalId
    tags: tags
  }
}

// Wire outputs to environment variables
output AZURE_OPENAI_ENDPOINT string = chat.outputs.endpoint
output AZURE_OPENAI_CHAT_DEPLOYMENT string = chat.outputs.chatDeploymentName
output AZURE_OPENAI_API_VERSION string = chat.outputs.apiVersion
  1. Run azd provision

After provisioning

Verify your environment variables are set:

azd env get-values | grep AZURE_OPENAI

You should see:

AZURE_OPENAI_ENDPOINT="https://<name>.openai.azure.com"
AZURE_OPENAI_CHAT_DEPLOYMENT="gpt-4o-mini"
AZURE_OPENAI_API_VERSION="2024-12-01-preview"

Step 2: Install Dependencies

# Add to your existing requirements.txt
echo "openai>=1.50.0" >> requirements.txt
echo "azure-identity>=1.15.0" >> requirements.txt

# Install
pip install -r requirements.txt

Or install directly:

pip install openai>=1.50.0 azure-identity>=1.15.0

Step 3: Add the ChatClient to Your App

3a. Copy the client

# Copy into your source directory (adjust the target to match your project structure)

# If your app is at app/main.py:
cp blocks/chat/code/python/chat_client.py app/

# If your app is at src/main.py:
cp blocks/chat/code/python/chat_client.py src/

# Flat structure (main.py at project root):
cp blocks/chat/code/python/chat_client.py .

Note: The import path depends on your project structure. Use from chat_client import ChatClient for flat structures, or from app.chat_client import ChatClient for package structures.

3b. Create a chat endpoint

Add the following to your FastAPI app:

# src/main.py
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from pydantic import BaseModel

from chat_client import ChatClient

app = FastAPI()

# Initialize the client — reads config from environment variables.
# The system_prompt is optional but recommended.
chat = ChatClient(
    system_prompt="You are a helpful assistant for our product catalog."
)


class ChatRequest(BaseModel):
    message: str


class ChatResponse(BaseModel):
    response: str


@app.post("/chat", response_model=ChatResponse)
async def chat_endpoint(request: ChatRequest):
    """Chat with Azure OpenAI."""
    response = chat.complete([
        {"role": "user", "content": request.message}
    ])
    return ChatResponse(response=response)


@app.post("/chat/stream")
async def chat_stream(request: ChatRequest):
    """Stream a chat response token by token."""
    return StreamingResponse(
        chat.stream([{"role": "user", "content": request.message}]),
        media_type="text/event-stream",
    )


@app.get("/health")
async def health():
    """Health check — verifies Azure OpenAI connectivity."""
    return chat.health_check()

3c. Set environment variables for local dev

Create a .env file (or sync from azd):

# Sync from azd (recommended)
azd env get-values > .env

Or create manually:

AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com
AZURE_OPENAI_CHAT_DEPLOYMENT=gpt-4o-mini
AZURE_OPENAI_API_VERSION=2024-12-01-preview

Load .env in your app (add python-dotenv to requirements.txt):

from dotenv import load_dotenv
load_dotenv()

Step 4: Verify It Works

# Start your app
uvicorn src.main:app --reload

# Test the health endpoint
curl http://localhost:8000/health

# Test the chat endpoint
curl -X POST http://localhost:8000/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "What is Azure OpenAI?"}'

# Test streaming
curl -X POST http://localhost:8000/chat/stream \
  -H "Content-Type: application/json" \
  -d '{"message": "Explain cloud computing in 3 sentences."}'

Expected health check response:

{
  "status": "ok",
  "block": "chat",
  "message": "Azure OpenAI is reachable and the model deployment is responding.",
  "details": {
    "endpoint": "https://your-resource.openai.azure.com",
    "deployment": "gpt-4o-mini",
    "api_version": "2024-12-01-preview",
    "model": "gpt-4o-mini"
  }
}

Before & After

Before: A basic CRUD API

# src/main.py — BEFORE
from fastapi import FastAPI

app = FastAPI()

@app.get("/products")
async def list_products():
    return [{"id": 1, "name": "Widget", "price": 9.99}]

@app.get("/products/{product_id}")
async def get_product(product_id: int):
    return {"id": product_id, "name": "Widget", "price": 9.99}

After: CRUD + AI chat

# src/main.py — AFTER (additions highlighted with comments)
from fastapi import FastAPI
from pydantic import BaseModel                    # ← NEW
from chat_client import ChatClient                # ← NEW

app = FastAPI()

# ── New: AI chat client ──────────────────────────
chat = ChatClient(
    system_prompt="You are a helpful product assistant."
)

class ChatRequest(BaseModel):                     # ← NEW
    message: str

@app.post("/chat")                                # ← NEW
async def chat_endpoint(request: ChatRequest):
    response = chat.complete([
        {"role": "user", "content": request.message}
    ])
    return {"response": response}

@app.get("/health")                               # ← NEW
async def health():
    return chat.health_check()
# ── End new ───────────────────────────────────────

@app.get("/products")
async def list_products():
    return [{"id": 1, "name": "Widget", "price": 9.99}]

@app.get("/products/{product_id}")
async def get_product(product_id: int):
    return {"id": product_id, "name": "Widget", "price": 9.99}

What changed:

  • Added chat_client.py (1 file)
  • Added 2 dependencies to requirements.txt
  • Added ~15 lines of code to main.py
  • The existing endpoints are untouched

API Reference

ChatClient(*, endpoint, deployment, api_version, system_prompt, credential)

Parameter Type Default Description
endpoint str | None $AZURE_OPENAI_ENDPOINT Azure OpenAI URL
deployment str | None $AZURE_OPENAI_CHAT_DEPLOYMENT Model deployment name
api_version str | None $AZURE_OPENAI_API_VERSION API version
system_prompt str | None None Default system message
credential Any | None DefaultAzureCredential() Azure credential

client.complete(messages, *, temperature, max_tokens, **kwargs) → str

Returns the assistant's complete response.

client.stream(messages, *, temperature, max_tokens, **kwargs) → Generator[str]

Yields response tokens as they arrive.

client.health_check() → dict

Returns {"status": "ok", ...} or {"status": "error", "message": "..."}.

Configuration Options

Environment Variables

Variable Required Default Description
AZURE_OPENAI_ENDPOINT Yes — Azure OpenAI resource URL
AZURE_OPENAI_CHAT_DEPLOYMENT Yes — Deployment name
AZURE_OPENAI_API_VERSION Yes — API version

Bicep Parameters

Parameter Default Description
modelName gpt-4o-mini Model to deploy
modelVersion 2024-07-18 Model version
deploymentCapacity 10 TPM capacity (thousands)
location resource group location Azure region

Override via azd:

azd env set AZURE_OPENAI_MODEL gpt-4o
azd env set AZURE_OPENAI_DEPLOYMENT_CAPACITY 30
azd provision

Common Pitfalls

Based on real-world error research. See TROUBLESHOOTING.md for full details.

1. "DefaultAzureCredential failed to retrieve a token"

Cause: Not logged in to Azure CLI.

Fix:

az login
# For multi-tenant: az login --tenant <tenant-id>

2. "403 Forbidden" on API calls

Cause: Missing RBAC role. Having Contributor on the resource group is NOT enough.

Fix:

az role assignment create \
  --assignee $(az ad signed-in-user show --query id -o tsv) \
  --role "Cognitive Services OpenAI User" \
  --scope <azure-openai-resource-id>

The Bicep module creates this role assignment automatically. If you get 403, it may take up to 5 minutes for RBAC to propagate after provisioning.

3. "Model not available in region"

Cause: Not all models are in all regions. gpt-4o-mini is broadly available but not everywhere.

Fix:

azd env set AZURE_LOCATION swedencentral  # or eastus, westus3, etc.
azd provision

4. Quota exceeded

Cause: Subscription TPM limit reached for this model in this region.

Fix: The block defaults to 10K TPM (conservative). If even that fails:

azd env set AZURE_OPENAI_DEPLOYMENT_CAPACITY 5
azd provision

Or request a quota increase in the Azure Portal.

5. SDK version conflicts

Cause: Old openai or azure-identity package installed.

Fix:

pip install --upgrade openai>=1.50.0 azure-identity>=1.15.0

The ChatClient validates SDK versions at import time and will print a clear error if they're too old.

6. "AZURE_OPENAI_ENDPOINT is not set"

Cause: Environment variables not synced after provisioning.

Fix:

azd env get-values > .env
# Then restart your app

Next Steps

  • Add RAG: azd add rag — Add semantic search over your app's data
  • Customize the model: Change AZURE_OPENAI_MODEL to use GPT-4o or other models
  • Add conversation history: Store messages in a list and pass the full history to complete()
  • Add content safety: Layer Azure AI Content Safety on top (coming soon as a block)
  • Deploy: azd deploy pushes your app to Azure with managed identity already configured