Skip to content

Latest commit

 

History

History
1775 lines (655 loc) · 31.9 KB

File metadata and controls

1775 lines (655 loc) · 31.9 KB

Deploy ClassyMail from scratch (new tenant)

Goal: A person clones this repo and deploys the full stack in a

fresh Azure tenant they own — no prior infrastructure, no CI/CD.

Estimated time: 45–60 minutes (including model provisioning wait times).


Table of Contents

  1. Prerequisites

  2. Clone and Configure

  3. Provision Infrastructure

  4. Build and Push Container Image

  5. Deploy AI Models

  6. Local Development Setup

  7. Verify and Smoke Test

  8. Troubleshooting

  9. Cleanup / Teardown


1. Prerequisites

1.1 Azure Account

| Requirement | Details |

|-------------|---------|

| Azure Subscription | Active subscription with billing enabled |

| Permissions | Owner or Contributor + User Access Administrator at subscription level |

| Quota | Container Apps (2 apps, 0.5 CPU/1Gi each), Cosmos DB Serverless, Service Bus Standard |

| Region | swedencentral recommended (best EU model availability). Alternatives: westeurope, eastus, eastus2 |

1.2 Tools (install on your machine)

# Verify all are installed

az --version          # Azure CLI >= 2.60

terraform --version   # Terraform >= 1.5

docker --version      # Docker Desktop or Podman

node --version        # Node.js >= 18

python --version      # Python 3.12

uv --version          # uv (Python package manager)

Install links:

1.3 Register Azure Resource Providers

Resource providers must be registered once per subscription (idempotent):

az login --tenant <YOUR_TENANT_ID>

az account set --subscription <YOUR_SUBSCRIPTION_ID>



# Register all required providers (takes 1-5 min each; runs in background)

$providers = @(

  "Microsoft.Storage",

  "Microsoft.ServiceBus",

  "Microsoft.DocumentDB",

  "Microsoft.CognitiveServices",

  "Microsoft.App",

  "Microsoft.EventGrid",

  "Microsoft.Insights",

  "Microsoft.OperationalInsights",

  "Microsoft.ManagedIdentity",

  "Microsoft.ContainerRegistry"

)

foreach ($p in $providers) {

  az provider register --namespace $p --wait

  Write-Host "Registered: $p"

}

1.4 Verify Mistral Document AI Availability

Mistral Document AI is deployed directly through Microsoft AI Foundry as a

Serverless API — no Azure Marketplace subscription is required.

  1. Go to Microsoft AI Foundry

  2. Select your project (created by Terraform in Step 3)

  3. Navigate to Model catalog and search for "Mistral Document AI"

  4. Verify the model is available in your region (recommended: swedencentral)

If the model is not available in your region, check

Azure AI model availability

and consider switching your location in terraform.tfvars.


2. Clone and Configure

2.1 Clone the Repository

git clone https://github.com/<owner>/ClassyMail.git

cd ClassyMail

2.2 Create a Container Registry (if you don't have one)

# Choose a globally unique name (letters+numbers only, 5-50 chars)

$ACR_NAME = "emailpoctestacr"

$ACR_RG   = "rg-acr-shared"

$LOCATION = "swedencentral"



az group create --name $ACR_RG --location $LOCATION

az acr create --name $ACR_NAME --resource-group $ACR_RG --sku Basic --admin-enabled false

2.3 Create terraform.tfvars

Copy-Item infra/terraform.tfvars.example infra/terraform.tfvars

Edit infra/terraform.tfvars with your values:

# REQUIRED

subscription_id = "<YOUR_SUBSCRIPTION_ID>"



# First deploy: use a placeholder image (real image built in Step 4)

container_image = "mcr.microsoft.com/azuredocs/containerapps-helloworld:latest"



# ACR for the managed identity to pull images

acr_name           = "emailpoctestacr"

acr_resource_group = "rg-acr-shared"



# Customize prefix to avoid naming collisions

prefix   = "classymail-dev"

location = "swedencentral"



# Keep defaults for a clean new-tenant deployment

cosmos_use_rbac                = true

enable_model_deployments       = false    # Deploy models manually (Step 5)

deploy_language_service        = false    # Optional, enable later

deploy_document_intelligence   = true     # Recommended: standalone DI for OCR fallback

tag_policy_enabled             = true

security_cost_policy_enabled   = true



# Your public IP for Cosmos DB data-plane access during local dev

# Find it: (Invoke-WebRequest ifconfig.me).Content.Trim()

allowed_ip_ranges = ["<YOUR_PUBLIC_IP>"]

Tip: Find your public IP with:

(Invoke-WebRequest -Uri "https://ifconfig.me" -UseBasicParsing).Content.Trim()


3. Provision Infrastructure

3.1 Automated (recommended)

.\infra\deploy.ps1 -TenantId "<TENANT_ID>" -SubscriptionId "<SUBSCRIPTION_ID>"

The script will:

  1. Verify Azure CLI auth

  2. Set ARM_USE_MSI=false (Fortinet/corporate firewall workaround — prevents IMDS calls)

  3. Register the required Azure resource providers (idempotent)

  4. Run terraform init -upgrade

  5. Run terraform plan and show the plan

  6. Ask for confirmation before terraform apply

  7. After apply, discover the app managed identity and add only the missing RBAC role assignments (idempotent verify step)

Linux/macOS: bash infra/deploy.sh --tenant-id <TENANT_ID> --subscription-id <SUBSCRIPTION_ID>

Verify only (double-check / repair roles on an existing RG, no Terraform): .\infra\deploy.ps1 -VerifyOnly -ResourceGroup <prefix>-rg (Linux/macOS: bash infra/deploy.sh --verify-only --resource-group <prefix>-rg)

3.2 Manual (for transparency)

az login --tenant <TENANT_ID>

az account set --subscription <SUBSCRIPTION_ID>



# Fortinet / corporate firewall workaround (mandatory on corp networks)

$env:ARM_USE_MSI  = "false"

$env:ARM_USE_OIDC = "false"



terraform -chdir=infra init -upgrade

terraform -chdir=infra plan -var "subscription_id=<SUBSCRIPTION_ID>" -out tfplan

terraform -chdir=infra apply tfplan

Linux/macOS: use export ARM_USE_MSI="false" and export ARM_USE_OIDC="false" instead.

3.3 Expected Resources Created

After terraform apply, you should see ~25 resources:

| Resource | Name |

|----------|------|

| Resource Group | classymail-dev-rg |

| Storage Account | emailpoctestst |

| Blob Container | pdf-inputs |

| Service Bus Namespace | classymail-dev-sbus |

| Service Bus Queue | pdf-processing-queue |

| Event Grid Topic | classymail-dev-blob-events |

| Microsoft AI Foundry | classymail-dev-aifoundry |

| Cosmos DB Account | classymail-dev-cosmos |

| Cosmos DB Database | emailsdb |

| Cosmos Containers | emails, chat_history, vector_cache |

| Managed Identity | classymail-dev-id |

| Log Analytics | classymail-dev-logs |

| App Insights | classymail-dev-appi |

| Container App Env | classymail-dev-env |

| Container App: API | classymail-dev-api |

| Container App: Worker | classymail-dev-worker |

Rating: The Container Apps will be running the placeholder image at this point.

That's expected — we'll update them in Step 4.

RBAC: Terraform assigns exactly 6 roles to the managed identity (Storage Blob Data Contributor,

Service Bus Data Owner, Custom Cosmos App Role, Cognitive Services User, AcrPull,

and optionally Cognitive Services Language Reader). No extra manual roles are needed.

The verification scripts will warn if any unexpected roles are found.

See RBAC_AUDIT.md §8 for historical context.

3.4 Rating Terraform Outputs

terraform -chdir=infra output

Save these values — you'll need AI_ENDPOINT for configuring model deployments.


4. Build and Push Container Image

4.1 Install Frontend Dependencies and Build

cd frontend

npm install

npm run build

cd ..

4.2 Fetch Vue Runtime

.\scripts\fetch_vue_runtime.ps1

4.3 Build and Push to ACR

Option A — Remote build (recommended, no Docker needed locally):

.\scripts\build_acr.ps1 -AcrName "emailpoctestacr" -Tag "v1"

Option B — Local Docker build and push:

$IMAGE = "emailpoctestacr.azurecr.io/classymail-agent:v1"

az acr login --name emailpoctestacr

docker build -t $IMAGE .

docker push $IMAGE

4.4 Update Terraform with Real Image

Edit infra/terraform.tfvars:

container_image = "emailpoctestacr.azurecr.io/classymail-agent:v1"

Re-apply:

terraform -chdir=infra plan -var "subscription_id=<SUBSCRIPTION_ID>" -out tfplan

terraform -chdir=infra apply tfplan

This updates both Container Apps (API + Worker) to use your real image.


5. Deploy AI Models

Models must be deployed manually in Microsoft AI Foundry.

Availability varies by region and tenant.

5.1 Open Microsoft AI Foundry

  1. Go to Microsoft AI Foundry

  2. Select your project: classymail-dev-project

  3. Navigate to Deployments > + Deploy model

5.2 Minimum Viable Models

Deploy these three models (strictly required):

| Model | Deployment Name | Type | Data Zone | Regions (Hub/Project) | Purpose |

|-------|----------------|------|:---------:|----------------------|---------|

| Phi-4 | phi-4 | Serverless API | ✅ | eastus, eastus2, northcentralus, southcentralus, swedencentral, westus, westus3 | Email classification |

| Mistral Document AI 2512 | mistral-document-ai-2512 | Global Standard | ✅ (US + EU) | All regions (Global Standard) ¹ | OCR / PDF extraction |

| text-embedding-3-small | text-embedding-3-small | Standard (Global) | ✅ | All Global Standard regions ¹ | RAG embeddings |

¹ Full Global Standard region table

5.3 Recommended Optional Models

These models appear in the UI model selector and are used by various features.

Deploy them as Global Standard (Azure OpenAI) or Serverless API (partner) deployments in Microsoft AI Foundry with the exact deployment names below.

| Model | Deployment Name | Type | Data Zone | Regions (Hub/Project) | Purpose |

|-------|----------------|------|:---------:|----------------------|---------|

| GPT-4.1-mini | gpt-4.1-mini | Global Standard | ✅ | 20+ regions (all major) ¹ | Fallback classifier, vision, anonymization (GA, retires 2027-10-14) |

| GPT-4.1-nano | gpt-4.1-nano | Global Standard | ✅ | 20+ regions (all major) ¹ | Category assessment (fast, default) |

| GPT-5-nano | gpt-5-nano | Global Standard | ✅ | 20+ regions (all major) ¹ | Category assessment (alternative) |

| GPT-5-mini | gpt-5-mini | Global Standard | ✅ | 20+ regions (all major) ¹ | Higher-quality classification |

| GPT-5.1 | gpt-5.1 | Global Standard | ✅ | eastus2, swedencentral + more ¹ | RAG chat reasoning model (GA, retires 2027-05-15; use CHAT_API_VERSION=preview) |

| Kimi-K2.5 | Kimi-K2.5 | Serverless (Moonshot AI) | ❌ | See Foundry model catalog ² | Multilingual classification |

| GPT-4.1 | gpt-4.1 | Global Standard | ✅ | 20+ regions (all major) ¹ | Agentic tier3 + red-team (GA, retires 2027-10-14) |

¹ Full Global Standard region table

² Serverless API region availability

Rating: The UI dynamically fetches available deployments from /api/admin/deployments.

Models not deployed will still appear as selectable options (from a hardcoded fallback list)

but will fail at inference time. Deploy at minimum GPT-4.1-mini and GPT-4.1-nano.

5.4 Verify Deployments

# After generating secrets.env (Step 6), verify:

uv run python scripts/list_deployments.py

6. Local Development Setup

6.1 Generate secrets.env

This script discovers all Azure resources and writes a local config file:

.\scripts\write_secrets_env.ps1 `

  -ResourceGroup "classymail-dev-rg" `

  -Prefix "classymail-dev" `

  -Force

6.2 Assign RBAC Roles for Local Development

Your Azure CLI user needs data-plane access to Storage, Service Bus, and Cosmos DB:

.\scripts\assign_local_dev_roles.ps1 `

  -StorageAccountName "emailpoctestst" `

  -ServiceBusNamespace "classymail-dev-sbus" `

  -CosmosAccountName "classymail-dev-cosmos" `

  -ResourceGroup "classymail-dev-rg"

Wait 2-5 minutes for RBAC propagation before testing.

6.3 Install Python Dependencies

uv sync --dev

6.4 Run Locally

uv run uvicorn classymail.app:app --reload --port 8000

Open http://localhost:8000 in your browser.


6.5 Upload PDFs to Blob Storage

After infrastructure is deployed and RBAC is assigned, you can populate the

pdf-inputs container with PDF files. **Every .pdf uploaded triggers the

full classification pipeline automatically** (Event Grid → Service Bus →

Worker → OCR → Classification → Cosmos DB).

Important constraints:

  • Only .pdf files are processed. ZIP, DOCX, images, etc. are ignored by Event Grid.
  • Shared-key / SAS token access is disabled on the storage account. All methods must use Entra ID (Azure AD) authentication.
  • You need the Storage Blob Data Contributor role on the storage account (assigned in Step 6.2).
  • Each PDF triggers one pipeline run. Uploading 10 000 files = 10 000 pipeline runs. The Service Bus queue buffers messages and the Worker auto-scales via KEDA.

Path structure

The API stores uploads under a dated prefix:


pdf-inputs/

  uploads/

    2026/

      02/

        17/

          a1b2c3d4-my-document.pdf

          e5f6a7b8-another-file.pdf

When uploading directly (bypassing the API), you can use any structure — the

only requirement is that each blob path ends with .pdf and is unique

(uploading to the same path overwrites the file and re-triggers the pipeline).

Recommended convention for bulk uploads:


pdf-inputs/uploads/{filename}.pdf

Or mirror the API date-based structure:


pdf-inputs/uploads/2026/02/17/{filename}.pdf


Method 1: Web UI (small batches)

Best for: Quick tests with a few files.

  1. Open the application URL (local: http://localhost:8000, or the Container App URL)

  2. Navigate to Upload

  3. Drag-and-drop or select PDF files

  4. Click Upload

Limits: 10 files per upload, 10 MB per file.


Method 2: Azure Portal — Storage Browser (moderate batches)

Best for: GUI upload of tens to hundreds of files without installing anything.

  1. Go to Azure Portal

  2. Navigate to your Storage Account (e.g. emailpoctestst)

  3. In the left menu, click Storage browserBlob containerspdf-inputs

  4. Click Upload (top toolbar)

  5. Click Browse for files and select your PDFs (multi-select supported)

  6. Set Upload to folder to uploads (or uploads/2026/02/17)

  7. Click Upload

Rating: The portal supports selecting hundreds of files at once. For 10K+

files, use AzCopy (Method 4) instead — the portal may time out.


Method 3: Azure CLI — az storage blob upload-batch (scripted bulk)

Best for: Scripted, repeatable uploads of thousands of files from a local folder.

PowerShell:

# Upload all PDFs from a local folder into the "uploads" path

az storage blob upload-batch `

  --account-name emailpoctestst `

  --destination pdf-inputs `

  --destination-path uploads `

  --source "C:\MyPDFs" `

  --pattern "*.pdf" `

  --auth-mode login `

  --overwrite false



# Upload from a subdirectory tree (preserves folder hierarchy)

az storage blob upload-batch `

  --account-name emailpoctestst `

  --destination pdf-inputs `

  --destination-path uploads `

  --source "C:\MyPDFs\batch-2026-02" `

  --pattern "**/*.pdf" `

  --auth-mode login `

  --overwrite false

Bash (Linux / macOS):

az storage blob upload-batch \

  --account-name emailpoctestst \

  --destination pdf-inputs \

  --destination-path uploads \

  --source ./my-pdfs \

  --pattern "*.pdf" \

  --auth-mode login \

  --overwrite false

--auth-mode login is required (shared keys are disabled).

--overwrite false prevents accidental re-processing of already-uploaded files.


Method 4: AzCopy (10K+ files — fastest)

Best for: Very large datasets. AzCopy uses parallel transfers and is

significantly faster than the Azure CLI for thousands of files.

Install: AzCopy download

PowerShell:

# 1. Authenticate with Entra ID (required — SAS/shared keys are disabled)

azcopy login --tenant-id <YOUR_TENANT_ID>



# 2a. Copy all PDFs from a local folder

azcopy copy "C:\MyPDFs\*.pdf" `

  "https://emailpoctestst.blob.core.windows.net/pdf-inputs/uploads/" `

  --recursive



# 2b. Copy a directory tree (only PDFs, preserves subfolders)

azcopy copy "C:\MyPDFs\*" `

  "https://emailpoctestst.blob.core.windows.net/pdf-inputs/uploads/" `

  --recursive `

  --include-pattern "*.pdf"



# 2c. Server-side copy from another Azure storage account (no local download)

azcopy copy `

  "https://sourceaccount.blob.core.windows.net/source-container/*" `

  "https://emailpoctestst.blob.core.windows.net/pdf-inputs/uploads/" `

  --recursive `

  --include-pattern "*.pdf"

Bash (Linux / macOS):

azcopy login --tenant-id <YOUR_TENANT_ID>



azcopy copy "./my-pdfs/*.pdf" \

  "https://emailpoctestst.blob.core.windows.net/pdf-inputs/uploads/" \

  --recursive

Server-side copy: If your PDFs are already in another Azure storage

account, use AzCopy account-to-account copy — data moves within Azure

without downloading locally. You need Storage Blob Data Reader on the

source and Storage Blob Data Contributor on the destination.


Method 5: VS Code — Azure Storage Extension (IDE)

Best for: Developers who prefer working inside their editor.

  1. Install the Azure Storage extension for VS Code

  2. Sign in to Azure in the sidebar (Azure icon → Sign in)

  3. Expand Storage Accounts → your account (e.g. emailpoctestst)

  4. Expand Blob Containerspdf-inputs

  5. Right-click pdf-inputsCreate Virtual Directory → name it uploads

  6. Right-click uploadsUpload Files...

  7. Select your PDF files or an entire folder

  8. Files are uploaded with your Entra ID credentials automatically


Method 6: Azure Storage Explorer (desktop GUI)

Best for: Moderate batches with drag-and-drop from a desktop application.

Install: Azure Storage Explorer

  1. Open Storage Explorer and sign in with your Azure account

  2. Navigate to Storage Accounts → your account → Blob Containerspdf-inputs

  3. Click New Folder to create uploads (or the date-based path)

  4. Navigate into the folder

  5. Click UploadUpload Files or Upload Folder

  6. Select your PDFs and confirm

Storage Explorer uses your Entra ID session automatically — no SAS tokens needed.


Upload methods comparison

| Method | Best for | Volume | Auth | Requires install? |

|--------|----------|--------|------|--------------------|

| Web UI | Quick test | ≤ 10 files, 10 MB each | App session | No |

| Azure Portal | Moderate GUI upload | Hundreds | Entra ID | No |

| az storage blob upload-batch | Scripted bulk | Thousands | --auth-mode login | Azure CLI |

| AzCopy | Large datasets | 10K – millions | azcopy login | AzCopy binary |

| VS Code extension | IDE workflow | Hundreds | Entra ID | VS Code extension |

| Storage Explorer | Desktop drag-and-drop | Hundreds | Entra ID | Desktop app |


What happens after upload

Every .pdf uploaded to pdf-inputs triggers this automated flow:

  1. Event Grid fires a BlobCreated event (filtered to .pdf / .PDF)

  2. Service Bus receives the event in the pdf-processing-queue

  3. Worker picks up the message and creates a PROCESSING record in Cosmos DB

  4. Pipeline runs: PDF download → Mistral OCR → Phi-4 classification → embedding

  5. Cosmos DB receives the final PROCESSED (or REVIEW_REQUIRED) record

  6. Result appears in the UI

Monitor progress

# Check Service Bus queue depth (messages waiting to be processed)

az servicebus queue show `

  --namespace-name classymail-dev-sbus `

  --resource-group classymail-dev-rg `

  --name pdf-processing-queue `

  --query "countDetails.activeMessageCount"



# Count blobs in the container

az storage blob list `

  --account-name emailpoctestst `

  --container-name pdf-inputs `

  --auth-mode login `

  --query "length(@)"



# Check processed records via the API

curl https://<your-api-url>/api/emails?limit=1 | jq '.total'

Tip: After a large bulk upload, monitor the queue depth. When it reaches

0 and all Cosmos records show PROCESSED or REVIEW_REQUIRED, the

batch is complete.


7. Verify and Smoke Test

7.1 Run Verification Script

.\scripts\verify-mvp-setup.ps1 -ResourceGroup "classymail-dev-rg"

This checks all 12 components: auth, resource group, managed identity, storage,

Cosmos DB, Service Bus, AI Foundry, Language service, Container Apps, RBAC,

API endpoints, and provides a summary.

7.2 End-to-End Test

  1. Open the API Container App URL (from terraform output or the verify script)

  2. Upload a PDF via the UI

  3. Verify:

    • PDF appears in Blob Storage (pdf-inputs container)

    • Service Bus queue receives a message

    • Worker processes the PDF (check Container App logs)

    • Classification result appears in Cosmos DB (emails container)

    • Result is visible in the UI

7.3 Run Unit Tests

uv run pytest

8. Troubleshooting

8.1 Mistral OCR Failures and DLQ Recovery

Mistral Document AI can be temporarily unavailable (outage, throttling, regional issues).

The system has 3 layers of automatic retry before a message lands in the Dead Letter Queue (DLQ):

| Layer | Mechanism | Config | Default |

|-------|-----------|--------|---------|

| 1. Apply retry | Tenacity exponential backoff | MISTRAL_OCR_MAX_ATTEMPTS | 2 attempts (1-10s waits) |

| 2. Circuit breaker | pybreaker | fail_max=5, reset_timeout=60s | Opens after 5 consecutive failures |

| 3. Service Bus delivery | Auto-retry by Azure | max_delivery_count | 5 deliveries before DLQ |

Total: up to 2 x 5 = 10 OCR attempts per message before DLQ.

Common Mistral OCR Errors

| HTTP Code | Meaning | Retried? | Action |

|-----------|---------|----------|--------|

| 429 | Rate limited (throttled) | Yes | Wait - auto-recovers. Increase MISTRAL_OCR_MAX_ATTEMPTS if needed |

| 500, 502, 503, 504 | Mistral backend issue | Yes | Transient - replay DLQ after service recovers |

| 422 | Unprocessable PDF (corrupt) | No | Check PDF file; re-upload a valid version |

| 401, 403 | Auth / RBAC error | No | Verify Cognitive Services User role on the managed identity |

| Timeout | Network or server slow | Yes | May need httpx timeout increase in env |

Recovery: Replay DLQ Messages

After the root cause is resolved (e.g. Mistral OCR is back online), replay the failed

messages back into the active queue:

Option A: Via the UI (recommended)

  1. Open the dashboard → Failures tab

  2. Review the DLQ messages (click "View" for details)

  3. Click "Replay All" to re-enqueue all messages for reprocessing

  4. The worker will pick them up and retry OCR automatically

Option B: Via the API

# Replay all DLQ messages back to the active queue

curl -X POST https://<your-api-url>/api/admin/replay-dlq



# Response:

# {"status": "success", "replayed": 42, "errors": []}

Option C: Via Azure CLI

# List DLQ message count

az servicebus queue show --name pdf-processing-queue \

  --namespace-name <sb-namespace> -g <resource-group> \

  --query "countDetails.deadLetterMessageCount"



# For manual replay of individual messages, use Service Bus Explorer

# in the Azure Portal: Service Bus > Queues > pdf-processing-queue > Dead-letter

Tip: If you only want to delete failed messages without retrying, use the

"Purge All" button instead. This permanently removes them from the DLQ.

Tuning Retry Behavior

Set these environment variables on the Worker Container App:

# Increase OCR retry attempts (default: 3)

MISTRAL_OCR_MAX_ATTEMPTS=5



# Increase Service Bus delivery attempts (requires Terraform change)

# In infra/main.tf: max_delivery_count = 10

Common Issues

| Problem | Cause | Fix |

|---------|-------|-----|

| terraform apply fails on policies | Tenant restricts custom policy creation | Set tag_policy_enabled = false and security_cost_policy_enabled = false in terraform.tfvars |

| Cosmos DB 403 Forbidden | RBAC not propagated yet | Wait 5-10 min, then restart Container Apps: az containerapp restart --name classymail-dev-api -g classymail-dev-rg |

| Mistral deployment fails | Model not available in region | Check model catalog in Microsoft AI Foundry; try swedencentral or eastus. See Section 1.4 |

| Container App stuck "Provisioning" | Placeholder image or ACR pull failure | Verify acr_name and acr_resource_group in tfvars, ensure AcrPull role is assigned |

| Event Grid messages not arriving | Service Bus local auth disabled by tenant policy | Check az servicebus namespace show --name <ns> -g <rg> --query disableLocalAuth — must be false for Event Grid |

| Model not available in region | Regional model availability | Try swedencentral, eastus, or check Azure AI model availability |

| write_secrets_env.ps1 fails | Resources not found with prefix | Pass -ResourceGroup and -Prefix explicitly |

| Docker build fails with python:3.12-slim | Docker Hub blocked in corporate network | Use an internal mirror or build via ACR (.\scripts\build_acr.ps1) |

| azapi_resource 403 / IMDS error on vector_cache | See detailed fix below | Pull latest code then follow checklist below |

Fix: ChainedTokenCredential / IMDS 403 on azapi resources

The azapi provider v1.13 defaults use_msi = true (unlike azurerm which defaults to false).

On corporate networks where firewalls (FortiGuard, Zscaler) block the IMDS endpoint (169.254.169.254), this causes a 403 HTML response that crashes the credential chain.

Checklist for the deployer:

# 1. Pull the latest code (the fix is already committed)

git pull origin main



# 2. Use the deploy script (sets ARM_USE_MSI=false automatically)

.\infra\deploy.ps1 -SubscriptionId "<SUBSCRIPTION_ID>"

# Linux/macOS: bash infra/deploy.sh --subscription-id <SUBSCRIPTION_ID>



# -- OR if deploying manually --



# 2b. Set environment variables BEFORE terraform commands:

$env:ARM_USE_MSI  = "false"   # PowerShell

$env:ARM_USE_OIDC = "false"

# export ARM_USE_MSI="false"  # bash

# export ARM_USE_OIDC="false"



# 3. Delete cached providers and re-initialize

Remove-Item -Recurse -Force infra\.terraform -ErrorAction SilentlyContinue

Remove-Item -Force infra\.terraform.lock.hcl -ErrorAction SilentlyContinue

terraform -chdir=infra init -upgrade



# 4. Verify Azure CLI login is active:

az account show



# 5. Re-run plan/apply

terraform -chdir=infra plan -out=tfplan

terraform -chdir=infra apply tfplan

Required RBAC: Owner (or Contributor) on the resource group is sufficient.

Subscription-level Owner is NOT required.

skip_provider_registration = true is set in both providers so no subscription-level

resource-provider registration is attempted.

Useful Commands

# Check Container App logs

az containerapp logs show --name classymail-dev-api -g classymail-dev-rg --follow



# Check Worker logs

az containerapp logs show --name classymail-dev-worker -g classymail-dev-rg --follow



# Restart Container Apps after config changes

az containerapp restart --name classymail-dev-api -g classymail-dev-rg

az containerapp restart --name classymail-dev-worker -g classymail-dev-rg



# Update Cosmos DB firewall with your IP + Container App IPs

.\scripts\update_cosmos_firewall.ps1 -ResourceGroup "classymail-dev-rg" -IncludeLocalIP



# Check all Terraform state

terraform -chdir=infra show

9. Cleanup / Teardown

Destroy All Infrastructure

terraform -chdir=infra destroy -var "subscription_id=<SUBSCRIPTION_ID>"

Delete ACR (if created in Step 2.2)

az group delete --name rg-acr-shared --yes --no-wait

Remove Local Files

Remove-Item -Path secrets.env -ErrorAction SilentlyContinue

Remove-Item -Path infra/terraform.tfvars -ErrorAction SilentlyContinue

Remove-Item -Path infra/tfplan -ErrorAction SilentlyContinue

Remove-Item -Path infra/terraform.tfstate -ErrorAction SilentlyContinue

Remove-Item -Path infra/terraform.tfstate.backup -ErrorAction SilentlyContinue

Remove-Item -Recurse -Path infra/.terraform -ErrorAction SilentlyContinue

Architecture Reference

flowchart TD

    A["API + UI - Container App"] --> B["Service Bus Queue"]

    A --> C["Blob Storage: pdf-inputs"]

    C --> D["Event Grid"]

    D --> B

    B --> E["Worker - Container App"]

    E --> F["Mistral OCR"]

    E --> G["Phi-4 Classification"]

    E --> H["Cosmos DB: emails"]

    A --> H

    A --> I["text-embedding-3-small"]

Loading

For the full architecture, see #docs/ARCHITECTURE.md.


Quick Reference Card

| What | Command |

|------|---------|

| Deploy infra | .\infra\deploy.ps1 -TenantId <T> -SubscriptionId <S> |

| Build + push image | .\scripts\build_acr.ps1 -AcrName <acr> -Tag v1 |

| Generate secrets.env | .\scripts\write_secrets_env.ps1 -ResourceGroup <rg> -Prefix <pfx> -Force |

| Assign local RBAC | .\scripts\assign_local_dev_roles.ps1 -ResourceGroup <rg> ... |

| Verify setup | .\scripts\verify-mvp-setup.ps1 -ResourceGroup <rg> |

| Run locally | uv run uvicorn classymail.app:app --reload --port 8000 |

| Run tests | uv run pytest |

| Destroy all | terraform -chdir=infra destroy -var "subscription_id=<S>" |