feat(deploy,infra,security): deployment metadata persistence (customer-pinnable region) + log scrubbing

[Cataldir]

Problem statement

Capability 43's locked data-residency + log-scrubbing contract:

• Deployment metadata stored in Cosmos containers per region. The user picks a metadata region from the same dropdown they pick the deployment region from. Default: same. v1 supports West Europe, East US 2, Brazil South.
• Logs scrubbed. Subscription IDs replaced with sub_<sha256[0:12]>. Email addresses replaced with user_<oid>. Full mappings stored only in the encrypted Cosmos record, accessible only by the deployment-service managed identity.
• Audit trail. Every deploy / pre-flight / delete writes an immutable audit record to App Insights + an append-only blob.

This issue ships the Cosmos region selection, the structured-logging filter that scrubs sub IDs and emails, and the append-only audit blob writer.

Acceptance criteria

• Cosmos container (per issue C1 feat(infra): deploy-portal Bicep module (Container Apps + APIM + Key Vault + Cosmos) #1027) deployed in three regions (West Europe, East US 2, Brazil South) with the user's metadata region pinned at deployment-record creation.
• User metadata-region selection on /deploy/configure (dropdown defaults to the deployment region).
• Structured logging filter applied to the FastAPI app:
  • Subscription IDs in any log/trace replaced with sub_<sha256[0:12]> (deterministic per-sub hash)
  • Email addresses replaced with user_<oid> where the OID is known
  • PR-level test asserts no plaintext sub ID survives in App Insights queries
• Mapping table (sub_hash → sub_id, user_<oid> → email) stored only in the encrypted Cosmos record; access scoped to the deployment-service managed identity.
• Audit writer:
  • App Insights custom event per deploy / pre-flight / cleanup / delete
  • Append-only blob (immutable storage) writes the audit record per event with WORM lock
  • Audit record contains: timestamp, action, deployment ID, hashed sub ID, hashed user OID, outcome
• Audit retention: 30 days for deployment metadata after delete (per issue C10 feat(deploy,ui,infra): exit/portability — "Delete this deployment" action with type-the-RG confirmation #1036), then purged. Audit blob retains per the immutable storage policy (longer).
• PR includes a synthetic-trace test that runs a mock deploy and confirms zero plaintext sub IDs / emails in any sink.

Risks and dependencies

Risk	Mitigation
PII leakage in logs.	Scrubbing filter; sampling validation; App Insights ingestion rules enforce.
Cosmos region availability mismatch (e.g., user picks Brazil South but deployment goes to a region without Foundry capacity).	Cosmos region is a metadata pin only; deployment region is a separate selection; UI clarifies the difference.
Audit blob WORM policy prevents legitimate corrections.	Corrections written as new audit records ("supersedes #N"); originals preserved.
Hash collision exposes one user's logs to another.	sha256 truncated to 12 chars produces an effectively unique mapping; full hash stored in Cosmos for verification.

Blocked by: epic 41 #1020; C1 #1027 (multi-region Cosmos); C5 #1031 (OBO/audit hooks); R2 epic #1008; R1 epic #990.

Evidence links

• infra/
• apps/
• docs/architecture/ADRs.md

ADR impact

• ADR-034 (audience-segmented IA) — implementation step.

Branch

feature/<this-issue-id>-deploy-metadata-residency-and-log-scrubbing per ADR-018.

BPMN process

%%{init: {'theme':'base', 'themeVariables': {
  'primaryColor':'#FFB3BA',
  'primaryTextColor':'#000',
  'primaryBorderColor':'#FF8B94',
  'lineColor':'#BAE1FF',
  'secondaryColor':'#BAE1FF',
  'tertiaryColor':'#FFFFFF'
}}}%%
flowchart LR
  A[Analyze Current Code] --> B[Design Change]
  B --> C[Implement on Issue Branch]
  C --> D[Open PR]
  D --> E[Validation and Fixes]
  E --> F[Merge to Main]
  F --> G[Monitor Workflows]
  G --> H[Close Issue and Cleanup]

Loading