feat(basilica): opt-in admin key rotation after bootstrap

[epappas]

Summary

• Adds rotate_admin_key() primitive in deployments/basilica/lifecycle.py and a new TenantSpec.rotate_admin_after_bootstrap: bool = False field (default off). When enabled, provision() runs the rotation as a post-step so the bootstrap admin key returned to the caller is invalidated by the time provisioning returns.
• Wires the rotation through the CLI as python -m deployments.basilica.cli rotate-admin-key --tenant-id ... --config ... --proxy-instance-id ... (with optional --new-key), and through the tenant-lifecycle workflow as a new action: rotate-admin-key choice (workflow_dispatch + repository_dispatch).
• Workflow ::add-mask::s the returned admin_key BEFORE any cat result.json line, same pattern as the existing api_key mask; emits it as a step output for the dispatching app to consume via the Actions API.

Mechanism

The Basilica SDK exposes no env-patch primitive — only create_deployment, delete_deployment, and restart_deployment (which rolls pods without touching env). Rotation therefore deletes the existing proxy UUID and creates a fresh one with LLMTRACE_AUTH_ADMIN_KEY=<new_key> in its env. Consequence: proxy_instance_id and proxy.url change. Callers MUST persist the post-rotation proxy_instance_id / proxy_url over the bootstrap values; both are returned in the result JSON.

This deviates slightly from the literal task description (which assumed an in-place env patch); I chose to be honest about the SDK reality and document it explicitly in code + README rather than fake an env-patch path that doesn't exist on the SDK. The added function arg proxy_spec: ComponentSpec is required for the same reason — Basilica's get_deployment does not echo env back, so we cannot recover the existing env shape and must use the caller's spec.

Trade-off

Opt-in because the rotation adds one proxy re-roll (~30s) to provisioning time. Recommended for production tenants where the bootstrap admin key must be invalidated immediately; leave off for dev / sandbox tenants.

Validation evidence

• python3 -c \"from deployments.basilica import lifecycle, cli; print('ok')\" — pass (with basilica SDK on PYTHONPATH).
• python3 -m unittest deployments.basilica.tests.test_rotate_admin_key — 9 tests pass. Tests mock only the SDK HTTP boundary (BasilicaClient.create_deployment / delete_deployment / get_deployment); the rotation logic itself runs unmodified. Covered: fresh key generated and injected; caller-supplied new_key honoured; old proxy deleted then new created; rejection of empty proxy_instance_id; tenant-id validation; custom proxy_name_template; default rotate_admin_after_bootstrap=False; rotation skipped when enable_proxy_auth=False; full provision→rotation flow returns the new key + new UUID.

What's unvalidated

• Live rotation against Basilica. Not runnable from this environment (no BASILICA_API_TOKEN, no live tenant). Live rotation tested by maintainer after merge.

Files

• deployments/basilica/lifecycle.py — rotate_admin_key(), RotationResult, TenantSpec.rotate_admin_after_bootstrap, post-step in provision().
• deployments/basilica/cli.py — rotate-admin-key subcommand + config wiring for the new field.
• deployments/basilica/tests/test_rotate_admin_key.py — 9 unit tests.
• .github/workflows/tenant-lifecycle.yml — action: rotate-admin-key choice, new_admin_key input, validation + arg wiring, admin_key mask + step output.
• deployments/basilica/README.md — new "Admin key rotation" subsection under auth.
• deployments/basilica/configs/examples/starter.yaml, pro.yaml — opt-in flag comments.

Test plan

• Maintainer: live rotate-admin-key dispatch against an existing tenant; confirm the new proxy reaches phase=ready, the new URL returns 401 for the old key and 200 for the new, and the workflow step output admin_key is masked in run logs but available via the Actions API.
• Maintainer: provision with rotate_admin_after_bootstrap: true in the YAML; confirm the returned api_key is not the bootstrap value (which only lives in the lifecycle library's process memory).