[iris] Add tpu_pools config sugar and allocation tier blocking

[rjpower]

Add tpu_pools YAML key that expands into per-(size, zone) scale groups with
topology-derived fields and allocation tier metadata. When the autoscaler sees
quota-exceeded or backoff at tier N in a quota_pool, it blocks tiers >= N in
that pool (per-zone), preventing wasteful crawl-up through larger TPU sizes
that GCP guarantees are also unavailable. Migrates all production configs to
the new format and updates the autoscaler dashboard with collapsible pool
grouping and tier chain visualization.

Design doc: lib/iris/docs/tpu-pool-expansion.md

[rjpower]

🤖 Specification (>500 LOC)

Problem: The autoscaler crawls up the priority waterfall trying larger TPU sizes when smaller ones fail. GCP TPU capacity is monotonic per-zone: if v5p-8 is unavailable, v5p-16/32/etc will also be unavailable. This wastes API rate limit tokens, adds latency, and risks accidental over-allocation. The production config also had ~35 nearly-identical scale group entries differing only in size-derived fields.

Approach:

1. Config expansion (config.py): New _expand_tpu_pools() function runs before _expand_multi_zone_groups(). For each pool x size x zone, emits a fully-specified scale group with topology-derived fields (device_variant, num_vms, device_count from TpuTopologyInfo) and allocation metadata (quota_pool = pool_name/zone, allocation_tier = 1-based index in sorted sizes).

2. Proto (config.proto): Added quota_pool (string) and allocation_tier (int32) fields to ScaleGroupConfig.

3. Types (types.py): Added TPU_FAMILY_VARIANT_PREFIX dict mapping family names to variant prefixes (v5e -> v5litepod, v6e -> v6e, etc) and tpu_variant_name() helper.

4. Autoscaler (autoscaler.py): _pool_blocked_tiers() computes the minimum blocked tier per quota_pool by scanning groups in QUOTA_EXCEEDED or BACKOFF state. _is_tier_blocked() filters matching groups in route_demand() before budget assignment.

5. Dashboard (AutoscalerTab.vue): Groups scale groups by quota_pool with collapsible sections showing tier chain visualization ([T1 ok] -> [T2 blocked] -> [T3 blocked]). Blocked tiers render at reduced opacity.

Key code: The tier blocking filter in route_demand():

pool_blocked = _pool_blocked_tiers(sorted_groups, ts)
# ... inside entry loop, after matching_groups computed:
if pool_blocked:
    matching_groups = [g for g in matching_groups if not _is_tier_blocked(g, pool_blocked)]

Tests:

• TestTpuPoolExpansion (10 tests): expansion correctness, topology derivation, priority override, zone handling, validation errors (unknown family/size, empty sizes, duplicate zones, name collision), coexistence with manual groups, multiple pools same family.
• TestAllocationTierBlocking (5 tests): tier blocked when lower tier quota-exceeded, higher tiers blocked but lower unaffected, independent pools not affected, groups without pool unaffected, same-tier groups independent.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 40a6d07ab2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[yonromai]

The overall direction looks good: the config sugar is much easier to maintain, the targeted config/routing test suites passed locally, and the dashboard changes line up with the new metadata. I left one follow-up about preserving an accurate unmet-demand reason when allocation-tier filtering removes all otherwise-matching groups.

Generated with Codex.

[yonromai]

P2 When allocation-tier filtering removes every hard-matching group, this falls into _diagnose_no_matching_group(...), which reports no_matching_group even though matching groups do exist and were only skipped because the pool is tier-blocked. I reproduced this with a tier-1 quota_exceeded group plus a tier-2 match: routing correctly refuses the tier-2 group, but the unmet reason becomes no groups match device=tpu:v5p-16, which is misleading for operators and the dashboard. Recommended fix: preserve the pre-filter match set and emit a tier-blocked / no-capacity reason when filtering is what emptied the candidate list.

Generated with Codex.