feat(plugins): capacitytiers — ordered capacity-tier gang scheduling

[pfernandes21]

What type of PR is this?

/kind feature

What this PR does / why we need it:

Adds a new capacitytiers scheduler plugin implementing reserved-first spot fallback natively in Volcano (Phase 1 of the gang capacity-preference design). A PodGroup opts in with an ordered tier list; the plugin keeps the whole gang homogeneous on one capacity type at a time and advances tiers on a durable clock:

• Spec annotations (set by the client on the PodGroup):
  • exa.ai/capacity-tiers: "reserved,spot" — ordered node capacity types (values of karpenter.sh/capacity-type)
  • exa.ai/tier-fallback-seconds: "300" — per-tier acquisition budget (default 300s; malformed values fail closed to the default)
• State annotations (managed by the plugin, patched onto the PodGroup so scheduler restarts don't reset the clock):
  • exa.ai/capacity-tier — tier currently in force
  • exa.ai/capacity-tier-since — RFC3339 tier entry time

Per session open, for each opted-in job:

tier = current annotation (or first tier if absent/stale)
if acquiring (phase not Running/Completed) and now - since > budget and a next tier exists:
    tier = next tier; patch PodGroup (merge patch, retried next session on failure)

A hard PredicateFn (UnschedulableAndUnresolvable) then rejects any node whose karpenter.sh/capacity-type label (key overridable via plugin argument capacitytiers.nodeLabelKey) differs from the gang's current tier — so all-or-nothing gang admission via the existing gang plugin + Statement commit/discard can never produce a mixed reserved/spot gang. The clock freezes once the gang is Running; non-opted-in jobs are untouched. Malformed tier lists log and leave the gang unrestricted (predicate not applied), matching pre-plugin behavior.

Uses the same PodGroup patch RBAC already granted for gpuFragmentation stamping (#11).

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Deployment wiring (scheduler config tier, PodGroup annotations from exa_flyte, chart pin) lands separately in the monorepo. Unit tests cover annotation parsing/validation, tier resolution (fresh entry, budget expiry, last-tier hold, frozen clock when running), and session-level predicate behavior via uthelper fake sessions.

Does this PR introduce a user-facing change?

Added the capacitytiers plugin: PodGroups annotated with exa.ai/capacity-tiers are scheduled homogeneously on one capacity tier at a time, falling back to the next tier after exa.ai/tier-fallback-seconds of failed acquisition, with tier state persisted on the PodGroup.

Link to Devin session: https://app.devin.ai/sessions/100be51de8b84c958695756c345708b9
Requested by: @pfernandes21

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

Superseded: gang capacity selection is moving to static pod specs in exa_flyte (mixed reserved+spot gangs, same-AZ required, reserved preferred) with the exa-scale first-workload latch covering assembly time — no Volcano plugin needed.