Epic: first-class SNI-rotation + pluggable attested 3p providers (FleetRouter + ProviderRegistry + VerificationEngine)

[Evrard-Nil]

Tracking epic for the cloud-api inference-stack refactor. Goal: (1) make SNI-rotation first-class via stable address-derived backend handles (fixes the GLM-5.1 "does not match any attested fingerprint" signature-fetch bug + the pin-eviction hazard), (2) a pluggable teep-style attested-provider abstraction behind a single registry, (3) a per-model attestation policy (NEAR + attested-3p fallback / attested-3p-only / non-attested) where fallback can never cross an attestation boundary.

Phases (each independently shippable; flag-gated; GLM-5.1 as canary)

• P0 model-proxy: stable-handle SNI -b<hash> + select_backend_by_handle + /backends/list — nearai/model-proxy#38 (additive; deps: none)
• P1 cloud-api: extract FleetRouter from VLlmProvider (still u64, flag-gated, characterization tests first) (deps: none)
• P2 cloud-api: ProviderRegistry + capability handles; move the 5 NEAR-only no-ops off InferenceProvider; delete ExternalProvider attestation stubs (deps: P1)
• P3 cloud-api: FleetRouter u64→BackendHandle; per-handle FingerprintState; handle-based discovery (preserve pubkey harvest). Fixes the bug. (deps: P0 deployed to gpu11+gpu30, P1, P2)
• P4 cloud-api: VerificationEngine factor pipeline + per-step traits + RawAttestation + Blocked(); port NEAR verbatim (signing_algo fix, shared codec, golden vectors) (deps: P2)
• P5 cloud-api: DB migration — drop chk_external_provider_no_attestation; add attestation_policy + providers/model_providers(fallback_order) (deps: none; schema-only)
• P6 cloud-api: loader-merge NEAR+3p under one model_id; ProviderTier vs RequiredPolicy filter in get_providers_with_fallback (fail-closed, within-tier RR) (deps: P2, P5)
• P7 cloud-api: onboard first attested-3p (Venice) — ExternalProvider::new_pinned + report-data verifier + concrete supply-chain verifier (deps: P3, P4, P6)

Design: FleetRouter + ProviderRegistry + VerificationEngine (capability-struct provider, teep-style factor pipeline, content-addressed handles, split live-tier vs required-policy). Reference: github.com/13rac1/teep.

🤖 Generated with Claude Code

[Evrard-Nil]

Progress: P0 → nearai/model-proxy#38 (review changes from @PierreLeGuen addressed: salted handles + auth-gated /backends/list + is_routable(); re-requested). P5 → #737 (open, schema-only). Next: P1 (extract FleetRouter behind a flag, characterization tests first).

[Evrard-Nil]

Tick update:

• P0 (model-proxy#38) MERGED ✅ (Pierre approved after the salt/auth fixes). Build & Deploy succeeded on main (:staging). ⚠️ Before P3 flips, verify the new model-proxy is live on both peers gpu11 + gpu30.
• P5 (feat(db): per-model attestation_policy migration (V0056) #737): Pierre's 2 blockers addressed — kept chk_external_provider_no_attestation (no premature behavior change; CI green) and made attestation_policy nullable/no-default (new rows NULL, not misclassified). The constraint-drop + NOT-NULL + attested-3p enablement move to the routing phase. Re-requested.
• Next: P1 — extract FleetRouter behind a flag (characterization tests first).

[Evrard-Nil]

Tick update:

• P5 (feat(db): per-model attestation_policy migration (V0056) #737): APPROVED + all checks green (Lint/Test Suite/security_audit) — ready to merge (awaiting user).
• P1 scope: user chose full extraction (FleetRouter behind a FLEET_ROUTER flag, provably identical flag-on/off). Started on branch feat/fleet-router-extraction → draft P1: group providers by attestation (attested::nearai / non_attested) + extract FleetRouter #741. Landed the characterization-test foundation (bucket-side pin/unpin + empty-chat_id asymmetry); extraction commits next. Will request review once it's coherent.

[Evrard-Nil]

P1 complete → #741 ready for review (PierreLeGuen + lloydmak99). FleetRouter extraction done: VLlmProvider is now a thin trait wrapper over FleetRouter, which owns all model-proxy routing state + logic — the seam P3 needs to swap the positional u64 rotation index for a stable BackendHandle (the GLM-5.1 signature-fetch fix). 11 commits, every step kept green (unit + fmt + clippy + services build) and validated with a full local e2e_all run (546 passed) between steps. No dual-path flag (single mechanical extraction; the flag lands with P3). Next after merge: P2 (ProviderRegistry + capability handles).