Skip to content

GLM-5.3 Flash: pooled HiCache on one TP4 replica #251

Description

@PierreLeGuen

Prepare one GLM-5.3 Flash TP4 replica to reuse its in-CVM CPU cache through pooled direct transfers, retaining the other replica as the HiCache-disabled control.

The runtime must include the complete hybrid restore correction from sgl-project/sglang#38212, preserve target/MTP/index/recurrent state, and carry no lab scheduler-control hooks. The candidate uses a deployment-configurable total RAM budget (GLM53_HICACHE_RAM_BUDGET, default 80% of available CVM/container RAM at startup), write-through policy and page-first direct I/O; model, TP4/EP4, adaptive EAGLE, serving limits and conversation affinity retain their current settings.

Acceptance:

  • Include startup RAM sizing with explicit GB/GiB overrides, shared TP/component accounting and cgroup-aware safety checks.
  • Merge the reviewed, reproducible runtime recipe and signed publication workflow.
  • Publish and verify the immutable candidate digest, provenance, SBOM and signature.
  • Open the activation PR changing r2 only, with distinct telemetry and a tested rollback path.
  • Qualify the exact TEE/PPCIe topology in staging, including the required 30-minute soak and confirmed CPU-cache restorations.
  • Measure the authorized canary's TTFT, decode latency, cache-tier reuse, errors and memory against the control.

Native GPU byte tests and CPU regressions are prerequisite evidence; they do not establish TEE/PPCIe performance. No deployment is included in the preparation change.

Runtime preparation PR: #252

Activation PR: #255 (tracking #254). The signed image was published successfully in https://github.com/nearai/cvm-compose-files/actions/runs/34515370239. Scan: zero fixable CRITICAL findings and 62 fixable HIGH findings. Exact-topology staging, rollback qualification and production approval remain pending; no production deployment has been performed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions