Skip to content

Port the block-level SSD prefix cache (kill O(chain²) snapshot writes) #2580

Description

@jjang-ai

The disk tier writes 4-6 full-KV boundary records per turn (measured 12 GB/90s in a live tool chain; ladder rungs move every turn so dedup never fires; 5 hits/312 misses acceptance) and bakes ~300 MB of SSM state into every record. The Python engine's design — 64-token chain-hashed delta blocks, separated SSM companion store (measured 20x on a 43.7k-token doc), partial-boundary rehydration across restarts, leaf-first forest eviction — is fully mapped with file:line in the speed-parity plan (vmlx-swift#383 head comment + local SPEED-PARITY-PLAN.md §A/§D-P1). Staging: (a) delta store + chain-walk restore behind a flag, (b) restart boundary index, (c) eviction, (d) flip default after the restart-resume matrix beats the snapshot tier everywhere. This also removes the main pressure source behind the allocator collapse fixed in #2572.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions