Skip to content

[RL][Phase 5] Implement bounded replay and an end-to-end self-play smoke test #1080

Description

@utilForever

Parent: #1057
Epic: #1052

Goal

Retain recent valid self-play samples in the smallest bounded replay implementation needed by the MVP.

Scope

  • Append complete episode samples
  • Sample mini-batches for both trainers
  • Enforce a deterministic capacity and eviction rule
  • Preserve schema/version metadata
  • Exercise episode generation, serialization, replay insertion, and sampling together

Non-goals

  • Prioritized replay
  • A database service
  • Distributed writers
  • Long-term archival

Acceptance criteria

  • Capacity is never exceeded.
  • Eviction and fixed-seed sampling are reproducible.
  • C++ and Python consume replay batches with the same values.
  • Interrupted or invalid episode writes do not add partial samples.
  • One complete mirror-match episode reaches both trainer batch loaders.

Verification

Add focused replay tests and one end-to-end self-play-to-batch smoke test.

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-rlCategory: Search, self-play, reinforcement learning, and training.P-importantPriority: Other work depends on this, or it is low-level and critical.T-featureType: New capability or supported behavior.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions