Skip to content

fix(core): Add pass observations for replay training, observe_event API, and fix stale legal_actions - #169

Merged
smly merged 10 commits into
mainfrom
fix/bug-replay
Mar 12, 2026
Merged

fix(core): Add pass observations for replay training, observe_event API, and fix stale legal_actions#169
smly merged 10 commits into
mainfrom
fix/bug-replay

Conversation

@smly

@smly smly commented Mar 11, 2026

Copy link
Copy Markdown
Owner

Fix a critical bug in KyokuStepIterator where pass (not calling) decisions were missing from replay iteration, making it impossible to learn when not to call pon/chi/ron.😇

  1. Generate pass (none) observations from replays. KyokuStepIterator now yields synthetic observations for players who implicitly declined a claim opportunity (pon/chi/ron) after an opponent's discard. Each observation is paired with an ActionType::Pass action, enabling behaviour cloning models to learn when not to call — eliminating the positive-action bias present in replay-only training data.
  2. Add observe_event(event, player_id) API. A new method that applies an MJAI event and returns an Observation only when the specified player has legal actions. Returns None for non-decision events (start_game, start_kyoku, dora, hora, ryukyoku, etc.). This simplifies online inference code by combining state update and observation retrieval into a single call. The existing apply_event(event) (renamed from apply_mjai_event) remains available for fire-and-forget replay parsing.
  3. Fix stale legal_actions after start_game. The constructor's internal reset() left current_player = 0 and phase = WaitAct, so get_observation(0) returned stale discard actions before any real events were applied. Added a StartGame handler that sets current_player to the sentinel value and clears active_players, consistent with StartKyoku. Fixed in both 4P and 3P state machines.

@smly smly added this to the v0.5.0 milestone Mar 11, 2026
@smly smly self-assigned this Mar 11, 2026
@smly smly added bug Something isn't working priority: high core labels Mar 11, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves replay-based training fidelity and simplifies online inference by extending replay iteration to include implicit “pass” decisions, introducing an event-driven observation API, and fixing stale legal_actions state after start_game in both 4P and 3P state machines.

Changes:

  • Add synthetic ActionType::Pass samples to KyokuStepIterator/KyokuStepIterator3P for players who could have claimed a discard but did not.
  • Add Python API RiichiEnv.apply_event(event) and RiichiEnv.observe_event(event, player_id) plus integration tests for observe-driven inference flows.
  • Clear stale action state on StartGame and adjust round start/turn state so get_observation() doesn’t surface pre-event discard actions.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
riichienv-core/src/replay/mod.rs Queue and emit synthetic pass observations during replay iteration; add helper(s) to detect immediate claimers.
riichienv-core/src/state/event_handler.rs Track reaction windows by populating current_claims/active_players; clear stale state on StartGame (4P).
riichienv-core/src/state_3p/event_handler.rs Same as 4P plus kita reaction handling; clear stale state on StartGame (3P).
riichienv-python/src/env.rs Expose apply_event and new observe_event convenience API in Python bindings.
src/riichienv/_riichienv.pyi Add stubs/docs for apply_event and observe_event.
tests/env/test_apply_event.py New integration tests validating observe_event behavior in 4P and 3P.
README.md Document the event-driven API and recommended online inference usage.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread riichienv-core/src/replay/mod.rs
Comment thread riichienv-core/src/replay/mod.rs Outdated
Comment thread riichienv-python/src/env.rs
Comment thread src/riichienv/_riichienv.pyi
Comment thread tests/env/test_apply_event.py
Comment thread riichienv-core/src/replay/mod.rs Outdated
Comment thread README.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread riichienv-core/src/state/event_handler.rs Outdated
Comment thread riichienv-core/src/replay/mod.rs
@smly

smly commented Mar 12, 2026

Copy link
Copy Markdown
Owner Author

Fixed. I'm fairly confident this is correct, but will compare behavior cloning models trained before and after the fix to confirm the issue is resolved. Training will take a few hours.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +892 to +905
let skip_check = matches!(
ev,
MjaiEvent::StartGame { .. }
| MjaiEvent::StartKyoku { .. }
| MjaiEvent::ReachAccepted { .. }
| MjaiEvent::Dora { .. }
| MjaiEvent::Hora { .. }
| MjaiEvent::Ryukyoku { .. }
| MjaiEvent::EndKyoku
| MjaiEvent::EndGame
| MjaiEvent::Other
);

with_variant_mut!(self, |s| s.apply_mjai_event(ev));

Copilot AI Mar 12, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

matches!(ev, ...) moves ev (MjaiEvent is not Copy), so ev cannot be passed to apply_mjai_event(ev) afterwards. This should not compile as written. Use matches!(&ev, ...) (or compute skip_check after applying the event) to avoid consuming ev before it is applied.

Copilot uses AI. Check for mistakes.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a false positive. matches!(ev, MjaiEvent::StartGame { .. } | ...) expands to a match expression where every arm uses { .. } wildcard patterns that bind no fields, so no move occurs. ev remains valid for the subsequent apply_mjai_event(ev) call.

@smly

smly commented Mar 12, 2026

Copy link
Copy Markdown
Owner Author

Verified with a quick smoke test. Ran a 3P game with the pre-fix model (seats 0–1) vs the post-fix model (seat 2). The post-fix model correctly produces Pass actions (4 passes observed), while the pre-fix models produce zero — confirming that pass decisions are now being learned.

from riichienv import RiichiEnv, ActionType
from riichienv_ml.agents import Agent

CONFIG_PATH = "src/riichienv_ml/configs/3p/bc_logs.yml"
MODEL_PATH = "/data/workspace/riichienv-ml/3p/bc_logs_6Mr4.pth" # before fix
MODEL2_PATH = "/data/workspace/riichienv-ml/3p/bc_logs_6Mr5_step500000.pth" # after fix

agent = Agent(CONFIG_PATH, MODEL_PATH, device="cuda")
agent2 = Agent(CONFIG_PATH, MODEL2_PATH, device="cuda")
agents = {0: agent, 1: agent, 2: agent2}

env = RiichiEnv(game_mode="3p-red-half")
obs_dict = env.reset()
pass_count = {0: 0, 1: 0, 2: 0}
while not env.done():
    actions = {pid: agents[pid].act(obs) for pid, obs in obs_dict.items()}
    for pid, action in actions.items():
        if action.action_type == ActionType.Pass:
            pass_count[pid] += 1
    obs_dict = env.step(actions)

print(env.ranks(), env.scores())
print("Pass counts:", pass_count)
$ uv run python scripts/test_bc_3p.py 
[2, 3, 1] [43100, 12100, 49800]
Pass counts: {0: 0, 1: 0, 2: 4}

@smly
smly merged commit c6f4887 into main Mar 12, 2026
11 checks passed
@smly
smly deleted the fix/bug-replay branch March 12, 2026 05:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working core priority: high

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants