[SriRangaTarun]

Motivation

Closes #1697.

--max-weight-staleness bounds how far behind a drained batch's generating policy may be,
but it says nothing about what happens within the batch. The drained batch flows through the
normal consume path and is split into
rollout_batch_size * n_samples_per_prompt // global_batch_size optimizer steps
(miles/backends/training_utils/data.py). With more than one step, steps 2..N update weights
that have already moved away from the policy that generated the data — per-batch off-policyness
stacked on top of the inter-batch gap the staleness filter deliberately allows.

The two knobs are configured independently and nothing connects them, so the compounding is
silent. Fully async assumes one optimizer step per drain; that invariant currently lives only in
a docstring in miles/ray/rollout.py.

Changes

1. Warn at validation time (miles/utils/arguments.py) when --max-weight-staleness is set
   and the effective steps-per-rollout is > 1. Warning, not assertion: multi-step is valid for
   overlapped-async recipes that account for the compounding themselves, and erroring would break
   working runs. The step count is derived from --num-steps-per-rollout when given, otherwise
   inferred from --global-batch-size, so a user who never touches the former still gets the
   warning. --use-dynamic-global-batch-size is exempt — it resizes the global batch to the
   drained batch and therefore always yields exactly one step.

2. Rewrite the --use-dynamic-global-batch-size help text. It described the mechanism
   ("disable trim samples in rollout buffer") rather than the purpose, which is avoiding
   sample-trim waste when the collected count is variable — and, in fully async, pinning the
   drain to one step.

3. Document the invariant in docs/user-guide/fully-async.md, in a new "Steps per rollout"
   subsection under the buffer/staleness arguments, which previously covered off-policyness only
   via --max-weight-staleness.

Tests

TestMultiStepStalenessWarning in tests/fast/utils/test_arguments.py covers the four-way matrix
(staleness × multi-step, via explicit --num-steps-per-rollout and via implied
--global-batch-size) plus the --use-dynamic-global-batch-size exemption.

No behavioural change to training: this only adds a log line and edits help text and docs.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.