feat: parquet data format for rollout communication

[erfanMhi]

Overview

This PR introduces Parquet-based serialization for window data communication between trainers and miners, replacing text-based formats with snappy-compressed binary format.

---

Note

Track last seen window and increment the evaluation counter only when the window changes, avoiding per-loop overcounting.

• Trainer orchestration loop (grail/neurons/trainer.py):
  • Add self._last_seen_window to track window changes for evaluation scheduling.
  • Increment self._windows_since_last_eval only when current_window changes; remove per-iteration increment.
  • Add debug logging for window change and counter updates.

^{Written by Cursor Bugbot for commit 8d98ec4. This will update automatically on new commits. Configure here.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)

• dev

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch main

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}