refactor: Revisit Scheduler's public API #4870

xxuejie · 2025-04-09T06:08:29Z

Previous code marks all fields in the Scheduler structure as public, while at the same time expose enough function to build a debugger. However, by looking at the expose APIs and their docs, one cannot have a clear picture of how to interact with a Scheduler, if they don't read the whole implementation in full.

This change aims at exposing enough public APIs of the Scheduler structure, enabling as much outside usage as possible, while at the same time also maintaining a sane view of the Scheduler structure.

What problem does this PR solve?

Issue Number: close #xxx

Problem Summary:

What is changed and how it works?

Proposal: xxx

What's Changed:

Related changes

PR to update owner/repo:
Need to cherry-pick to the release branch

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code ci-runs-only: [ quick_checks,linters ]

Side effects

Performance regression
Breaking backward compatibility

Release note

None: Exclude this PR from the release note.
Title Only: Include only the PR title in the release note.
Note: Add a note under the PR title in the release note.

xxuejie · 2025-04-09T06:08:47Z

cc @mohanson

script/src/scheduler.rs

Copilot

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

script/src/scheduler.rs:964

[nitpick] Since boot_vm has been converted from a public API to an internal helper, consider renaming it (e.g., _boot_vm) to clearly indicate its internal usage.

fn boot_vm(&mut self, location: &DataLocation, args: VmArgs) -> Result<VmId, Error> {

Copilot · 2025-04-09T07:03:49Z

script/src/scheduler.rs

@@ -138,13 +139,50 @@ where
        }
    }

+    /// If current scheduler is terminated
+    pub fn terminated(&self) -> bool {
+        self.states[&ROOT_VM_ID] == VmState::Terminated


Direct indexing into self.states may panic if ROOT_VM_ID is unexpectedly missing. Consider using a safe accessor like get() with proper error handling.

Suggested change

self.states[&ROOT_VM_ID] == VmState::Terminated

match self.states.get(&ROOT_VM_ID) {

Some(state) => *state == VmState::Terminated,

None => false, // Return false if ROOT_VM_ID is missing

}

eval-exec · 2025-04-09T07:12:33Z

script/src/scheduler.rs

+            });
+        }
+
+        let (id, _) = self.iterate_outer(&Pause::new(), u64::MAX)?;


If self.terminated() is false, should we return the remaining_cycles from self.iterate_outer(&Pause::new(), u64::MAX)?; to exit_status?
Probably not, because IterationResult::exit_status.1 is intended to store the consumed_cycles, not the remaining_cycles?

IterationResult::exit_status.1 should always contain consumed cycles up to the execution point.

eval-exec · 2025-04-09T07:28:55Z

script/src/types.rs

+    /// VM ID that gets executed
+    pub executed_vm: VmId,
+    /// Exit status
+    pub exit_status: Option<(i8, Cycle)>,


Suggest add code comment for Cycle:

pub exit_status: Option<(i8, /*! consumed_cyles */ Cycle)>,

I actually split a separate type to better indicate usage.

eval-exec · 2025-04-09T07:37:45Z

script/src/scheduler.rs

@@ -774,6 +835,16 @@ where
        Ok(())
    }

+    fn terminated_result(&mut self) -> Result<(i8, Cycle), Error> {
+        assert_eq!(self.states[&ROOT_VM_ID], VmState::Terminated);


How about reuse assert!(self.terminated()); and move this method to be adjacent to pub fn terminated(&self) -> bool since they are both related to termination.

xxuejie · 2025-04-10T07:07:01Z

~~Please hold this for now, I'm addressing a potential bug in a tool that relies on this change. Chances are the public APIs are not designed good enough.~~

The bug should be resolved by now. Feel free to review the updated PR.

Previous code marks all fields in the Scheduler structure as public, while at the same time expose enough function to build a debugger. However, by looking at the expose APIs and their docs, one cannot have a clear picture of how to interact with a Scheduler, if they don't read the whole implementation in full. This change aims at exposing enough public APIs of the Scheduler structure, enabling as much outside usage as possible, while at the same time also maintaining a sane view of the Scheduler structure.

Earlier Meepo implementation has a bug: when IO processing code suspends or resumes VMs, the cycles consumed by the suspending / resuming operation will not be reflected in the `current cycles` syscalls made by the subsequent VM execution. While the bug can be thought as a misimplementation, it is causing regression when we are moving IO processing code, and a script is using `current cycles` syscall for eal.

…ight fork

Copilot

Pull Request Overview

This PR refines the Scheduler public API by hiding internal fields, introducing structured results for termination (TerminatedResult) and single-iteration steps (IterationResult), and adding helper methods to inspect scheduler state without exposing internals.

Restrict direct field access by making many struct fields private and providing getters (state, sg_data, terminated, etc.).
Introduce TerminatedResult and IterationResult types and update run and related methods to use them.
Add new public methods: peek, iterate, and terminated, and internal helpers (boot_root_vm_if_needed, iterate_outer, etc.).

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
script/src/verify.rs	Updated return types to use `TerminatedResult` for consistency.
script/src/types.rs	Added `iteration_cycles` to `FullSuspendedState` and new result types (`TerminatedResult`, `IterationResult`).
script/src/scheduler.rs	Made fields private, added getters and new public APIs (`state`, `sg_data`, `peek`, `iterate`, `terminated`), and refactored run/iterate logic to use new result types.

Comments suppressed due to low confidence (2)

script/src/scheduler.rs:285

The doc comment above pub fn run still refers to Result<(i8, Cycle), Error>. Please update the documentation to reflect the new Result<TerminatedResult, Error> return type and describe exit_code and consumed_cycles.

/// * Other terminating errors

script/src/scheduler.rs:304

This new public API iterate (as well as state, sg_data, peek, and terminated) should have corresponding unit tests to cover both normal iterations and termination scenarios.

pub fn iterate(&mut self) -> Result<IterationResult, Error> {

Copilot · 2025-05-22T07:35:04Z

script/src/scheduler.rs

+    /// executed VM ID(so caller can fetch later data). This can be used when more
+    /// finer tweaks are required for a single VM.


[nitpick] In the next line you wrote “more finer tweaks”. Consider changing to “finer tweaks” to avoid the redundant comparative.

Suggested change

/// executed VM ID(so caller can fetch later data). This can be used when more

/// finer tweaks are required for a single VM.

/// executed VM ID(so caller can fetch later data). This can be used when finer

/// tweaks are required for a single VM.

xxuejie requested a review from a team as a code owner April 9, 2025 06:08

xxuejie requested review from doitian and quake and removed request for a team April 9, 2025 06:08

eval-exec reviewed Apr 9, 2025

View reviewed changes

script/src/scheduler.rs Show resolved Hide resolved

eval-exec requested a review from Copilot April 9, 2025 07:03

Copilot AI reviewed Apr 9, 2025

View reviewed changes

eval-exec reviewed Apr 9, 2025

View reviewed changes

eval-exec added t:enhancement Type: Feature, refactoring. m:vm labels Apr 9, 2025

eval-exec reviewed Apr 9, 2025

View reviewed changes

eval-exec approved these changes Apr 9, 2025

View reviewed changes

eval-exec reviewed Apr 9, 2025

View reviewed changes

eval-exec previously approved these changes Apr 9, 2025

View reviewed changes

eval-exec requested review from zhangsoledad and driftluo April 9, 2025 07:39

xxuejie dismissed eval-exec’s stale review via 849dac6 April 9, 2025 07:47

mohanson approved these changes Apr 9, 2025

View reviewed changes

eval-exec previously approved these changes Apr 9, 2025

View reviewed changes

zhangsoledad previously approved these changes Apr 10, 2025

View reviewed changes

xxuejie dismissed stale reviews from zhangsoledad and eval-exec via ac4b624 April 10, 2025 08:05

xxuejie added 3 commits May 13, 2025 06:29

fix: Review fixes

eca9fa7

fix: Move process_io invocation to the last of each iteration

341303a

xxuejie force-pushed the revisit-scheduler-public-apis branch from ac4b624 to 341303a Compare May 13, 2025 06:30

eval-exec previously approved these changes May 13, 2025

View reviewed changes

driftluo previously approved these changes May 13, 2025

View reviewed changes

xxuejie dismissed stale reviews from driftluo and eval-exec via 045645f May 14, 2025 03:38

Add assertion lines, better crash than accepting invalid block that m…

750510e

…ight fork

driftluo approved these changes May 21, 2025

View reviewed changes

zhangsoledad requested a review from Copilot May 22, 2025 07:33

Copilot AI reviewed May 22, 2025

View reviewed changes

Merge branch 'develop' into revisit-scheduler-public-apis

ded45d8

chenyukang approved these changes May 22, 2025

View reviewed changes

zhangsoledad approved these changes May 22, 2025

View reviewed changes

zhangsoledad added this pull request to the merge queue May 22, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 22, 2025

xxuejie added this pull request to the merge queue May 27, 2025

Merged via the queue into nervosnetwork:develop with commit 99cd201 May 27, 2025
33 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor: Revisit Scheduler's public API #4870

refactor: Revisit Scheduler's public API #4870

Uh oh!

xxuejie commented Apr 9, 2025

Uh oh!

xxuejie commented Apr 9, 2025

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 9, 2025

Uh oh!

eval-exec Apr 9, 2025 •

edited

Loading

Uh oh!

xxuejie Apr 9, 2025

Uh oh!

eval-exec Apr 9, 2025

Uh oh!

xxuejie Apr 9, 2025

Uh oh!

eval-exec Apr 9, 2025 •

edited

Loading

Uh oh!

xxuejie Apr 9, 2025

Uh oh!

xxuejie commented Apr 10, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI May 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

-        self.states[&ROOT_VM_ID] == VmState::Terminated
+        match self.states.get(&ROOT_VM_ID) {
+            Some(state) => *state == VmState::Terminated,
+            None => false, // Return false if ROOT_VM_ID is missing
+        }

		/// executed VM ID(so caller can fetch later data). This can be used when more
		/// finer tweaks are required for a single VM.

refactor: Revisit Scheduler's public API #4870

refactor: Revisit Scheduler's public API #4870

Uh oh!

Conversation

xxuejie commented Apr 9, 2025

What problem does this PR solve?

What is changed and how it works?

Related changes

Check List

Release note

Uh oh!

xxuejie commented Apr 9, 2025

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 9, 2025

Choose a reason for hiding this comment

Uh oh!

eval-exec Apr 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xxuejie Apr 9, 2025

Choose a reason for hiding this comment

Uh oh!

eval-exec Apr 9, 2025

Choose a reason for hiding this comment

Uh oh!

xxuejie Apr 9, 2025

Choose a reason for hiding this comment

Uh oh!

eval-exec Apr 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xxuejie Apr 9, 2025

Choose a reason for hiding this comment

Uh oh!

xxuejie commented Apr 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI May 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eval-exec Apr 9, 2025 •

edited

Loading

eval-exec Apr 9, 2025 •

edited

Loading

xxuejie commented Apr 10, 2025 •

edited

Loading