feat(action-plugin): run dynamic-run over RPC - #16256
Draft
rgrinberg wants to merge 6 commits into
Draft
Conversation
rgrinberg
force-pushed
the
push-xzxwkzrvqrtl
branch
from
August 29, 2026 20:55
577c6d5 to
bfd27fb
Compare
Move the reusable thread-pool implementation out of dune-scheduler so lower-level libraries can use it without depending on the scheduler. Parameterize worker creation so the scheduler retains Thread0's signal masking and tracing behavior while other clients can use ordinary system threads. Keep the existing worker-accounting and capacity tests on the scheduler wrapper. Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
Replace the temporary-file dynamic-run handshake with versioned DAP RPC procedures. Dune registers each active action, passes its RPC endpoint and action ID to the plugin, and serves dependency requests while staged execution continues in one process. Provide a monad-parametric DAP client with explicit structured errors, plus a blocking implementation backed by the shared thread pool. Keep the DAP payload representations private while exposing the request witnesses required by caller-provided Dune RPC clients. Record RPC-built dependencies in the action result and remove the obsolete action-execution continuation used to respawn plugins between stages. Update tests, procedure digests, and documentation for single-process staging. Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
Extend the action-plugin helper so a rule producing one target requests a second target whose rule depends on the first. Record the existing timeout to demonstrate that serving the dependency from an unrelated RPC fiber loses the Memo cycle-detection path and deadlocks. Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
Capture the Memo cycle-detection context before action execution and restore it when an RPC handler builds dynamic dependencies. This reconnects the handler's build to the active rule's Memo stack, allowing indirect cycles to be detected before either side blocks. Use the general Memo cycle detector for direct target dependencies as well, removing the duplicate target matching logic and restoring Dune's standard cycle diagnostics. Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
Extend the blocking action-plugin helper to hold an initialized action open and publish its action ID and RPC endpoint. Start a second blocking client with the same ID to demonstrate that the server currently accepts a live action from an unrelated RPC session. Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
Generate unpredictable action IDs instead of process-local counters and bind an action to the RPC session that initializes it. Reject duplicate initialization, uninitialized dependency requests, and requests from any other session. Update the blocking-client regression to verify that a second session can no longer reuse a live action ID. Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
rgrinberg
force-pushed
the
push-xzxwkzrvqrtl
branch
from
August 30, 2026 10:21
bfd27fb to
8ce818b
Compare
rgrinberg
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace the temporary-file handshake used by
dynamic-runwith Dune RPC requests. Keeping the plugin alive lets staged computations preserve process state and avoids re-executing earlier stages while Dune builds newly discovered dependencies.