Skip to content

fix(core): add transformed ESM fallback for ESM server bundle execution - #8290

Draft
SyMind wants to merge 22 commits into
mainfrom
systemjs-runner
Draft

fix(core): add transformed ESM fallback for ESM server bundle execution#8290
SyMind wants to merge 22 commits into
mainfrom
systemjs-runner

Conversation

@SyMind

@SyMind SyMind commented Aug 17, 2026

Copy link
Copy Markdown
Member

Summary

This PR adds a transformed-ESM fallback for evaluating ESM server bundles when Node.js does not expose vm.SourceTextModule.

The server runner now selects an execution strategy based on the bundle format and runtime capabilities:

Bundle output Runtime capability Runner
CommonJS Any supported Node.js runtime CommonJsRunner
ESM vm.SourceTextModule available EsmRunner
ESM vm.SourceTextModule unavailable TransformedEsmRunner

TransformedEsmRunner lazily transforms emitted ESM modules into a small async runtime protocol and evaluates them with AsyncFunction. Bundle modules are managed by a runner-scoped module graph, while external dependencies are resolved relative to the importer and loaded through native import().

This allows Rsbuild to execute ESM server output without requiring --experimental-vm-modules.

Design

This PR:

  • adds TransformedEsmRunner as the fallback when vm.SourceTextModule is unavailable;
  • adds a version-pinned SWC WASM plugin that rewrites ESM imports and exports into the runner protocol;
  • preserves live imported and re-exported bindings through namespace property reads;
  • preserves unbound function calls and tagged-template calls;
  • supports static imports, dynamic imports, cyclic bundle graphs, and top-level await;
  • provides Node-compatible import.meta.url, import.meta.dirname, and import.meta.filename;
  • resolves external modules relative to the importing bundle file;
  • validates missing static exports with native ESM-style errors;
  • preserves source maps and maps runtime errors back to the original bundle locations;
  • builds the WASM plugin in GitHub CI instead of committing the generated binary.

Why not SystemJS

An earlier implementation used SWC's SystemJS transform and evaluated the resulting System.register modules. This works for a closed SystemJS module graph, but it cannot reliably preserve ESM semantics when the graph also contains native external modules.

SystemJS propagates imported values through dependency setters. For a native external module loaded with import(), the setter can only be invoked with the module namespace once. If the external module later mutates an exported binding, the native namespace reflects the new value, but the value copied into the SystemJS-generated local binding remains stale.

For example:

// external.mjs
export let ready = false;

export const mark = () => {
  ready = true;
};
import { ready, mark } from './external.mjs';

mark();
console.log(ready); // The SystemJS fallback could still observe false.

This also affects re-exports and multiple bundle modules importing the same external binding. Native ESM does not expose a subscription mechanism that would let the SystemJS runtime notify dependency setters whenever an external export changes.

The generic SystemJS transform introduced additional complexity:

  • Rsbuild had to implement the SystemJS registration, instantiation, setter propagation, cycle handling, and evaluation lifecycle.
  • The SWC SystemJS transform required an ES5-target workaround for destructuring assignments.
  • Correct execution ordering between bundle modules and native external modules had to be reconstructed by the custom runtime.

The transformed-ESM approach avoids this impedance mismatch. Imported identifiers are rewritten to read directly from the native module namespace, preserving live bindings without requiring external modules to participate in a SystemJS runtime.

Why not module.registerHooks()

A native ESM loader hook could serve bundle output directly from memfs, but it cannot provide safe cache invalidation across rebuilds:

  • module.registerHooks() was added in Node.js 22.15.0, while @rsbuild/core supports Node.js ^20.19.0 || >=22.12.0.
  • Node.js resolves and caches ESM by URL in a cache separate from require.cache. Reusing a stable URL such as /dist/index.js after a rebuild returns the previous module without calling the load hook again.
  • Adding a build ID to the URL, such as /${timestamp}/dist/index.js, loads the latest source but creates a new cached module graph on every rebuild. Node.js exposes no public API for evicting these ESM cache entries, so old module namespaces and their retained state remain alive and memory usage can grow throughout a long-running watch session.

SystemJsEvaluator instead owns a runner-scoped module cache. Replacing the runner releases the whole module graph for garbage collection, while still allowing bundle source to be read from memfs.

References: Node.js module.registerHooks(), ES modules are cached as URLs, and ESM uses a separate cache.

Related Links

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 18, 2026

Copy link
Copy Markdown

Deploying rsbuild with  Cloudflare Pages  Cloudflare Pages

Latest commit: 1ba198d
Status:🚫  Build failed.

View logs

@SyMind

SyMind commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

@codex review

@SyMind
SyMind marked this pull request as ready for review August 18, 2026 08:18

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ac9db0accd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/core/src/server/runner/systemJs.ts Outdated
@SyMind

SyMind commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2159a4aec0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/core/src/server/runner/systemJs.ts Outdated
Comment thread packages/core/src/server/runner/systemJsTransform.ts Outdated
@SyMind

SyMind commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9a94c3b5eb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/core/src/server/runner/systemJs.ts Outdated
Comment thread packages/core/src/server/runner/systemJs.ts Outdated
Comment thread packages/core/src/server/runner/systemJs.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new execution path for ESM server bundles when the runtime does not support vm.SourceTextModule, by evaluating bundles via a SystemJS (System.register) transform + runtime. This enhances compatibility across supported Node.js versions while keeping the existing ESM VM-based runner when available.

Changes:

  • Make the runner factory async and select between CommonJsRunner, EsmRunner, and the new SystemJsRunner based on bundle format and vm.SourceTextModule availability.
  • Introduce SystemJsRunner/SystemJsEvaluator plus an SWC-based transformToSystemJs helper for lazy ESM→SystemJS transformation with inline source maps.
  • Add unit tests covering import.meta behavior, source-map stack mapping, and missing static external export diagnostics.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
packages/core/tests/systemJsRunner.test.ts Adds tests validating SystemJS runner behavior (import.meta, source maps, missing exports).
packages/core/src/server/runner/type.ts Updates runner factory interface to return Promise<Runner>.
packages/core/src/server/runner/systemJsTransform.ts Adds SWC-based ESM→SystemJS transform with inline source map handling.
packages/core/src/server/runner/systemJs.ts Implements SystemJS evaluator/runtime, module lifecycle, and external resolution.
packages/core/src/server/runner/index.ts Updates runner selection logic to include CommonJS and SystemJS fallback, and awaits factory creation.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread packages/core/src/server/runner/systemJsTransform.ts Outdated
Comment thread packages/core/src/server/runner/systemJs.ts Outdated
Comment thread packages/core/tests/systemJsRunner.test.ts Outdated
@SyMind
SyMind marked this pull request as draft August 18, 2026 10:01
@SyMind

SyMind commented Aug 20, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 14dd5716c2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/core/src/server/runner/transformedEsm.ts
Comment thread packages/core/src/server/runner/transformedEsm.ts
Comment thread packages/core/swc-plugins/esm-runner-transform/src/lib.rs
@SyMind SyMind changed the title fix(core): add SystemJS fallback for ESM server bundle execution fix(core): add transformed ESM fallback for ESM server bundle execution Aug 20, 2026
@SyMind
SyMind marked this pull request as ready for review August 20, 2026 03:46

@chenjiahan chenjiahan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m concerned about two things:

  • The potential edge cases introduced by using a plugin to transform ESM into a custom module system
  • The various trade-offs of introducing a Wasm plugin, including bundle size, performance, and maintenance costs

@SyMind
SyMind marked this pull request as draft August 21, 2026 05:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants