Skip to content

[CLI] Add a native Wasmtime Playground CLI with PHP 7.4–8.5 and extensions - #3837

Draft
adamziel wants to merge 31 commits into
trunkfrom
recovered-wasmtime-playground-cli
Draft

[CLI] Add a native Wasmtime Playground CLI with PHP 7.4–8.5 and extensions#3837
adamziel wants to merge 31 commits into
trunkfrom
recovered-wasmtime-playground-cli

Conversation

@adamziel

@adamziel adamziel commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator

What it does

Adds a Node-free WordPress Playground runtime backed by persistent PHP WASI Preview 2 components and Wasmtime. It supports start, server, run-blueprint, and build-snapshot, including WordPress installation, mounts, snapshots, real PHP CLI/WP-CLI execution, configurable parallel workers, SQLite range locks, and process-local WAL coordination.

Adds an unpublished-to-npm @wp-playground/cli-native package as a drop-in experiment for the declared @wp-playground/cli surface. The private package provides ESM, CommonJS, declarations, wp-playground-cli, structured one-shot/server results, an async-disposable server, and a protocol-v2 Playground proxy.

The runtime now ships base and extended WASIp2 components for PHP 7.4, 8.0, 8.1, 8.2, 8.3, 8.4, and 8.5. Extended components contain Redis 6.3.0, Memcached 3.4.0, and the PHP-series-compatible Xdebug release. The base component remains the default; Redis and Memcached follow the Node CLI's JSPI default at the npm boundary, and explicit flags select either profile.

This PR remains a draft and does not publish to npm or add a stable release path, embedded native executable, .cwasm, or default production download URL. The package stays private, and host acquisition requires an explicit WP_PLAYGROUND_NATIVE_HOST_BASE_URL. A one-off GitHub prerelease is available solely for macOS testing.

Rationale

The original core-Wasm prototype reproduced a large PHP host environment in Rust. At the giant-host baseline, host.rs plus php.rs occupied 23,041 physical lines. The current PHP/WASIp2 host, component bridge, and authored WIT total 10,629 lines, a 54% reduction while also adding real PHP CLI, WAL, cancellation, PHP 7.4–8.5, and extensions. The complete Rust src/ tree is 41,845 lines versus 52,795 at that baseline.

The portable unit is the PHP component, not the Wasmtime host. Wasmtime depends on native JIT, virtual memory, and OS handles, so the npm shell carries raw component Wasm and acquires the correct verified native host on first use. runtime install can prewarm Wasmtime's platform cache; no source build or Wasm-to-native conversion is required at ordinary startup.

SQLite normally shares its wal-index through xShm* memory. Independent component workers cannot share guest memory, while rollback-journal mode serializes mixed and admin traffic. The native host therefore keeps a sharded canonical wal-index and exchanges typed deltas with each worker at SQLite lock and barrier boundaries.

Implementation

  • Gives every HTTP worker an independent warm Wasmtime Store and persistent PHP instance around one shared compiled component. Explicit pools stay warm; automatically sized excess workers retire after 10 seconds; workers recycle after 3,000 requests by default.
  • Adds a real CLI SAPI export. playground.cli(argv, { env, cwd }) reserves pool capacity, starts a fresh interruptible component, streams stdout/stderr, returns PHP's exit code, and runs the bundled WP-CLI Phar.
  • Builds 14 checksum-addressed PHP components: base and extended variants for PHP 7.4–8.5. The build pins PHP sources, WASI SDK 33, SQLite 3.51.3, extension sources, patches, and toolchain inputs.
  • Selects the extended component only when Redis, Memcached, or Xdebug is enabled. Xdebug is explicit; network-extension defaults match the upstream Node/JSPI boundary. The runtime provides outbound TCP and DNS but continues to deny guest TCP bind and UDP.
  • Adapts PHP 7.4's integer-returning zend_list_free() through an exact void-returning resource-destructor callback, avoiding a typed WebAssembly call_indirect trap during resource release.
  • Implements SQLite byte-range locks and typed WAL shared-memory open, reset, current-epoch, and atomic exchange operations. Coordination is process-local; separate native CLI processes must not concurrently open the same mounted WAL database.
  • Uses a loopback-only bearer-authenticated control channel with bounded protocol-v2 frames, incremental streaming, backpressure, cancellation, disconnect cleanup, and trap recovery. Controlled host processes are terminated when the Node owner exits or is interrupted.
  • Keeps Rust authoritative for command semantics through a no-acquisition argv probe. The Node layer validates upstream command/option shapes, rejects unsupported values before host acquisition, and exposes the supported runCLI, parseOptionsAndRunCLI, server, event, filesystem, and UniversalPHP surfaces.
  • Supports Studio's existing child-process contract without an API adapter: fresh and persisted sites, PHP 7.4–8.5 configurations, phpMyAdmin, post and Site Editors, running-site studio wp, restart persistence, Blueprints, Xdebug, and JSPI-gated Redis/Memcached.
  • Keeps publication and package boundaries fail-closed: no lifecycle scripts, default public URL, native host, .cwasm, Cargo output, or stable release workflow enters the private tarball.

Experimental macOS prerelease

Download the one-off macOS prerelease. It targets exact source commit 9a437319d6c5ac47d92cb8f0f265e6ed1b36914b, is marked prerelease and non-latest, and contains one private npm tarball plus separate Intel and Apple Silicon gzip hosts, a combined manifest, provenance, and checksums. It is not published to npm.

The PR-scoped build workflow has read-only repository permission. It built and verified each architecture independently, assembled flat GitHub asset paths, and then installed the exact final tarball on fresh macOS x64 and ARM64 runners. Both final runners acquired their hashed host and passed PHP 8.2 HTTP and CLI execution before the inspected bytes were published.

Testing instructions

Run the repository and installed-package gates:

npm exec -- nx run playground-cli-native:verify --output-style=stream
WP_PLAYGROUND_NATIVE_KEEP_VERIFY_TEMP=1 \
  npm exec -- nx run playground-cli-native:verify:installed-package --output-style=stream

Run the real PHP/extension matrix with Redis and Memcached on PATH:

nix --extra-experimental-features 'nix-command flakes' \
  shell nixpkgs#redis nixpkgs#memcached -c \
  packages/playground/cli-native/scripts/verify-php-runtime-matrix.sh

The final local repository gate passed 133 npm tests, 371 Rust library tests, 20 Rust binary tests, all 16 serial real-process native smokes, and all 188 existing Node Playground CLI regression tests. The PHP matrix passed all 21 PHP 7.4–8.5 cases: base/parallel/resource behavior, real Xdebug DBGp handshakes, and real Redis/Memcached TCP round trips. All 14 checked-in component digests match php-assets.json; isolated PHP 7.4 base and extended rebuilds are byte-identical.

The lifecycle-disabled installed-package gate passed against the final staged tarball, including package-boundary checks, concurrent acquisition, offline reuse, both-profile prewarm, packaged WordPress/Blueprint/snapshot, ESM/CJS/bin, and live proxy/control smokes. Its SHA-256 is 1c6e1d3c664de030f81ef5dde0b9c7fac736d8b53e1323ca79795c5bf46543b5; the matching compressed Linux x64 host fixture is 645b6787e8c9cb890ed919c0f07bfe02f1a80900e0ac236d0be3879552590d94.

The external Studio harness uses the same tarball and native-host fixture in two modes:

# Node 22: upstream base-component gate; Redis and Memcached remain disabled.
npm run cli:test-native-integration -- \
  --native-package-tarball /absolute/path/to/wp-playground-cli-native.tgz \
  --native-host-fixture /absolute/path/to/fixture \
  --wp-cli-phar /absolute/path/to/wp-cli.phar \
  --log /absolute/path/to/verify-node22.log

# Repeat under Node 24 with redis-server and memcached on PATH.

Both final reports have status PASS. They cover the unchanged Studio child contract and an ordinary built studio CLI: create/start/stop/restart, fresh WordPress installation, frontend, admin, post editor, Site Editor, phpMyAdmin, concurrent HTTP/WP-CLI WAL access, in-flight cancellation and recovery, Blueprints, PHP 7.4–8.5 selection/reopen, and Xdebug. Node 24 additionally performs real Redis and Memcached set/get/delete operations through both HTTP and studio wp for every PHP version. This is local integration evidence; the Studio harness is not part of this repository's CI.

The tables below are the latest matched WAL benchmark set. They were collected before the PHP-version and extension artifact update, so they describe the current concurrency/WAL design rather than the exact final component binaries. Six workers were pinned to six physical cores; the load generator used six other cores.

Workload Node Playground CLI Wasmtime WAL nginx/native PHP
Public WordPress, concurrency 6 12.32 req/s 49.51 req/s 64.64 req/s
Mixed WordPress, concurrency 6 9.04 req/s 36.63 req/s 54.33 req/s
Admin WordPress, concurrency 6 6.25 req/s 20.27 req/s 37.89 req/s
Site Editor metric Node Playground CLI Wasmtime WAL nginx/native PHP
TTFB 766.8 ms 274.7 ms 128.4 ms
FCP / first meaningful paint 1,560 ms 1,104 ms 932 ms
Fully loaded 3,813.1 ms 2,508.8 ms 2,249.9 ms
CPU workload Node Playground CLI Wasmtime WAL nginx/native PHP
Public, CPU/request (average cores) 445 ms (5.39) 109 ms (5.14) 50 ms (3.26)
Mixed, CPU/request (average cores) 533 ms (4.79) 145 ms (5.12) 68 ms (3.68)
Admin, CPU/request (average cores) 817 ms (5.13) 253 ms (5.20) 113 ms (4.30)
Memory metric Node Playground CLI Wasmtime WAL nginx/native PHP
Warm-idle PSS 1,625.9 MiB 327.0 MiB 145.1 MiB
Peak-active PSS 1,672.1 MiB 358.7 MiB 145.5 MiB
Peak cgroup memory 1,713.5 MiB 359.6 MiB 131.0 MiB

PSS is the primary memory comparison because summed RSS counts PHP-FPM's shared libraries and shared OPcache mapping once per child. Final-head CI at 9a437319 is green: 62 checks passed and 3 intentionally skipped, including all six native OS/architecture jobs, the PHP WASI component build, and all five macOS prerelease build/assembly/exact-final checks.

@adamziel
adamziel force-pushed the recovered-wasmtime-playground-cli branch from 2ce9526 to 8ca5193 Compare July 3, 2026 15:07
@adamziel
adamziel force-pushed the recovered-wasmtime-playground-cli branch 3 times, most recently from bad74d7 to bf49548 Compare July 5, 2026 17:11
@adamziel adamziel changed the title [CLI] Add experimental native Playground CLI host [CLI] Add experimental Wasmtime Playground host Jul 11, 2026
@adamziel
adamziel force-pushed the recovered-wasmtime-playground-cli branch from a5912f2 to f7a7daa Compare July 12, 2026 01:14
@adamziel adamziel changed the title [CLI] Add experimental Wasmtime Playground host [CLI] Add a component-based Wasmtime Playground runtime Jul 12, 2026
@adamziel
adamziel force-pushed the recovered-wasmtime-playground-cli branch 6 times, most recently from ae0f4c9 to f22ecf5 Compare July 12, 2026 14:51
@adamziel adamziel changed the title [CLI] Add a component-based Wasmtime Playground runtime [CLI] Add a native Wasmtime runtime backed by a PHP WASI component Jul 12, 2026
@adamziel
adamziel force-pushed the recovered-wasmtime-playground-cli branch from f22ecf5 to 9445759 Compare July 13, 2026 15:21
@adamziel adamziel changed the title [CLI] Add a native Wasmtime runtime backed by a PHP WASI component [CLI] Add a Wasmtime runtime and private native CLI package Jul 16, 2026
adamziel added 9 commits July 16, 2026 19:33
Encode zero-valued filesystem timestamps without treating them as missing so packaged WordPress scripts remain cacheable while mounted edits still revalidate. Return the first Wasmtime-owned PHP output buffer directly instead of copying it.
Mirror each component's private wal-index through a process-local canonical host image using typed open, reset, epoch, and atomic exchange operations. Synchronize at SQLite lock and barrier boundaries, including exclusive-mode transitions and close/reopen recovery, while retaining per-file lock sharding and bounded delta history.
@adamziel
adamziel force-pushed the recovered-wasmtime-playground-cli branch from 7e206b0 to f64f392 Compare July 16, 2026 19:01
@adamziel adamziel changed the title [CLI] Add a Wasmtime runtime and private native CLI package [CLI] Add a Wasmtime runtime and private native CLI compatibility layer Jul 17, 2026
@adamziel adamziel changed the title [CLI] Add a Wasmtime runtime and private native CLI compatibility layer [CLI] Add a native Wasmtime Playground CLI with PHP 7.4–8.5 and extensions Jul 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant