Porting to webassembly

[saolsen]

Hey,

I'm thinking about trying to get blink running in the browser (via webassembly). My goal isn't just to run c code in the browser (could just use emscripten for that) but to actually run ape x64 executables in an interpreter where I can build a debugger and some visualization tools to see what's happening. (Similar to what blink already does).
I'm wondering if anybody has tried to compile blink for webassembly or what you think some of the challenges would be.

[jart]

Sounds awesome. I haven't tried it myself. I imagine the only major obstacle might be 32-bit. JavaScript only supports 32-bit integers. Blink runs on i386 and other 32-bit CPUs, and is regularly tested on them. However Blink was created in a 64-bit world, and as such, Blink has a 64-bit bias, therefore, you might not get optimal performance out of Blink in your 32-bit environments.

[unicomp21]

This would be awesome!!!

[unicomp21]

WebAssembly/tail-call#15 (comment)

We're still waiting on tail calls for webassembly, lol. Wish there was another performant option.

[unicomp21]

Someday we'll be able to use co_await in webassembly, hope I'm still around to see it happen, lol.

WebAssembly/tail-call#14 (comment)

[unicomp21]

emscripten-core/emscripten#10991 (comment)

[jart]

Blink doesn't need tail calls.

Also, good news! Blink is now stable on 32-bit platforms. I don't know if webassembly is multi-core, but the threading issues Blink was having earlier on 32-bit have been resolved.

Your biggest obstacle is most likely going to be completely replacing everything in blink/syscall.c so it interfaces with WASI or something similar instead.

[unicomp21]

https://github.com/copy/v86

[unicomp21]

@GorNishanov @tlively @madmongo1 @tomoliv30 any ideas on easiest way to implement linux syscall.c for webassembly?

#8 (comment)

WebKit/WebKit#2065

[unicomp21]

Dumb/crazy question, what sort of perf hit if blink runs nested within itself? I'm wondering if the outer vm could implement syscall.c for the inner vm? And handle threading etc. using co_await? Then outer vm could provide stupid simple interfaces for tunneling packets, etc.?

[jart]

It's possible to run Blink within Blink. There's noticeable slowdown, but it's not a showstopper. There's caveats, such as you can only have a single of the nested Blink instances make use of the linear memory optimization. The other nestings need to pass blink -m to turn it off, otherwise the memory allocations will collide.

[Vogtinator]

any ideas on easiest way to implement linux syscall.c for webassembly?

emscripten provides a surprisingly wide set of runtime APIs which might even be enough to "just work" already.

[Vogtinator]

Also, good news! Blink is now stable on 32-bit platforms. I don't know if webassembly is multi-core, but the threading issues Blink was having earlier on 32-bit have been resolved.

With web workers and SharedArrayBuffers it's possibly to effectively have threads and emscripten even exposes those through pthreads as much as possible.

any ideas on easiest way to implement linux syscall.c for webassembly?

emscripten provides a surprisingly wide set of runtime APIs which might even be enough to "just work" already.

I gave it a quick try and it does almost build out-of-the-box with just emmake make -j8. It just complains about sa_len and wait4 missing. With those worked around I get a blink.wasm of unknown quality, not tested.

[jart]

I got it to compile too. When I tried to run Blink in Node, this happened, and I have no idea what it means.

master jart@turfwar:~/blink$ node o//blink/blink
/home/jart/blink/o/blink/blink:2148
  function ___invoke_$struct_Machine*_$struct_System*_$struct_Machine*(
                                    ^

SyntaxError: Unexpected token '*'
    at wrapSafe (internal/modules/cjs/loader.js:915:16)
    at Module._compile (internal/modules/cjs/loader.js:963:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
    at internal/main/run_main_module.js:17:47

I configured the Makefile to build an HTML file and ran it in the browser. I got the same thing.

I'm going to push the fixes I made right now. Could you please help me figure out what's wrong?

[Vogtinator]

You might need newer LLVM or emscripten: emscripten-core/emscripten#12551

I'm using emscripten main with clang 15.0.6 (should be 16 mmeanwhile, but it works 🤷)

[Vogtinator]

Works!

/tmp/blink> node o/blink/blink -m /cwd/third_party/cosmo/tinyhello.elf
hello world

The WASM page size is 64KiB, which means that #14 happens, but by doing the awful hack of just pretending that it's actually 4096 it can even run busybox-static from the host here:

/tmp/blink> (cd /usr/bin; node /tmp/blink/o/blink/blink -m /cwd/busybox-static sh -c "echo Hello world!")
I2023-01-04T19:25:47.197000:blink/syscall.c:2610:42 missing syscall 0x111
I2023-01-04T19:25:47.198000:blink/syscall.c:2610:42 missing syscall 0x14e
warning: unsupported syscall: __syscall_prlimit64

I2023-01-04T19:25:47.200000:blink/syscall.c:1857:42 getrandom() flags not supported yet
Hello world!

It did require some hacks and workarounds though:

• Expose the working directory in the virtual filesystem below /cwd
• emscripten does not pass envp to main, use environ instead (crashes in LoadArgv otherwise)

diff --git a/blink/blink.c b/blink/blink.c
index f6c0506..a241fd0 100644
--- a/blink/blink.c
+++ b/blink/blink.c
@@ -155,8 +155,10 @@ static void HandleSigs(void) {
   unassert(!sigaction(SIGSEGV, &sa, 0));
 #endif
 }
-
+#include <emscripten.h>
 int main(int argc, char *argv[], char **envp) {
+  EM_ASM({FS.mkdir('/cwd'); FS.mount(NODEFS, {root : '.'}, '/cwd');});
+  if(!envp) envp = environ;
   g_blink_path = argc > 0 ? argv[0] : 0;
   GetOpts(argc, argv);
   if (optind_ == argc) PrintUsage(argc, argv, 48, 2);

• Function pointers work differently, they don't really contain an address so comparing > 4096 fails:

diff --git a/blink/debug.h b/blink/debug.h
index a996413..5e64aa9 100644
--- a/blink/debug.h
+++ b/blink/debug.h
@@ -10,7 +10,7 @@
 #define IB(x)                      \
   __extension__({                  \
     __typeof__(x) x_ = (x);        \
-    unassert((intptr_t)x_ > 4096); \
+    unassert((intptr_t)x_ > 0); \
     x_;                            \
   })
 #else

• emscripten's mmap does not support address hints:

diff --git a/blink/memorymalloc.c b/blink/memorymalloc.c
index 3d0113b..b0b9266 100644
--- a/blink/memorymalloc.c
+++ b/blink/memorymalloc.c
@@ -64,6 +64,10 @@ void FreeBig(void *p, size_t n) {
 
 void *AllocateBig(size_t n) {
   void *p;
+#ifdef __EMSCRIPTEN__
+  p = Mmap(NULL, n, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0, "big");
+  return p != MAP_FAILED ? p : 0;
+#endif
   u8 *brk;
   if (!(brk = atomic_load_explicit(&g_allocator.brk, memory_order_relaxed))) {
     // we're going to politely ask the kernel for addresses starting

I built it with emmake make -j8 && emcc -g o//blink/blink.o o//blink/blink.a -lm -pthread -lrt -o o//blink/blink -s INITIAL_MEMORY=1073741824 -s EXIT_RUNTIME=1 -lnodefs.js