Scala.js binding generator

[vigoo]

Resolves #1159

I've only enabled the tests for linux, because on the macos runner it was extremely slow and on windows the setup-scala action does not seem to properly setup scala.

[tanishiking]

Hi @vigoo! // cc @sjrd @alexcrichton

I was wondering—does this code generator assume a frontend Wasm setup using jco?

Oops, sorry you've already mentioned that!

genreates Scala code that, when compiled with Scala.js, produces JS code compatible with jco/componentize.js
#1159 (comment)

(deleted comment, I misunderstood it 😅)

Currently, Scala.js's Wasm backend only generates modules that depend on JavaScript, meaning they can't be executed in standalone runtimes like Wasmtime. It also doesn't export linear memory or cabi_realloc, nor does it generate instructions that directly manipulate linear memory (e.g., i32.store/i32.load). Because of this, it shouldn't be able to support the Component Model—at least in standalone Wasm runtimes. Does jco provide any helpers for these operations? (I'm not very familiar with jco 😞)

Aside from that, there's an ongoing effort to support the Wasm Component Model and WASI in server-side Wasm runtimes: scala-js/scala-js#5121.

In our implementation, instead of exposing memory manipulation APIs at the source language level and generating Scala code that interacts with them via wit-bindgen, we take a different approach: we keep memory operations hidden from the source language and delegate them to the linker backend. This means the conversion between Scala objects and the Canonical ABI is handled by the linker backend, rather than in user code.
A lot of Scala.js's standard libraries don't work in standalone Wasm runtimes yet, but we're planning to re-implement some of them using pure Scala and WASIp2.

So, if this implementation is aimed at frontend Wasm using jco (and if wit-bindgen community accepts it), great work! This would mean Scala.js will have wit-bindgen support for both frontend Wasm (jco) and server-side Wasm.

If the goal is to develop a wit-bindgen for server-side Wasm, then aligning it with our approach could be great. 🙂 Would love to hear your thoughts!

---

So it goes: Scala.js -(js backend)-> JS - (componentize.js)-> running JS on JS engine compiled to Wasm. In this case, many of Scala.js libraries should work out of the box.

Aside from that, there's an ongoing effort to support the Wasm Component Model by directly generating Wasm Component (compatible) binary: scala-js/scala-js#5121.

Directly emitting Wasm component binary could run faster than JS on JS-Engine in Wasm, but it would take some time to support many of Scala.js libraries.
While we continue development to support directly emit Wasm binaries, I also believe that supporting Wasm via a JS engine on Wasm is a good approach to quickly (can reuse JS ecosystem) and reliably (no need for GC/Exception proposals in VM) providing users with Wasm Components, great job 👍 🚀

[vigoo]

The direct support would definitely be better than this solution, once it's done! Looking forward to it.

[sjrd]

Hi. Lead maintainer of Scala.js here.

I am honestly baffled that I never heard of this before just now. It's lucky that we even notice this PR at all.

The solution presented here (Scala.js->JS->Wasm) is better than nothing; that's for sure. It's a good workaround if you desperately want to use Scala in the component model today, rather than in 6-12 months. But I don't think this is the solution we want to present under the raw scalajs namespace.

As @tanishiking mentioned, direct Scala.js support for the Wasm component model is under construction (we already shipped a fully functional Scala.js->WasmGC-with-JS-host backend). But the source API won't look anything like this, since it won't actually use JS types. Once it's there, I would hope we could claim the scalajs namespace for the direct solution. We won't be able to do that in a backward compatible way if we already shipped the current workaround under that namespace.

I suggest to rename the crate so that it is clear that it goes through jco. Perhaps scalajs-jco? (no idea whether - is a thing Rust people use in their crates' names, sorry)

[sjrd]

Given that Scala has nullable reference types by default, the consensus in Scala.js is to represent "nullable A" as A. So a nullable String is a String.

For primitives, we use their boxed variants : a nullable Int is a java.lang.Integer.

[vigoo]

This is added to not loose the WIT type information of something being optional<>. Even though Componentize.js decided to represent optionals as nulls we don't have to expose it like that for Scala, in my opinion.

[sjrd]

I disagree. The philosophy of Scala.js is to present JS as it is, without artifacts. Then, user-land libraries can build abstractions on top of the real thing.

Also, when targeting Scala 3, we can actually write A | Null, so that users using -Yexplicit-nulls can benefit from it all.

[vigoo]

I understand this philosophy of Scala.js - but in this case I'm not trying to present JS at all - I'm trying to present a component model interface which has optionals. Having JS in between is just a temporary "annoyance" that, as you wrote earlier, will go away.

So in my opinion the users of this would like to define a WIT interface containing optionals, and implement it in Scala. How it is represented in JS is not relevant for them.

About targeting Scala 3, I've added a command line option to set the dialect (2 vs 3) but not really taking advantage of it yet, it can be added easily.

[vigoo]

@sjrd I'm fine with not calling it scalajs bindgen but something more specific, or to just get rid of it once the real solution is done. For now this provides our users a way to use Scala right now - we have an sbt plugin as well that uses our forked bindgen at the moment, but I wanted to get create this PR upstream to contribute back and make it more accessible for people.

[alexcrichton]

Thanks for the PR here @vigoo and for chiming in @tanishiking and @sjrd!

I can perhaps clarify from my perspective of maintaining wit-bindgen that I think it's reasonable to add this and facilitate it being more visible, but I also definitely would want to make sure that no one feels like their toes are being stepped on. I am no Scala (nor Scala.js) expert myself so I'd lean on y'all to guide how best this can be done.

The suggestion of naming this scalajs-jco or something along those lines sounds quite reasonable to me! It might also be worth documenting how this is a temporary solution with pointers to upstream work too?

[sjrd]

Thanks for the rename. This looks good to me. (although I'm only judging the Scala.js aspects of it; I haven't looked at the Rust code)

[vigoo]

Renamed it to scalajs-jco and applied some of the suggested changes.

Also note that for fully working with Componentize.js unfortunately there are two small tweaks in spidermonkey-embedding-splicer because of some specifics of what JS Scala.js generates. I believe these are small and generic enough to get included in componentize.js (and they are options so does not affect anything existing) and will open a PR for that there too - just our fork is a bit behind and I did not have time to update it yet.

[alexcrichton]

Currently scalajs-jco is ~20 minutes in CI, would it be possible to optimize/cache things to get that more within 10 minutes? Currently C# is around 30 minutes but everything else is 10, and my hope is to stay roughly in the 10 minute time frame for cycle time

[vigoo]

I'm not sure how to speed it up keeping the existing test structure - I believe the artifacts are cache and shared between the test runs, it's just starting up a jvm and sbt has a big overhead.

Maybe instead of generating a separate test case for each codegen test, we could generate one big scala project containing all, and compile it once? That would make it harder to quickly test things while working on the codebase, but would definitely speed up CI significantly.

[alexcrichton]

I'm unfortunately not familiar enough with Scala.js or the infrastructure here to know how best to solve this. If rearchitecting things works well then that seems like the way to go yeah. I'm basically wary of adding more languages that slow down CI here as it's already quite a drag to wait for C# CI when it's significantly longer than all the others.