trunk test

[thedodd]

abstract

Lots of discussion to be had here. The main idea is that we want a trunk test command which will do the right thing in terms of running unit tests, integration tests &c.

A few ideas were being tossed around on the Yew community Discord. Right now, I am definitely leaning towards leveraging wasm-pack test as much as possible. It already handles a lot of the heavy lifting.

Currently, this issue is in discussion phase. We need to determine what would be best for users.

[flosse]

Are there any plans to go further with this discussion?
wasm-pack is currently not maintained and it would be awesome to have s.th. like
trunk watch -t -h firefox where t = test and h = headless that runs the tests automatically :)

[ranile]

I've been playing around with testing without wasm-pack in yewstack/yew#2528. trunk test command could simply do the following:

• ensure wasm-bindgen-cli is there
• pull geckodriver/chromedriver/safaridriver (as per https://rustwasm.github.io/wasm-bindgen/wasm-bindgen-test/browsers.html#configuring-which-browser-is-used-1)
• run cargo test --target wasm32-unknown-unknown

We would need to ask users to ensure that their .cargo/config has the test runner configured though: https://rustwasm.github.io/wasm-bindgen/wasm-bindgen-test/usage.html#configure-cargoconfig-to-use-the-test-runner

[ranile]

I took a shot at implementing it but there's design discussion left to be done:

• How will we ensure that .cargo/config.toml is preset and properly setup? Do we just ignore that? Do we manually pull wasm-bindgen-test-runner and use it (it's not published in the wasm-bindgen releases)
• How will we locate the Cargo.toml file? For build command, we can easily use the index.html. But if we do the same but test, we would be preventing trunk from being used to test libraries. My main use case for this was to use trunk to test Yew, which I ended up doing with wasm-bindgen-cli directly (see Don't use wasm-pack in CI yewstack/yew#2648 and Wasm doc tests yewstack/yew#2651). Also, integration tests (tests/ directory) requires a library crate to be present.
• What about doc tests?

[dnaka91]

Thanks for trying out to implement this. I think this is a rather big change, so I'll go ahead and implement this. Regarding the issues you faced, I think we can solve most of them rather easy.

Easy to solve

• We don't need to care about .cargo/config.toml, as we can use env vars to override the runner. You can see here, that CARGO_TARGET_<triple>_RUNNER exists. We can set CARGO_TARGET_wasm32-unknown-unknown_RUNNER=wasm-bindgen-test-runner and should be good to go.
  • Eventually, it's worth to parse a minimal version of the Cargo config file, both project local and global one, and print a warning if the runner is already set.
• The wasm-bindgen-test-runner is released as pre-compiled binary, as part of the wasm-bindgen releases. We just don't extract it currently.
  • Need to expand our tool download logic, to share the same download archive for the wasm-bindgen-cli and the test runner, so we wouldn't download the same thing twice.
• The browser drivers can probably handled the same way as we do with other tools, downloading pre-compiled binaries automatically if system-wide installed versions don't exist.
  • Might just start with system-only binaries as a starter and add the auto-download later on.
  • Download would only work for the Firefox and Chrome driver, Safari doesn't have anything we could download ourselves.
  • Downloading the driver still doesn't ensure the browser is installed in the right version, too. That's more of a problem with Chrome, but not that bad with Firefox as the driver runs with many versions.
  • Have to look a bit at wasm-pack to find out how they run the Node.js variant. I guess the test runner does that by itself and just uses the node binary if available, failing otherwise.

Problematic

• Locating the Cargo.toml file is probably rather easy, but completely different from the build command. At the same time, we want to support binaries too. So we'd need some hybrid solution that works for projects with and without an index.html.
• Could you share some more details about doc tests? My assumption would have been that cargo runs them the same way as the other tests (plus the wrapping and stuff to run the snippets as tests of course). Have you discovered some differences or problems when running doc tests?