gleam test should run all targets by default

[inoas]

I accidentially missed adding tests when contributing to gleam/stdlib because I didn't know I had to run --target javascript separatly (--target erlang is run by default it seems?)

[lpil]

An interesting suggestion. Are there other tools and languages that do this?

If we were to have something like this we couldn't do it by default as we don't know which targets are supported by any particular code. There would need to be some additional config to say which ones to run for.

[inoas]

Then the gleam.toml file could set the default targets?

Maybe like so:

[unit-testing]
default-targets = [
  "erlang",
  "javascript"
]

[lpil]

We would need something like that, aye.

I'm interested in what sort of impact this has on the DX of a language etc. Do you know of any other languages or tools that have this behaviour?

[inoas]

Do you know of any other languages or tools that have this behaviour?

No I don't :/.

---

Related: #1628

1. Changing gleam test now could also break some CI/CDs?
   • I still think it might be worth it to at least add an orange warning that target javascript is not tested if no --target (or --runtime) flags are given and it just tests against erlang target/runtime?
2. Having multiple runtimes makes this harder without for instance some toml flags/support.

Could there in terms of toml settings be targets and nested to that runtimes for tests (and builds)?

1. If no runtimes are specified we can fall back to erlang->beam and javascript->nodejs?
2. If no targets are specified we can fallback to target erlang only?

[inoas]

I feel if this is not tackled especially newbies that target JS will think their tests are OK when they maybe are not?

For 1.0 maybe gleam test could just run all targets (and gleam.toml could define which runtime to use for a target with defaults for erlang/nodejs (but could be lumen/deno)?).

[lpil]

I don't think it would be a safe default as the majority of projects wouldn't be designed to work on all targets, and it would be confusing to the user.

[ffigiel]

Many open source projects explain how to run the tests in the README, even if it's something very basic like make test or pytest .. I think that is very useful, especially to new contributors who are not familiar with the project.

Maybe a README would suffice until multiple targets become a popular feature?

[inoas]

Do you mean gleam new generating entries in README.md about running tests against Erlang and/or JavaScript?

[ffigiel]

I meant authors just adding this to their readme manually if they have multiple targets.

But I didn't think about case where a project only has a js target - for that case being able to specify a default target in gleam.toml would be very nice.

[lpil]

You can do that today :)

name = "mylib"
version = "1.0.0"
target = "javascript"