Standard interface for alternative backends (compilation targets)

[SichangHe]

Gleam currently compiles to Erlang or JavaScript, officially, but there is an unofficial Nix backend that forks the whole compiler, and there are interests for C/C++ targets, Go Lang, and WASM target.

It would be nice if the implementation of a compilation backend can be separated from the rest of the compiler, so these efforts can be parallelized and the Gleam project does not necessarily have to maintain all of them.

Hope this helps,
Steven Hé (Sīchàng)

[lpil]

Making compiler public compiler APIs would be a much larger maintenance burden than the two existing backends. We have little developer time and very little money to spend here, so unless we get significantly more funding or complete all higher impact work then this is not something that we can do.

[lpil]

Ah! So there's two features:

1. A way to run commands for multiple targets.
2. Some support for other backends.

From my point of view we are extremely low on funding so anything that produces work without clear advantage to all Gleam users isn't something we can consider adding to the backlog. This makes anything relating to additional compiler backends unviable.

[SichangHe]

• A way to run commands for multiple targets.

This is planned, right?

• Some support for other backends.

I think the question is: what is a good way for one to implement a standalone alternative Gleam backend and make it work with the rest of the ecosystem? Glistix forked the whole compiler, which does not seem the nicest to me; maybe we should ask @PgBiel. But, the fact that Glistix works indicates that other "unofficial" Gleam backends could be a common thing, and the Gleam team does not maintain them anyway.

[lpil]

No plans for that currently. If you have ideas make a discussion to detail the use cases, prior art, and what the design might be.

The recommendation is to fork. As I said we are extremely low on funding and cannot afford to spend any time on supporting unofficial Gleam derivatives, and we also want to make very clear that they are unofficial. The more official something looks the more time we have to spend answering support requests which turn out to be about the third party project, and this is already a notable drain of core's limited developer time.

[SichangHe]

• A way to run commands for multiple targets.

This is planned, right?

No plans for that currently. If you have ideas make a discussion to detail the use cases, prior art, and what the design might be.

Are you talking about the former quote or the support for backends?

The more official something looks the more time we have to spend answering support requests which turn out to be about the third party project, and this is already a notable drain of core's limited developer time.

Okay, this is the real big issue here I overlooked. If B runs gleam build to build A's custom target and it does not work, B would likely seek answers from the Gleam project instead of that custom backend.

[lpil]

The former!

[SichangHe]

So, @lpil, the TL;DR seems to be: your take is that Gleam should only support backends that are part of the project itself, and unofficial backends should fork the compiler; having more official backends is desirable but a future work. Did I get it close?

[lpil]

Yes, we will not accept any additional workload. We do not have enough money.

[PgBiel]

Hi,

I'd like to provide my two cents on this.

Having Gleam support compiling to extra targets (invoking 3rd-party compilers or providing APIs), as well as upstreaming those in any way, has already been discussed, and it likely won't happen.

However, one particular pain point right now is that extra targets very much need an extra value for @target(...), however invalid values for @target fail at the parser level. This means that a package author which wants to add support for Glistix users, for example, won't be able to do so without creating and publishing a copy of their package. This means they'd have two copies on Hex: one with only the official targets and one only with an additional (unofficial) target.

This makes me wonder if we can improve this situation. One solution I thought of: what if we allowed specifying an extra-targets = ["nix"] field (bikesheddable name) which would specify extra valid @target values, but those values would always be ignored by the official compiler? That way, one package can optionally have support for JS, Erlang and desired extra targets without affecting users of the official compiler. What do you think?

[lpil]

No changes will be accepted to core for other targets unless we had significantly more funding.