Fix processwrapper in multiplatform builds

[dzbarsky]

The problem is that cfg = "exec" is not well-defined when you have multiple exec platforms. We want to build the wrapper for the same platform that the toolchain will be used on, and the way to achieve that is by making it an attribute of the toolchain itself.

Previously, building on a linux remote executor from a windows host was broken.

[UebelAndre]

I haven't fully understood the implementation here but I feel like cfg = "exec" should be whatever the configuration is for the platform running an action. If this is not the case I feel this should be addressed in Bazel directly and not something rules have to account for. Is there an issue for this somewhere already?

[dzbarsky]

I haven't fully understood the implementation here but I feel like cfg = "exec" should be whatever the configuration is for the platform running an action. If this is not the case I feel this should be addressed in Bazel directly and not something rules have to account for. Is there an issue for this somewhere already?

It looks like this may not be necessary for correctness in rules_rust today as we are emitting a single action, but as soon as we have cases where a rule does multiple actions, it will be broken because there's nothing forcing the tool into the same exec as the toolchain. (This is what exec groups do, so fixing it that way could be another fix).

However, I do think it's nicer to have this binary (which is conceptually part of the toolchain needed to run rustc_compile) be on the toolchain itself. That means third-party rules that need to rust rustc_compile can just use the toolchain without needing to add an extra magic attribute; i.e. the toolchain is encapsulated better.

cc @fmeum in case you have additional thoughts.

[fmeum]

I haven't fully understood the implementation here but I feel like cfg = "exec" should be whatever the configuration is for the platform running an action. If this is not the case I feel this should be addressed in Bazel directly and not something rules have to account for.

This behavior wouldn't necessarily be correct: An action could be executed on Windows but run a Linux tool (configured for a special Linux exec platform) in a container. It also wouldn't be possible to implement this since the action is considered by arbitrary Starlark in the rule impl, but the configuration of the deps has to be known before that's executed.

@dzbarsky An alternative would be a dedicated toolchain that's resolved together with rustc.

[dzbarsky]

I haven't fully understood the implementation here but I feel like cfg = "exec" should be whatever the configuration is for the platform running an action. If this is not the case I feel this should be addressed in Bazel directly and not something rules have to account for.

This behavior wouldn't necessarily be correct: An action could be executed on Windows but run a Linux tool (configured for a special Linux exec platform) in a container. It also wouldn't be possible to implement this since the action is considered by arbitrary Starlark in the rule impl, but the configuration of the deps has to be known before that's executed.

@dzbarsky An alternative would be a dedicated toolchain that's resolved together with rustc.

True, do you think that's preferable though? I feel like that would also leak this implementation detail to consumers (they'd need to depend on an extra toolchain when conceptually they just want a rust compiler). By consumer I mean rulesets that want to invoke rust compilation such as https://github.com/google/crubit etc

[fmeum]

Right, the toolchain would have to be a toolchain requirement of the rust toolchain target so that it's picked up automatically. But that's probably just more complex than what you are doing in this PR.

[UebelAndre]

This behavior wouldn't necessarily be correct: An action could be executed on Windows but run a Linux tool (configured for a special Linux exec platform) in a container. It also wouldn't be possible to implement this since the action is considered by arbitrary Starlark in the rule impl, but the configuration of the deps has to be known before that's executed.

This feels like quite a far-fetched example but in this case, I would still expect this to be handled by something natively in Bazel.

1. I'm not sure I'd consider windows the "exec" platform in this case. If the goal is to run something in a container, then that feels more descriptive of an "exec" platform. The only trick would be how you start a container in a potentially cross-platform way.
2. If I had to somehow communicate that in a single action, one binary is cfg = "exec" for windows and another cfg = "exec" for a linux container, I'd expect to either change the linux cfg to something more nuanced or have some additional builtin attribute to communicate that "when exec == windows, build for exec == linux".

Neither are really providing solutions here but do they seem directionally correct to folks?

[fmeum]

The particular example is indeed contrived, but the more important point is that dependency configurations can't be determined based on how these deps' artifacts end up being used in the current rule - at least not with the flexibility that Starkark rule implementation functions have today.

At least for now there's not better option than labeling the dep edges with the correct exec group, which does indeed require quite a bit of care in the special case of a toolchain implementation.

[dzbarsky]

I think this would be a bit cleaner by reusing the same _process_wrapper attribute in both bootstrap and bootstrapped cases, to allow this to be swapped more cleanly. Will give that a try!

[UebelAndre]

Thanks! Sorry for the delay here. Some questions and requests for you!

[UebelAndre]

Should this be mandatory?

[dzbarsky]

yeah, we never expect it to be missing right? we can force that

[UebelAndre]

If it's internal can we do

Suggested change

	bootstrapping = ctx.attr.bootstrapping,
	_bootstrapping = ctx.attr.bootstrapping,

[dzbarsky]

ok, will do. Though these bits probably need a bigger rethink / turning into actual providers instead of a ToolchainInfo with a ton of fields, in future.

[UebelAndre]

To avoid exposing an additional attribute, would it make sense to have a rust_bootstrap_toolchain rule that is effectively a mirror of rust_toolchain but never exposed publicly?

[dzbarsky]

we can certainly make another rule, but it should still be exposed publicly, so users can define toolchains outside this repository. So I'm not really sure how much that really helps.

[UebelAndre]

My hope is that users shouldn't need to care about bootstrapping. That this could be something done internally for all users. Do you think users will care about how the process wrapper is compiled?

[dzbarsky]

To clarify, by "users" I mean "advanced users specifying their own toolchains" aka me, not "average end user". But what I'm driving at is that toolchains are just DI, and rules_rust has hardcoded the interface, the provider, and the consumer all in the same repo, which is really limiting in flexibility. So moving forward there needs to be a mindset shift that the toolchain may be defined by other repos.

[UebelAndre]

I do want toolchains to be flexible for "advanced users" but I'm not super sure all users should need to be aware of this bootstrapping process for the process wrapper. I think it'd be a huge win to keep that internal and if that means rules_rust has some baked-in bootstrapping toolchains then I think that'd be an ergonomics win where users defining their toolchain only need to think about how their code is compiled and not how to bootstrap the process wrapper. Is that a happy middle ground?

[dzbarsky]

My use case:

"I want the wrapper to come from the toolchain to cleanly support dzbarsky@947022d, which is a C rewrite of the bash/bat process wrapper. There are users in the community concerned with hermiticity who have no problem with a C toolchain but issues with a bash dependency; we should not force it on them. I don't see this getting accepted into upstream rules_rust given all the work @UebelAndre put in to make the rules work without a C toolchain, but that's fine. The whole point of toolchains is to decouple the implementations. In the future I will likely fully rewrite the process wrapper to C to make the cleanup work even more smoothly, and it should be fully drop-in compatible, and that should "just work" via toolchainization."

If your concern is the boostrapping attr being confusing - well, nobody except me is currently configuring external toolchains, and the docs clearly say "do not use", so hopefully they just ignore it?

Beyond that, I don't see how your solution allows me to cleanly sidestep the existing processwrapper machinery in rules_rust, while this PR does. LMK if I'm missing something.

[UebelAndre]

If your concern is the boostrapping attr being confusing - well, nobody except me is currently configuring external toolchains

This is not true, I'm aware of a number of other cases where teams have configured custom rust toolchains.

There are users in the community concerned with hermiticity

Is there an existing issue with the existing bash/batch bootstrapping and hermiticity?

[dzbarsky]

If your concern is the boostrapping attr being confusing - well, nobody except me is currently configuring external toolchains

This is not true, I'm aware of a number of other cases where teams have configured custom rust toolchains.

Have they configured custom toolchains by wiring thing through directly into rust_toolchain rules? Or are you just talking about repository_set/etc? Either way, their existing definitions will continue to work, they can just do nothing and ignore this bootstrapping attribute?

There are users in the community concerned with hermiticity

Is there an existing issue with the existing bash/batch bootstrapping and hermiticity?

Yes, some users build everything from source and making a bazel-built from-source bash is pretty painful.