Description
IFD makes the Nix language a Monadic build system. RFC 92 dynamic derivations make the derivation layer a Monadic build system, so that the language doesn't suffer from the disadvantages of Monadicity, such as having to keep the language evaluator alive while building.
This was something I tooted, but got me thinking about a limitation we currently have.
For the purpose of build system analysis, I think we're good, but in terms of being a nice monad, the design seems incomplete.
Specifically we don't have a clean monadic join operation at the derivation level. In the abstract, it is the cleanest operation: you give it an output and as a result you get the output's output.
However, at the derivation level, we don't have this yet. Suppose you have a derivation computation whoseoutputOfdepth isn't known at evaluation time. As an example, the evaluator may instantiate a derivation of which it takes oneoutputOf, and that derivation contains some dynamic logic that determines whether it returns a build plan immediately, or defers to yet another derivation that it generated, and then returns the output of that.
This final output of operation is not an RFC 92 derivation primitive, and would need to be approximated by wrapping the generated dynamic derivation into a derivation that copies the content of the dynamic output into a non-dynamic output.Such an approximation is perhaps good enough for CA floating content, which should then land at the same address, but it does not work for input addressed derivations, and I generally find this to be a crutch. Why would users need to reimplement an operation that is entirely in the domain of Nix itself (copying File System Objects), and get a whole bunch of dependencies involved with that, and yet it isn't even the full solution.
It seems that we have multiple ways of creating an actual join operator.
a. "flatMap drv": Add a sandbox / build behavior that allows the builder to declare that an output is not new content, but instead a rewrite to a given store path.
b. "rewrite modifier": Add an "advanced attribute" that requires e.g. a symlink in$out, which will be dereferenced and turned into a rewrite. (a bit more static than letting abuilderdecide, as in (a), but still good enough)
c. "join drv": Add a primitive derivation (system = "builtin"; builder = "builtin:outputOf"; outputs.out = "${foo}^out^out^out";)
d. "join in output": Allow the outer, drv-generating drv to return not just a derivation as its output, but a deriving pathI quite like join in output because it generalizes dynamic derivations nicely, and it is in line with the broader idea of making outputs, not derivations first class.
It also removes a redundancy between returning derivation content between {The Thing That Replaces
recursive-nix} and$out, which I believe would be a UX problem for derivation generators, having to treat their root derivation differently.See also matrix discussion