Field punning (inherit)

[yannham]

Inherit

Nix has an inherit keyword that is used to define a field with the same name and value as a a variable in scope:

let foo = 1; in
{ inherit foo; }

# is equivalent to
{ foo = foo; }

This kind of syntactic sugar is also present in other languages (Haskell, Rust, OCaml, etc.) sometimes under the name record /field punning.

Nix inherit also supports multiple fields, and the specification of another record to take the fields from:

let record = {foo = 1; bar = 2;}; in
let baz = 3; blarg = 4; in

{
  inherit (record) foo bar;
  inherit baz blarg;
}

# is equivalent to
{
  foo = record.foo;
  bar = record.bar;
  baz = baz;
  blarg = blarg;
}

Recursive records in Nickel

While this is a nice shorthand, inherit is at most a nicety. However, in Nickel, records are recursive by default. This means that the expanded version above will just cause an infinite loop:

{foo = foo}

This make defining fields with the same name and content as an identifier in scope a pain: one either has to change the variable from the beginning:

let foo_ = 1 in
let bar_ = 2 in

{
  foo = foo_,
  bar = bar_,
}

But sometimes that is not desirable. Variables can be defined far away, and can be used in a lot of other places where it makes sense to keep the original name. The only solution left is to bind those variables to new ones locally, but this must be done before the definition of the record:

let foo = 1 in
let bar = 2 in

{
  sub_record =
    let foo = foo_ in
    let bar = bar_ in
    {
       foo = foo_,
       bar = bar_,
     }
}

Another case that I encountered in practice is when you need to recursively refer to a field defined in a parent record, but with the same name. There, you can't rename the original identifier either:

{
  meta = "metadata",
  sub = let meta_ = meta in {
    meta = meta_
  }
}

This starts to be really painful. Because records are recursive by default, I think having field punning is even more important for Nickel than for other languages.

Describe the solution you'd like

Have a syntactic construct, which can be inherit or something else, to define fields with the same name and value as a list of identifiers in scope.

Note that the common punning syntax for several languages (Rust, OCaml, etc.) is to just write a field without a definition. For example in OCaml:

type r = {foo : int}
let foo = 1 in {foo;}

Or in Rust:

struct Foo { bar: i32 }

fn mk_foo() -> Foo {
  let bar = 1;
  Foo {bar}
}

But this solution doesn't cope well with the general syntax of Nickel.

{
  foo | Num = 1, # a field with a contract and a definition
  foo | Num, # a field with a contract but without definition
  foo, # a field without a definition
}

a lone field named foo, is taken to be a field without definitions, and this is consistent with other notations. Beside being a breaking change, I don't think that would be intuitive to change this syntax to mean inherit foo.

[francois-caddet]

As discussed during last nickel meeting, keeping the Nix syntax ({ inherit var, ...}) is probably better.
@yannham also proposed to have a more intuitive syntax for inherit(record) var which is inherit var from record.
I agree with this proposal, but also feel it less a requirement for nickel than the simple form inherit var which as said above, is the only way to write something meaning let a = something in {a = a} in nickel because of records recursivity.

[toastal]

PureScript, JavaScript, et.al. would let you use the variable as a label such that { foo } ≍ { foo = foo } (though I think inherit ideas offer more flexibility)

[yannham]

@toastal, the end of the original post covers this case and explains why this wouldn't be optimal.

We have a notion of fields without a definition, and in general something that could be described as "partial configurations". Currently, {foo} is already a valid value, which is a record with a field foo that doesn't have yet a definition (which can be seen as a contract as well, which simply requires the existence of a field foo). While the {foo} ~ {foo = foo} is a natural and reasonable approach in other programming languages, that wouldn't work as well in Nickel, unfortunately.

[olorin37]

Yeah, it is indeed very unfortunate that undefined field notation is the same as punning could be. It would be very natural even for people comming from js, where you can write: const some_field = 7; const an_object = { some_field } and it is very handy... That's petty this opportunity has been lost (for making Nickel familiar for mainstream, in this case).

On the other side I think it would be good to avoid inherit keyword, it is of course sensible (especially comming from nix, but not ony), but:

• it is quite long,
• and more importantly, it will cause unnecessary associations with OOP for most adopters.

I have a bunch of ideas:

• symbolic:
  1. let foo = 7 in { <- foo }trying to allude to traditional assignment from pseudocode (or R, Haskell)
  2. let foo = 7 in { *foo }here is a pointer association or rather YAML aliases (and anchors)
• keyword:
  3. let foo = 7 in { put foo }
  4. let foo = 7 in { use foo }
  5. let foo = 7 in { add foo}

What do you think? @yannham?
I do not know if those examles are not in collision with other syntax of Nickel, didn't analyse it, are they?

[yannham]

I personally prefer keyword-based solutions, because they don't assume that the reader of Nickel code is familiar with C-like languages, with JavaScript, with functional programming or whatnot. Although it does have a cost: it's more verbose, and it sometimes prevents said keywords from being used as an identifier (which is backward incompatible). That being said, in this case, because the syntax of field names is very restricted, it's probably never ambiguous (that is, reserving a keyword like use or add in this position doesn't prevent from using it as an identifier elsewhere, because it never clashes).

I'm not too worried about the size of inherit, but the unnecessary association with OOP is a good point. add is confusing, because it makes you think of number addition. put is short and nice, but has an "imperative" feeling IMHO - like we are mutating something, which we aren't. use is a good one; we just have to be careful if one day we want to also have something like Nix with - but more principled, as described in this blog post - because use would be a good candidate for that as well, and I'm not sure the two would coincide.

Naming is hard 🙃 we can also add include to the possibilities

[toastal]

Is OOP still as pervasive as it was in the ’90s & ’00s?

[olorin37]

Yep, I agree with your conclusions about add and use keyword. Also put gives false intuition with mutations, but in this case I would treat it as "put this to the object which is under construction right now" not "put something to object already created", so, I would still consider it as good option. include sounds very good, so probably it is better than inherit (although same size).
Also insert could be considered, one letter less, but again it gives intuition related to mutations.

So I think include is a winner.

Is OOP still as pervasive as it was in the ’90s & ’00s?

@toastal yep, sure :)

[yannham]

I have a first implementation of this in #2204. However, a number of design questions arose during the implementation. I'm reporting them here for debate:

1. Should we allow to both include a field and also define it piecewise, as in {include foo, foo.bar = 1}? We could give a meaning to this expression using merging, but it might also be error-prone and unintended. For example, Nix doesn't allow to do so. It can still be useful if you want to enrich something from the environment, because otherwise, once again, you'd need to do something like let foo_ = foo in { foo = foo_ & {bar = 1}} or let foo_ = foo in { foo = foo_, foo.bar = 1}. On the other hand, maybe this is complex enough that we'd require this explicitness.
2. Should we allow to attach metadata to included fields? For included variables, I think the answer is yes - there is no good reason we shouldn't. You might want to include the content from an existing variable but still add some metadata for this specific record. I guess the syntax would be {include foo | doc "foo"}. However, given Forward the metadata when re-exporting a field #1968, the semantics for extracting fields from other records could reasonably be to take the metadata from there as well: in let foo = {bar | doc "hello" = true} in {include foo.bar}, we would propagate the documentation of bar. In fact, it's probably the main motivation for supporting include on record fields, because one could just do {bar = foo.bar} otherwise. There is no name clash because of recursivity (as opposed to {foo = foo}). Anyway, [Feat] Field punning (aka include aka inherit) #2204 doesn't support such includes, for now.
3. What should be the syntax for including many fields at once, if any? We could either:
   1. Not support this. You'd have to write {include foo, include bar, include baz}.
   2. Use a space delimiter. This is the more succinct way that doesn't clash with the current syntax: {include foo bar baz}. However, if we want to include many fields from one record, we'd need delimiters of some sort, as in {include foo.(bar baz)}, which isn't very consistent.
   3. Use the list syntax. It's a bit more verbose, but we reuse an existing constructs, hence the meaning should be clear: {include [foo, bar, baz]}. We would have a uniform syntax for including record fields: {include foo.[bar baz]}. One drawback is that it may look like the user could provide an arbitrary expression that evaluates to an array, as in {include [foo] @ [bar]}, which is obviously not allowed.
   4. Use a different delimiter for an include expression, such as ;, which would give back , as a delimiter within the include: {include foo, bar, baz;}. As for (ii), we still need an additional delimiter for field includes, because in {include foo.bar, baz;} it's not clear if we mean {include foo.bar, include foo.baz} or {include foo.bar, include baz}.

For multi include syntax, I think I like (ii) the most aesthetically, but I'm partial to (iii) for consistency.

[jneem]

Given the way that merging is currently used for piecewise definitions, I think that allowing it with include is pretty natural. Does nix document why they don't allow it?

I hadn't really thought about propagation before, but it seems reasonable to me.

For multiple fields, I think I like (iii) best. I don't find it too surprising that only list literals are allowed, and we already have some precedence in the import syntax: the filename looks like a string, but only string literals are allowed.