A metaprogramming proposal

[TsavyPrince]

Let's say you want boilerplate-free lenses. Then you might want to write something like:

pub type Lens(s, t, a, b) {
  Lens(get: fn(s) -> a, set: fn(b, s) -> t)
}

pub template lenses(t: RecordType) {
  pub fn lens[t, t.field]() -> Lens($(t), $(t), $(t.field.type), $(t.field.type)) {
    Lens(
      get: fn(x) { x.$(t.field) },
      set: fn(v, x) { $(t.constructor)(..x, $(t.field): v) },
    )
  }
}

which can be type-checked before monomorphization, which is possible because the declarations do not depend on t, and nothing about t is inspected except for the fact that it is a RecordType. For instance, you can't pattern match on the types of the fields because they don't exist before monomorphization.

For pub type Person {Person(name: String, age: Int)}, you use the template by writing derive lenses(Person) which monomorphizes into the following at build-time:

//This code would compile if the function names only had letters and underscores.

pub fn lens[Person, name]() -> Lens(Person, Person, String, String) {
  Lens(
    get: fn(x) { x.name },
    set: fn(v, p) { Person(..p, name: v) },
  )
}
pub fn lens[Person, age]() -> Lens(Person, Person, Int, Int) {
  Lens(
    get: fn(x) { x.age },
    set: fn(v, p) { Person(..p, age: v) },
  )
}

Conceptually, a template takes a SumType, considered as a list of constructors which are themselves lists of fields, and iterates over all its constructors and fields, generating declarations for each. ADTs with one constructor can be considered as a RecordType, which is merely a list of fields, in addition to being a SumType, to avoid Result-heavy logic, which would otherwise be used to account for the possibility of the absence of fields.

Edited: Adjusted syntax for the example template

[lpil]

Sounds interesting! I'd love to learn more.

To evaluate any proposal we need to have a shared understanding of the motivations and use cases for this the feature, and what the alternatives are (both in Gleam today and other potential additions to the language). Could you expand in this area please? 🙏

For the specific example given there I've some questions:

• What is RecordType?
• What is SumType?
• What is the $(...) syntax and what are its rules?
• What is the a[b] syntax and what are its rules?
• What are the rules for the template syntax?
• How does the mentioned iteration work?
• Could you expand on the derive bit and its rules?
• How is this feature typed?
• Are there any other languages which have this feature, or a similar feature?
• What would you say are the advantages and disadvantages of this feature?
• What does this feature enable in Gleam beyond making this sort of boilerplate more compact?

[TsavyPrince]

• Conceptually,

type Field {
  //Accessing the field of an element of a list would normally be invalid, but templates iterate over lists anyway, so it's fine.
  Field(name: String, type: Type)
}

type Constructor {
  Constructor(name: String, field: List(Field))
}

type SumType {
  SumType(constructor: List(Constructor))
}

type RecordType {
  RecordType(field: List(Field))
}

• $(...) receives using record access notation (if a template receives a RecordType, it can't access it using SumType fields and vice versa) according to what I outlined above, and tells the builder where and what to splice into code to monomorphize a template. Splicing is structural so that for instance $t.field cannot splice where it isn't a field of t. Hopefully, Gleam's existing parsing facilities can check this so that syntax errors do not occur after a template generates code. Although, since it probably does not add much more difficulty to parsing, I would recommend dropping the parentheses for just $....
• [b] is to tell the builder to iterate over bs to generate instances in the scope it is enclosed by. So, for instance, pub fn lenses[t, t.field] generates an individual function for each pair (t, t.field) in which t.field depends on t. It doesn't iterate if [...] encloses a splice like [$...]. In addition, I see no reason why [...] cannot be used for field/constructor declarations or expressions either. More examples are shown below.
• template acts like fn syntactically, but it has a single field which is a SumType or a RecordType, and its body is a list of declarations (and perhaps derives) with $(...)s and [...]s. I haven't thought about multiple fields or fields that are other declarations, such as function types or functions. If [...] is confusing or hard to parse then you might want to replace it with <...>.
• derive my_template(MyType) tells the builder to generate code by specializing the template's body with MyType. The code lives in the module in which the template was derived.
• Hopefully, templates can be type-checked using mostly Gleam's existing facilities. It is trivial to type a template's declaration, and I don't think anything beyond whatever is used to type-check Hindley-Milner is necessary to type-check template bodies, because the only template bodies that type-check would be those for which passing any RecordType/SumType generates code that type-checks, and there is no branching on types of an ADT or anything like that.
• GHC.Generics allows one to reflect on the structural definition of an algebraic data type, but makes use of typeclasses and Haskell's more extensive type system. Of course, generating code is nothing new.
• It fits well with Gleam's design philosophy, as one cannot implement things like typeclasses or HKTs and such, which involve global coherence or abstraction over inspection of things that only exist at runtime. It's also readable, easy to run in one's head, and would have good error messages due to templates being checked before generation. Finally, it reduces the Gleam ecosystem's reliance on external codegen tools, which also introduce issues with staleness and refactoring and so on. On the other hand, it requires a new declaration and build phase, so it would require extending tooling. It also introduces a lot of new syntax and concepts.
• You might be able to use it for boilerplate-free serialization and deserialization:

import gleam/json
import gleam/dynamic/decode

pub template json_stuff(t: RecordType) {
  pub type Encoders[t] {
    Encoders(encoder[t.field]: fn($t.field.type) -> Json)
  }

  pub type Decoders[t] {
    Decoders(decoder[t.field]: Decoder($t.field.type))
  }

  pub fn to_json[t](enc: Encoders[$t], value: t) -> Json {
    json.object([
      #[t.field]($t.field.name, enc.encoder[$t.field](value.$t.field)),
    ])
  }

  pub fn decoder[t](dec: Decoders[$t]) -> Decoder($t) {
    use [t.field] <- decode.field($t.field.name, dec.decoder[$t.field])
    decode.success($t([t.field]:))
  }
}

Note the use of [t.field] in expressions. Also note that use must be type-checked before being desugared, as desugaring occurs after the template is type-checked, but I think the Gleam compiler already type-checks use before desugaring anyway, so this is likely not far outside the bounds of what the type-checker can already do.

[lpil]

So $(...) accepts some sort of Field type? What happens if a field is used in a way that generates code that does not type check? For example, if in x.$(y) the value x does not have the field that y represents?

What happens in the lens example if a field does not have any labels?

Are there other things that a[b] can be used to iterate over other than fields of a variant?

Hopefully, templates can be type-checked using mostly Gleam's existing facilities.

We can implement type checking, but the typing rules of any new feature have to be designed, and then the implementation of these rules has to be designed. Gleam has a value language and a type language, so for anything to be added we have to have a design for both aspects.

It's also readable, easy to run in one's head, and would have good error messages due to templates being checked before generation.

Could you expand on the type checking you have in mind? 🙏

Finally, it reduces the Gleam ecosystem's reliance on external codegen tools, which also introduce issues with staleness and refactoring and so on.

Typically template metaprogramming systems do worse than code generators when it comes to automated refactoring, etc, as templates are opaque to the tools. There's no way for them to know how to update the templates to get the desired outcome of the refactoring. How would this system avoid that problem?

You might be able to use it for boilerplate-free serialization and deserialization:

If we could do that then that could be very compelling!

Could you add detailed comments to each line of that code, to explain what each part does? 🙏