Packages Boilerplate

[abueide]

Hi, I started looking into organist and am pretty new. Here's my very simple project.ncl atm:

let inputs = import "./nickel.lock.ncl" in
let organist = inputs.organist in

{
  shells = organist.shells.Bash,

  shells.build = {
    packages = { 
      gradle = organist.import_nix "nixpkgs#gradle",
      jdk21 = organist.import_nix "nixpkgs#jdk21",
    },
  },

  shells.dev = {
    packages.hello = organist.import_nix "nixpkgs#hello",
  },
}
  | organist.OrganistExpression

I thought it was interesting for each package you need to define blah = organist.import_nix "nixpkgs#blah', for every single package which reduces readability and maintainability in projects with lots of dependencies to manage. Normally in nix we use with, so I looked a bit and found this article which lists some problems with how nix's with works: https://www.tweag.io/blog/2023-01-24-nix-with-with-nickel/

It was a good read, but it didn't really offer any insights on what we should do to eliminate some of this cruft. I think a good compromise would be something that looks like this instead:

...
{
  shells = organist.shells.Bash,

  shells.build = {
    packages = { 
      nixpkgs = {
         gradle, jdk21,
      },
     other_cachix_server = {
         random_lib, random_lib2,
      },
    },
  },
  ...
  }

then we can define multiple sources and separate the resolving mechanism from the dependency declarations. Then the organist.import_nix resolver can be applied to all pkgs associates with the "nixpkgs" resolver. This will reduce boilerplate and allow external caches. In particular in the future I plan to maintain a repo separate from nixpkgs which packages java dependencies using nickel/nix instead of gradle/maven that can be use with organist.

Let me know if this makes sense or if I'm missing something that already exists.

[yannham]

Hello!

There might be better solutions on the organist side, but just for the record, on the Nickel side, you might do the following to reduce boilerplate a bit:

let inputs = import "./nickel.lock.ncl" in
let organist = inputs.organist in
let take_from_nixpkgs = fun pkgs =>
  pkgs
  |> std.array.map (fun pkg => { field = pkg, value = organist.import_nix "nixpkgs#%{pkg}"})
  |> std.record.from_array
in

{
  shells = organist.shells.Bash,

  shells.build = {
    packages = [
      "gradle",
      "jdk21",
      # etc..
    ]
    |> take_from_nixpkgs,
  },

  shells.dev = {
    packages.hello = organist.import_nix "nixpkgs#hello",
  },
}
  | organist.OrganistExpression

You could even abuse the Nickel syntax a bit, which allows fields without definition, to alternatively specify the packages as record - as in your example - instead of strings in an array:

# [...]
let take_from_nixpkgs = fun pkgs =>
  pkgs
  |> std.record.fields
  |> std.array.map (fun pkg => {field = pkgs, value = organist.import_nix "nixpkgs#%{pkg}"})
  |> std.record.from_array
in

# [...]

shells.build = {
    packages = { gradle, jdk }
    |> take_from_nixpkgs,
  },

# [...]

In a general setting, I wouldn't necessarily advise for such a pattern, because it turns something static (an explicit record definition) to something dynamic, computed at run-time, which could in particular mess with the LSP or make some errors be reported later in the pipeline. However, in the case of organist and Nix imports, I believe we already do something dynamic - passing a string to import_nix - and currently don't get any kind of completion or static check that the package actually exist, so I suppose it doesn't change much in this regard.

Of course you might want to put this function in a separate file and import it wherever needed.