How to use as config language for a program?

[tennox]

I developed yet another CLI tool, at some point wanted some config - and thought: why not use nickel for that?

But I skimmed through docs and tutorials, and it seems I would need to always export manually to e.g. JSON and then parse that?

Am I misunderstanding the concept or the scope of nickel? Or is it just not implemented yet?

[suimong]

Iirc Nickel provides Python and JS binding. So if your CLI is written in either of the two languages (or Rust) then you can directly parse the nickel file into the native data structure of that language.

[tennox]

I don't find any docs/package for that (JS), you have any idea?

[yannham]

I don't think there's good documentation right now for linking to JS, but we compile Nickel to WASM for the Nickel online playground.

If you have/can use Nix, the simplest is to run nix build .#wasm-repl at the root of the repository - this might be long but should produce in ./result a standard NPM package wrapping the Nickel REPL, which is relatively straightforward to use from JS. Although this is originally made for the REPL, you can always initialize a session, send one command, and be done with it if all you want to do is to evaluate one configuration.

If you don't use Nix or are on a platform that isn't supported by Nix, you can look at what we do in the flake.nix file to build the Wasm package. There's a bunch of things there, but with a bit of effort you should be able to do the build manually, using wasm-pack.

[jneem]

Maybe we could be publishing the npm package as part of the release process?

[jneem]

To expand on the case of rust specifically, nickel's RichTerm type implements serde's Deserializer trait, so if you annotate a rust struct with #[derive(serde::Deserialize)] then you can convert from the evaluated nickel straight to your rust struct. Just bear in mind that serde's error messages are not very user-friendly, so to get error messages you'll probably want to apply a contract on the nickel side that's strict enough to ensure that the serde conversion never fails.

[yannham]

Good question. From the top of my head, my intuition would be that deriving a Nickel contract from a Rust type is maybe the simplest conceptually. I think something like:

1. Introduce a trait (could be put in nickel_lang_core::typ for example) for Rust types that can be reified as a Nickel type

   pub trait ReifyAsNickelType {
       fn reify_as_nickel_type() -> Type;
   }

2. Write a procedural macro that derives this implementation for an arbitrary Rust type, ideally, so that you would just write:

   #[derive(ReifyAsNickelType)]
   struct Config {
       ....
   }

The generated implementation should build a Nickel contract that mimics the Rust type: if it's a struct, make a Nickel record with the same fields, and to each field attach a contract that corresponds to its Rust type. So for example, for struct A { foo: Foo, bar: Bar }, you want to build a contract like {foo | <Foo::reify_as_nickel_type()>, bar | <Bar::reify_as_nickel_type()>}. You'll find the builder library in nickel_lang_core::term::make::builder to be helpful there.

If it's an enum A { Foo(Foo), Bar(Bar) }, turns that into a Nickel enum type [|'Foo <Foo::reify_as_nickel_type()>, 'Bar <Bar::reify_as_nickel_type()> |]. I'm not sure what to do with variants with many arguments, like Binary(Left, Right), as Nickel enum can only have one argument. One possibility is to reject such enums in the macro. Another is to encode the tuple (Left, Right) as a record type {"0": Left, "1": Right}. Variants with anonymous struct as arguments are unproblematic: Foo {bar: Bar, baz: Baz} in Rust can be encoded as the variant 'Foo {bar | <Bar::reify_as_nickel_type()>, baz |<Baz::reify_as_nickel_type()>}.

We also want to provide instance for Rust base types: String, &str, i32/u32/i64/u64/etc..., bool, etc.

I think it's not going to be entirely trivial, writing the macros, generating some Nickel structures and so on, but the path is rather clear. We would gladly accept and help with such a PR!

[zoechi]

Is there an exsmple somewhere how to deserialize config.ncl using serde?
What dependency to use for loading Nickel in Rust?

[yannham]

@zoechi : Sorry, we should put that somewhere for sure. Here is a short example of how to deserialize some random Rust struct from a Nickel file at path, and additionally assuming that inside the configuration, the desired structure is a sub-expression located at field_path, for example:

# config.ncl
{
  machine1 = {..}
  machine2 = {
    config = { <DATA OF INTEREST>},
    ..
  },
  ..
}

Then you could use field_path = "machine2.config". You can ignore the field path stuff if you just want to read an entire Nickel config file and use the top-level value. The dependencies you need are nickel-lang-core and serde.

use nickel_lang_core::{
  error::{Error, NullReporter, report::ErrorFormat};
  eval::cache::CacheImpl,
  program::Program as Prog,
};

use serde::Deserialize;

#[derive(Deserialize)]
struct MyConfig {
    foo: usize,
    bar: String,
    baz: Vec<String>,
}

pub fn read_config(path: &str, field_path: &str) -> Result<MyConfig, Error> {
    // The output for `std.trace` in Nickel
    let err = std::io::stderr();
    // NullReporter is for warnings. Here we just ignore them. You can use nickel_lang_core::error::Sink if you want to
    // collect them instead, or use a custom reporter (print them, send them to the network,
    // whatever).
    let mut prog = Prog::<CacheImpl>::new_from_file(path, err, NullReporter {})
        .expect("IO error while reading the file");
    let field = prog.parse_field_path(field_path.to_owned())?;
    prog.field = field;

    let result = prog.eval_full_for_export()?;
    Ok(MyConfig::deserialize(result).expect("couldn't deserialize the result to `Config`"))
}

You'll probably want better handling of IO or deserialization errors. You can report the Nickel errors using program.report() (to stderr with color and stuff) or program.report_as_str() if not for a terminal.

[zoechi]

@yannham Thanks a lot.
I have used serde and found some deserialization code in nickel_lang_core, so I probably would have figured it out.
I was wondering if I had the wrong idea about the project because there wasn't even a pointer anywhere to what I assumed would be the main use case for the project.

Adding above code to at least one of

• the readme
• -in an /examples subdirectory or
• Rust codedocs

would be very helpful

[yannham]

I agree! I think the main use-case for the project isn't necessary to use it as a Rust library; it's probably to use the Nickel CLI to generate JSON or YAML directly, and feed that to whatever tool you're using (Terraform, Kubernetes, etc.). However using it as a native language for a Rust project is rather well-supported, so it's definitely a shame that the documentation doesn't explain/promote this aspect more...