Constant Relative Imports

[rohan-datar]

This probably is a relatively niche issue, so I understand if what I'm trying to do here isn't possible in pkl, but I thought I'd ask the question anyway.

I'm trying to build a relatively complex template, and as a result some template files rely on information contained in other ones. Here's a minimal example:

In package template

// template/foo.pkl

name: String
value: Int
config: Listing<String>

and another template that depends on foo.pkl

// template/bar.pkl

foo: Module
config: Listing

output {
   rendererer = new JsonRenderer {
       converters {
         [Listing] = // function that depends on foo.config
       }
    }
}

Then I have a package Configurator that depends on template.

Configurator
├── configuration1
│   ├── foo.pkl
│   ├── bar.pkl
├── configuration2
│   ├── foo.pkl
│   ├── bar.pkl
...

where the templates are filled out like this:

// configuration1/bar.pkl

amends "@template/bar.pkl"

foo = import('./foo.pkl')
config {
   // some values here
}

This works fine, but I am in a situation where the could be hundreds of different configurations and putting foo = import("./foo.pkl") inside every single bar.pkl is very inconvenient.

Is there a way I can create a relative import (or something else that allows me to access the contents of foo.pkl) in the parent template? I can enforce the relative path of the configuration files within each instance of the template, but it is easy for a user to forget the import expression, which breaks the template in unexpected ways because foo is then instantiated as an empty module.

Is there a better solution for this?

[bioball]

I think you are looking for sibling imports?

See this post here: #902 (reply in thread)

[bioball]

Hi, apologies, missed this.

What do you see in the trace? The only thing that error message is telling me is that it can't find a key in the mapping, not that it's empty.

Also, one thing that seems off about this is that the glob import's keys are the expanded glob pattern. In this case, it will be the relative path between the root module and the target module.

Typically, you'd have a structure like this:

| - some/nested/path/bar.pkl // <- amends "..."
| - some/nested/path/foo.pkl
bar.pkl 

Then you can then compute the relative path from some/nested/path/bar.pkl to bar.pkl:

import "bar.pkl" // self import
import* "**/foo.pkl" as fooModules

local relativePath = bar.relativePathTo(module)

mySiblingFoo = fooModules["\(relativePath.join("/"))/foo.pkl"]

[rohan-datar]

Here's the full trace:

❯ pkl eval config/bar.pkl
pkl: TRACE: import*("**.pkl") = new Mapping { ["bar.pkl"] { config {} }; ["foo.pkl"] { name = ?; value = ?; config {} } } (file:///home/rdatar/pkl-test/template/bar.pkl, line 7)
–– Pkl Error ––
Cannot find key `"file:///home/rdatar/pkl-test/config/foo.pkl"`.

11 | hidden fixed foo: Foo = fooModules["\(path)foo.pkl"]
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at bar#foo (file:///home/rdatar/pkl-test/template/bar.pkl, line 11)

17 | [Listing] = (x) -> "\(foo.value)-\(x)"
                           ^^^
at bar#output.renderer.converters[#1].<function#1> (file:///home/rdatar/pkl-test/template/bar.pkl, line 17)

106 | text = renderer.renderDocument(value)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at pkl.base#Module.output.text (https://github.com/apple/pkl/blob/0.27.2/stdlib/base.pkl#L106)

I tried the relative path method that you suggested at first (see my first post in this thread), but it lead to a "No descendant path error that I posted above"

[bioball]

Given this trace:

pkl: TRACE: import*("**.pkl") = new Mapping { ["bar.pkl"] { config {} }; ["foo.pkl"] { name = ?; value = ?; config {} } }

I can see that the glob is matching values.

Some changes:

1. Make sure you are defining the glob import logic within template/bar.pkl
2. Move everything within templates/ to the root level, and adjust your imports accordingly

[rohan-datar]

Moving the contents of template into root worked! The one thing I'm not sure about here is in my actual use case template is a dependency declared in a PklProject file. Will it still work?

[rohan-datar]

Yeah it fails when it's an outside dependency. I tried to set up local project dependencies like the example in the documentation, and I get this error:

❯ pkl project resolve
/home/rdatar/pkl-test/config/PklProject.deps.json
❯ pkl eval bar.pkl
–– Pkl Error ––
No descendent path exists between modules `projectpackage://example.com/pkl-test/[email protected]#/bar.pkl` and
`file:///home/rdatar/pkl-test/config/bar.pkl`.

9 | local path = bar.relativePathTo(module).join("/")
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
at bar#path (projectpackage://example.com/pkl-test/[email protected]#/bar.pkl)

11 | hidden fixed foo: Foo = fooModules["\(path)/foo.pkl"]
                                           ^^^^
at bar#foo (projectpackage://example.com/pkl-test/[email protected]#/bar.pkl)

17 | [Listing] = (x) -> "\(foo.value)-\(x)"
                           ^^^
at bar#output.renderer.converters[#1].<function#1> (projectpackage://example.com/pkl-test/[email protected]#/bar.pkl)

106 | text = renderer.renderDocument(value)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
at pkl.base#Module.output.text (https://github.com/apple/pkl/blob/0.27.2/stdlib/base.pkl#L106)