Need a java interop with lazy evaluation of methods

[sivachalabathid]

Hi,

In our org, we have a large repository of configurations which are just enormous configurations. They are encoded as java objects. But, we want to support yaml or json backends and lazy evaluation of methods because there are so many unused objects getting generated.

I want to lazily evaluate functions from java, but not sure how to do it from java. Is it possible at all?

[sivachalabathid]

Right now, each configuration class, creates a big tree object. It is composed so many methods and one root method. Use case is,

1. Converting the config to pkl
2. Using pkl to lazily evaluate the config and create objects at runtime lazily.
3. Support multiple data format backends for these config and use pkl and convert them java objects (lazily if possible).

[bioball]

Pkl's evaluator API provides ways to control exactly what gets evaluated, and to keep the rest un-evaluated.

For example, you can evaluate a specific expression:

var result = evaluator.evaluateExpression(myModuleSource, "some.expression")

This will only evaluate the dependencies of expression some.expression, and the rest stays un-evaluated.
If you do this, you should also make sure that you use the same evaluator (don't close it if you want to evaluate another expression). This way, you can benefit from Pkl cacheing results as it goes.

But: why is it important to you that evaluation stays lazy? Are you trying to guard against memory pressure issues?

[sivachalabathid]

Thanks @bioball for the reply.

Yes, I need lazy evaluation. Control over all method calls being lazy is important because, I am just expanding a single tree recursively.

Because, I am actually walking the tree. I have some config that creates millions of nodes. And, as I said most of them are unused. We are creating a tree and doing a graph traversal there (BFS with sub trees for specific fields completely expanded). So, I would only need nodes from next depth. I want expand specific nodes if needed for evaluating the object. If the node is valid, I would expand it further.

[sivachalabathid]

Even if it is possible to have a visitor with all the types like Methods, classes and other nodes, I would then make decisions about creating the objects out of them which is a lot.

I would actually be parsing the pkl config and making slightly different decisions for method calls alone. Also, I have to do the book keeping for methods and parameters. Is it possible? Is that the right thing to do here?