Skip to content

Discarding the CodeMap for Starlark source that's embedded into the binary #150

@linabutler

Description

@linabutler

Hi there! I'm embedding a Starlark interpreter in one of my projects, and exposing a handful of custom types to scripts. For some of those types, instead of implementing all their logic in Rust, it's actually easier to implement parts of the core logic in Starlark, and then have the Rust parts call into it.

To do that, I'm including the Starlark source directly in the Rust binary with include_str!(...), and then using OnceCell to lazily evaluate it once. Here's a concrete example:

/// Let's say that `expr` is an expression string to evaluate against the `input`.
/// This is an example of core logic that's much easier to write in Starlark than in Rust.
pub fn evaluate<'v>(
    input: Value<'v>,
    expr: &'v str,
    eval: &mut Evaluator<'v, '_, '_>,
) -> starlark::Result<Value<'v>> {
    static FUNCTION: OnceCell<OwnedFrozenValue> = OnceCell::new();
    let function = FUNCTION
        .get_or_try_init(|| {
            let ast = AstModule::parse(
                "<native>",
                // `evaluate.star` is a Starlark file with a public
                // `evaluate` function that implements the core logic.
                include_str!("./evaluate.star").to_owned(),
                &Dialect::Standard
            )?;
            let module = Module::new();
            {
                let mut eval = Evaluator::new(&module);
                eval.eval_module(ast, &Globals::standard())?;
                module.set_extra_value(module.get("evaluate").unwrap());
            }
            starlark::Result::Ok(module.freeze()?.owned_extra_value().unwrap())
        })?
        .owned_value(eval.frozen_heap());
    eval.eval_function(function, &[input, eval.heap().alloc_str(expr).to_value()], &[])
}

The methods on my custom types can then call evaluate as part of their implementations.

This approach works well, but I noticed that the module's frozen heap holds a full copy of the parsed source in its CodeMap. That's not necessary in this case, since (1) the Starlark source is in a &static str, and (2) the contents of evaluate.star are baked directly into the hosting Rust binary, so the location information isn't going to be super helpful or actionable if it shows up in stack traces.

I was wondering if Evaluator could have a method that evaluates an AstModule, but discards the CodeMap—maybe by replacing it with a CodeMap::empty_static, instead of using the one that AstModule::into_parts returns?—to support use cases like mine.

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions