feat: project evaluation scoped tool

[zoedsoupe]

No description provided.

[zoedsoupe]

for now its failing since we're not setting up the IPC communication channel properly

we could try to communicate directly via STDIO but i think that's kinda tacky since node.js already handles IPC natively

[zoedsoupe]

@josevalim arguments is a special value that refers to an array-like object that contains the values of all the arguments passed to that function when it was called, so to receive external arguments i needed to rename it to args

i also took the option to define it as a record, because it seems kinda strange the need of args[0] usage to access them when a named record is easier DX (AI-X in this case?) wdyt?

[zoedsoupe]

hm some node specific version errors...

[josevalim]

Do we want a separate execution environment? I'd say we actually don't want a separate environment, as the goal of project_eval is to run within the exact same instance that the web server is running.

arguments is a special value that refers to an array-like object that contains the values of all the arguments passed to that function when it was called, so to receive external arguments i needed to rename it to args

Couldn't we still use arguments somehow? Otherwise args is indeed better.

i also took the option to define it as a record, because it seems kinda strange the need of args[0] usage to access them when a named record is easier DX (AI-X in this case?) wdyt?

In any case, it should be an array, for consistency with the other implementations. It is important for the MCPs to be as consistent as possible.

[jonatanklosko]

Couldn't we still use arguments somehow? Otherwise args is indeed better.

We can have arguments as the MCP parameter, and args as the way to access it in the code. Hopefully with good description the LLM won't get confused? We could change to args everywhere but that's a breaking change for TC (we could send both, but that's not beautiful :p)

[josevalim]

I am just wondering if we can do a trick like this:

const args = [1, 2, 3];
const code = "arguments[0] + arguments[1] + arguments[2]";
const result = eval(`(function() { return ${code}; })`).apply(null, args);

or:

const code = "return arguments[0] + arguments[1] + arguments[2]";
const fn = new Function(code);
const result = fn(1, 2, 3); // 6

Both seem to work on the repl.

[jonatanklosko]

Oh, you are saying the actual arguments in that case have no use, so we may as well use those. Sounds good to me!

Btw. we want new Function over eval, since eval sees all variables in the scope, which we don't want.

[zoedsoupe]

We can have arguments as the MCP parameter

actually, neither as MCP parameter we can have since the input schema is derived from the type, so it complains that modules are automatically on strict mode ;-;

we need to go with args unfortunately

nevermind, we can rename the argument name internally, so for the MCP we have arguments and in our code args, but in the evaluated code it will be need to be args too

[zoedsoupe]

Do we want a separate execution environment? I'd say we actually don't want a separate environment, as the goal of project_eval is to run within the exact same instance that the web server is running.

i think i didn't understand completely the concern. the evaluated code have the same scope of the current project (aka the web server too), so it can access all local modules and dependencies, although restricted with dynamic imports since we're evaluating as a function.

my main point here is security, it does have access to the project scope, although restricted. what is the real concern here? cc @josevalim

[zoedsoupe]

const code = "return arguments[0] + arguments[1] + arguments[2]";
const fn = new Function(code);
const result = fn(1, 2, 3); // 6

i think we can expand the arguments array and pass it as separate ones to the function, would be that you're thinking on?

[josevalim]

One nit, double check CI, and we are good!