Feature Request: VM/Workers

[CoryGH]

A great feature to have would be a VM/Worker Thread which allows for finer control over threads (e.g. start/stop plus pause/resume, rate throttling, etc) along with an event engine attached to a VM/Worker state object. This could be achieved with external libraries in some ways, but would be far more efficient inside the actual Deno threading engine, allowing for things like server-side AWS-styled Lambda processors without having to do something absurd like extract an AST from a piece of code, modify it to wrap blocks in code allowing for hooks, and then recompile to JavaScript before injection into a worker.

[kitsonk]

Deno already supports Web Workers. It can support data URLs for the workers, and there is a feature request for blob URLs (#9210). Outside of that, there isn't anything actionable here.

Respectfully closing.

[CoryGH]

Web workers are pretty limited and lack state load/save in conjunction with pause/resume type functionality.

[lucacasonato]

If you want to do something like this, you should embed Deno in your application and control the isolates in rust. You can use deno_core, and the deno op crates to embed deno in your application.

Workers are too slow. And you cannot overwrite the window object, its read only. So there is no way to run imported modules in a sandbox. I need this sandbox on every http request, with workers this is extremly slow. So having something like node vm would be important.

[CoryGH]

Workers are too slow. And you cannot overwrite the window object, its read only. So there is no way to run imported modules in a sandbox. I need this sandbox on every http request, with workers this is extremly slow. So having something like node vm would be important.

I tend to agree with this. If Deno is supposed to be a superior node.js, at the bare minimum it should be able to do everything node.js does, just with a more verbose featureset.

[lucacasonato]

@einicher The Node VM module is not a security sandbox. It should in no scenario be used for security! If you need a security sandbox you have to use workers (or worker_threads in Node).

@lucacasonato I am not talking about security. Sandbox in a case where you run code separate from the main stack. It does not have to be secure. Just not interfering with the main application. If i import something that writes to the window object, i need that window object (or self) not to be the main applications window object.

EDIT
I found out that imports inside the worker is what slows it down. Now i bundle everything first and run it without internal imports, gets me down from 500ms to 50ms. Wherever i try to use the modern esModule approach, i end up ditching it and bundling everything again. I think i am finally too old to change ;'(

[caspervonb]

Workers are too slow. And you cannot overwrite the window object, its read only. So there is no way to run imported modules in a sandbox. I need this sandbox on every http request, with workers this is extremly slow. So having something like node vm would be important.

Workers and isolates have roughly about the same overhead.
Experimented with this for the test runner with potentially running each test in isolation, benchmarked a LOT.
On average we saw about ~10-50ms per instantiation for std modules.

[lucacasonato]

@einicher Pass --no-check when starting Deno. This should significantly speed up the time it takes to instantiate workers. If you ware looking for isolation in an isolate, Realms are the solution to this. They are not yet ready however.

@caspervonb its not the worker itself that makes this approach slow, its the whole concept about it: in a worker everything needs to be reloaded from scratch. You cannot share active objects between main application and sandbox. In my case: on every http request the whole cheerio tree gets reimported and rerun (takes 500ms). in a node vm i do a global.jQuery = require('cheerio'); at application startup and pass the jQuery into every vm i create. Its pretty fast and runs in a separate environment where i get to choose what parts of the main application are accessible and which not. So workers are a good option for security, if you really need a sandbox that can only share text values with the main app, but if you want to reuse active parts of the main app, you are pretty f*****. Also its pretty medieval to have a response/request interface to interact with the main app. Do you understand me? There is something missing in the middle. Workers are too strict, regular imports are too open. It would be helpfull just to have a plain empty containerModule where you can decide which main app variables are accessible and have an own global environment (like the workers self) inside.

[lucacasonato]

@einicher What do you actually intend to achieve with this separate vm context? Ie. what is the problem you are trying to solve?

[caspervonb]

Do you understand me? There is something missing in the middle. Workers are too strict, regular imports are too open. It would be helpfull just to have a plain empty containerModule where you can decide which main app variables are accessible and have an own global environment (like the workers self) inside.

That in, the middle bit would be probably be realms:
https://github.com/tc39/proposal-realms
https://github.com/tc39/proposal-ses
https://github.com/tc39/proposal-compartments

Yes Realms seem to be a very advanced way to achieve this. VMs seem simpler to me, they just give you separate context.