Expose a faster way to create objects with properties for Node-API

[devongovett]

What is the problem this feature will solve?

At the moment, there are a few ways to create objects using the napi C APIs.

• napi_create_object, and then napi_set_named_property for each property.
• napi_create_object, and use napi_define_properties to set all properties at once (under the hood it still sets them one by one though).
• use napi_define_class + napi_new_instance, and then add properties using one of the above methods. You can define accessors on the prototype, or primitive values but instance properties must be set the same way as above.

In my use case I need to create a lot of small objects of the same shape. I've noticed in profiling that v8 functions under napi_set_named_property are slow, specifically v8::internal::JSObject::MigrateToMap. From my research in the v8 source code, this indicates that the object is transitioning from a fast struct/class-like object to a slower dictionary representation (I could be wrong here).

I think v8 exposes ObjectTemplate and FunctionTemplate->InstanceTemplate which are meant to help with this by defining the instance properties up front. This way, the object property slots can be allocated in the object itself rather than in a separate hash map. I found some useful info about v8 object representations here. I see that Node makes use of ObjectTemplate for internal objects, and a previous issue nodejs/node-addon-api#1074 also found this to be the fastest way to create objects (though still not as fast as doing it in JS).

My problem is that napi does not expose a way to create an object together with its properties, or a way to define a template for class instance properties. This means, as far as I can tell, all objects constructed through napi will end up in the slow dictionary mode, leading to slower perf when both constructing and accessing properties.

What is the feature you are proposing to solve the problem?

It would be awesome if node exposed a new napi_create_object_with_properties function, which would accept a list of property descriptors like napi_define_properties and allocate an object and assign properties all at once. This could potentially allow using faster v8 methods to create the object and assign properties so that it doesn't go into hashmap mode when adding properties one by one.

Alternatively, a way to define an instance property template for classes would also work for me. For example, napi_property_attributes could be extended with a napi_instance attribute for defining properties on the instance template rather than the prototype template.

What alternatives have you considered?

I'm not a node or v8 internal expert, just noticed that setting properties on objects seemed slower than expected, so I could be totally wrong about everything above. Opening this issue to get a conversation started. Totally open to other suggestions!

[bnoordhuis]

This means, as far as I can tell, all objects constructed through napi will end up in the slow dictionary mode

Dictionary mode happens when an object has too many properties. In that respect ObjectTemplate vs. adding properties one by one doesn't matter.

Instantiating an ObjectTemplate is marginally faster because you're not going through the JS engine's "add property to object" code path repeatedly but it won't be noticeable unless you create millions of objects or your objects have many properties.

As a practical concern: support for JS engines other than V8 is a design goal of n-api and I don't think other engines have a concept that maps cleanly to ObjecTemplate. Spidermonkey's JSClass for example isn't a 1-to-1 fit.

[devongovett]

Dictionary mode happens when an object has too many properties

Hmm, in my benchmark all the objects I created have exactly 2 properties. Did I misinterpret the profile (what MigrateToMap does)?

I don't think other engines have a concept that maps cleanly to ObjecTemplate

They must have some way of constructing an object with properties upfront through, similar to creating an object literal. And if not, the API could always fall back to constructing an object and assigning properties one by one internally.

[bnoordhuis]

what MigrateToMap does

"Map" in V8 nomenclature is what other JS engines call "object shape" or "hidden class", it's a piece of metadata that describes the object's layout.

It's used every time an object's layout changes (at least in C++ code), not just when switching from fast to slow mode.

They must have some way of constructing an object with properties upfront through, similar to creating an object literal.

Oh yeah, they probably do.

I was thinking of lifetime issues. A JSClass has a static lifetime whereas an ObjectTemplate only lives as long as its isolate. Other engines tie such magic objects to the lifetime of the current execution context (shorter still.)

It's probably not insurmountable but some upfront research is necessary. (I'm not volunteering!)

[devongovett]

I created a simple benchmark in this repo showing the difference between various methods. Each creates 100000 objects with 2 properties and appends them to an array.

napi: 48.992ms
v8 tmpl: 16.973ms
js: 2.663ms

Even with ObjectTemplate, JS object literals are still significantly faster.

[kjvalencik]

I'm also encountering this while working on a serde (serilization) implementation for Neon (Node-API bindings for Rust). I'm having difficulty getting my direct Rust<->JS implementation to outperform JSON (JSON.parse(serde_json::to_string(data))).

I setup a benchmark with criterion serializing the pokedex data set and profiled with pprof and JSON is 2x faster.

Most time is spent in napi_set_property (~62%) with that time spent pretty evenly in ObjectSet between Object::AddDataProperty and PropertyKey::PropertyKey.

Looking at the V8 JSON parsing code, it looks like a lot of work goes into optimizing the construction of a JSON object from a list of properties.

It would be great if these optimizations could be brought over to a Node-API method. Perhaps something like:

node_api_create_object_with_properties(
    napi_env env,
    size_t property_count,
    const napi_property_descriptor* properties,
    napi_value* result
)

[devongovett]

Sounds like such an API could be made faster in JSC as well according to @Jarred-Sumner of Bun:

https://twitter.com/jarredsumner/status/1605566356895801344?s=20&t=ZlICNZzNZotl6gtaOtBULA

[bnoordhuis]

It would be great if these optimizations could be brought over to a Node-API method.

Someone would need to add the corresponding API to V8 first.

[devongovett]

v8 does have a version of v8::Object::New which accepts a list of names and values: https://v8.github.io/api/head/classv8_1_1Object.html#afdb58efc2065a4e7a012db394c302159.

But I noticed in the implementation it calls NewSlowJSObjectWithPropertiesAndElements so I don't think it's optimized. I benchmarked that and it's slower than ObjectTemplate. But seems like maybe that existing API could be made faster?

[mhdawson]

We discussed a bit in the Node-api team meeting. @vmoroz is going to take a look and do some experimentation.

[vmoroz]

My hope was that we can improve perf for setting properties by adding an API to create internalized strings (NewStringType::kInternalized). V8 like other JS engines internalizes all string keys, and we can help it by caching them. We must also allow napi_ref to keep strings in a cache (#45715 or #42557). Unfortunately, effect from such change is almost not visible in this case because we have only two properties per object.

It would be great to add an API like node_api_create_object_with_properties. The only question is how to make it work fast. Using ObjectTemplate does not help - it only shows good results when we reuse it multiple times. Any single use of ObjectTemplate makes the perf worse. If we want to expose ObjectTemplate, then we will have a lot of other issues such as lifetime issues, incompatibility with other JS engines, complex use scenarios, etc.

So far it seems that Node-API or V8 API cannot create JS objects as fast as the JS parser does from object literals. It makes sense: the parser has the full information upfront and can optimize the creation unlike anything we can do through the API. Adding a special V8 API as for JSON object might address this issue. But frankly, it all feels a bit wrong: on one side we have C++ or Rust that can be extremely efficient with handling large amount of data, and on the other side we have JavaScript that was never meant to be that efficient, and still when we use C++ we lose. The reason we lose is because we make C++ to play by JavaScript rules. Since the interop has an overhead, we theoretically cannot achieve the same perf as in JavaScript. The only way to win is to avoid playing by JavaScript rules. It means avoiding creation a lot of small objects from C++ and instead explore other techniques.

My understanding is that these small objects are needed to represent some UI scene graphs. Games written in C++ often avoid creation of many small objects and instead use arrays for better cache locality. Thus, the proposal is to rethink the JS code that works with the native APIs in a way that it keeps data mostly in big arrays and is not "chatty".

• The data that is kept in native arrays can be either completely native or be exposed to JS code through TypedArrays or Buffers.
• The API must not be "chatty": any call to the native code must result in some "juicy action" - a non-trivial work done by native code to process the data structures. This way any interop overhead will be negligible comparing with the useful work.

I realize that I am not helping to address the issue as it was stated, but I am quite interested to find a work around to this type of problems. It would be great to create a Node-API example that shows these techniques.

[devongovett]

Thanks for the investigation! In my case, I'm building a compiler in Rust which allows custom JavaScript visitor plugins. This consists of JS functions that are called when a node in the AST is visited. For a given file, the functions may be called hundreds or thousands of times, with small objects as arguments. It would be pretty hard to avoid creating all these objects in this case I think. If you're curious, the docs have more info.

Do you think it would be worth filing a ticket with V8 asking to optimize v8::Object::New similar to the JSON parser? Seems to me that if they know all the properties and values up front, it could work similarly. Internally, they probably have a way to do this for implementing language builtins or maybe even things like DOM APIs?

[vmoroz]

It is a very cool project! And it is completely different scenario from the scene graph... TypedArray or "less chatty" API will not work.

Based on the spec and examples it seems that each argument type may have several properties.
I wonder if the following approach would work:

• Each argument type must have a predefined prototype object with accessor (get/set) properties.
• Each new argument instance will use the corresponding prototype object and wrap associated native object pointer. No properties must be set per argument instance. They will be all "inherited" from the prototype.
• The property accessor methods (native functions) will take this argument, unwrap the associated native object pointer, and then access the required data.

Hopefully, this way we can reduce amount of work done per each argument object and mostly reuse all the code as prototypes.
EDIT: it all can be done by napi_define_class.

As an alternative, it is possible to use Proxy object. This example simulates ObjectTemplate behavior by using Proxy. I would expect the prototype to work better than Proxy, but I did not measure it.

As for asking V8 team for the new API, it should be worth doing it. We must be able to wrap it in a Node-API function if they implement it.

[kjvalencik]

In my use case, key names are known upfront and interning strings (with String objects and napi_ref) did have a significant improvement. However, it was still 2-3x slower than JSON with most of the time spent in napi_set_property.

Since I'm writing a Rust serializer, in most cases, keys and even types are known and can be aggregated up front, similar to the optimizations in the V8 JSON parser.

Interestingly, these optimizations appear to only exist in the JSON parser. I also tried writing a v8.Serializer implementation in Rust to improve performance, but v8.deserialize was many times slower than JSON.parse.