feat: support Float16Array views from Node.js 24 - #12
Merged
Conversation
Most already have it, but not all. We use an isort required-imports entry instead of flake8-future-annotations lint check because the flake8-future-annotations rules only add the import when certain conditions are met; I prefer to always have the same semantics across modules.
h4l
force-pushed
the
nodejs-24-float16-buffer
branch
from
September 29, 2025 07:39
b13106a to
854a4af
Compare
quarto/quartodoc create it automatically.
It makes more sense to me to evaluate the view tag first and only check the feature if necessary.
Previously type checking rejected assignment of concrete view types,
like JSUint8Array to their base type JSTypedArray. e.g this was an
error:
view: JSTypedArray = JSUint8Array(b'')
This also caused problems inferring the type of generic function
arguments, e.g. HostObjectSerializerFn[JSDataView | JSTypedArray],
implemented by NodeJsArrayBufferViewHostObjectHandler. When calling the
serialize_host_object, passing a specific view, like JSUint8Array, mypy
would fail to infer the type, because JSUint8Array is not assignable to
JSTypedArray.
This was happening because the generic type vars of the JS buffer types
(like JSTypedArray[JSArrayBufferT, ViewTagT]) were invariant rather than
covariant. In practice, because all the buffer types are immutable,
these parameters can be covariant.
So now, the buffer types use covariant generic type params, and the
concrete subtypes are assignable to the base types, and inferring the
generic type of arguments marked as the base type works as expected
(e.g. the case mentioned above).
EncodeV8SerializeError and subtypes used by v8serialize.encode are now defined in v8serialize._errors so that other modules imported by v8serialize.encode can also import them.
HostObjectDeserializer and HostObjectSerializer are now defined in v8serialize._values instead of the encode/decode modules themselves. This allows the v8serialize.extensions module to import them, as it's also imported by the encode and decode modules.
The NodeJsArrayBufferViewHostObjectHandler now explicitly subclasses HostObjectSerializerObj and HostObjectDeserializerObj protocols. This isn't neccissary for correct type checking, but makes it clearer for humans what the intention is.
h4l
force-pushed
the
nodejs-24-float16-buffer
branch
from
September 29, 2025 08:21
854a4af to
f655f4f
Compare
We already supported Float16Array buffer views for vanilla V8 serialization, but Node.js uses its own custom HostObject encoding for buffer views, and Node.js added support in version 24, after we added the initial support. We now support decoding and encoding Float16Array from Node.js's custom HostObject encoding. Deno 2 also seems to be using this custom encoding.
h4l
force-pushed
the
nodejs-24-float16-buffer
branch
from
September 29, 2025 08:37
f655f4f to
acf5dd7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We already supported Float16Array buffer views for vanilla V8, but Node.js uses its own custom HostObject encoding for buffer views, and Node.js added support in version 24, after we added the initial support.
This PR adds support decoding and encoding Float16Array from Node.js's custom HostObject encoding. Deno 2 also seems to be using this custom encoding.