Skip to content

[help wanted] deps: update V8 to 13.6 #57753

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 39 commits into
base: main
Choose a base branch
from
Open

[help wanted] deps: update V8 to 13.6 #57753

wants to merge 39 commits into from

Conversation

targos
Copy link
Member

@targos targos commented Apr 5, 2025

I hope it can replace #57114

Notable changes:

@nodejs/v8-update @nodejs/tsc

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/gyp
  • @nodejs/security-wg
  • @nodejs/tsc
  • @nodejs/v8-update

@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. dependencies Pull requests that update a dependency file. meta Issues and PRs related to the general management of the project. needs-ci PRs that need a full CI run. v8 engine Issues and PRs related to the V8 dependency. labels Apr 5, 2025
@targos
Copy link
Member Author

targos commented Apr 5, 2025

@joyeecheung (or maybe someone else from @nodejs/cpp-reviewers) Can you help finalizing 05eab64 ?
I ported all the referenced PRs from #52718 but there is an issue with node_mksnapshot:

[1903/1919] ACTION node: node_mksnapshot_9b7a2d2290b02e76d66661df74749f56
FAILED: gen/node_snapshot.cc
cd ../../; export BUILT_FRAMEWORKS_DIR=/Users/mzasso/git/nodejs/v8-next-update/out/Debug; export BUILT_PRODUCTS_DIR=/Users/mzasso/git/nodejs/v8-next-update/out/Debug; export CONFIGURATION=Debug; export EXECUTABLE_NAME=node; export EXECUTABLE_PATH=node; export FULL_PRODUCT_NAME=node; export PRODUCT_NAME=node; export PRODUCT_TYPE=com.apple.product-type.tool; export SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk; export SRCROOT=/Users/mzasso/git/nodejs/v8-next-update/out/Debug/../../; export SOURCE_ROOT="${SRCROOT}"; export TARGET_BUILD_DIR=/Users/mzasso/git/nodejs/v8-next-update/out/Debug; export TEMP_DIR="${TMPDIR}"; export XCODE_VERSION_ACTUAL=1630;/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot /Users/mzasso/git/nodejs/v8-next-update/out/Debug/gen/node_snapshot.cc

  #  /Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot[39108]: IsolatePlatformDelegate *node::NodePlatform::ForIsolate(Isolate *) at ../../src/node_platform.cc:530
  #  Assertion failed: (data.first) != nullptr

----- Native stack trace -----

 1: 0x104f36d9c node::DumpNativeBacktrace(__sFILE*) [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
 2: 0x1050a0e5c node::Assert(node::AssertionInfo const&) [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
 3: 0x10519b9a4 node::NodePlatform::ForIsolate(v8::Isolate*) [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
 4: 0x10519bcec node::NodePlatform::GetForegroundTaskRunner(v8::Isolate*, v8::TaskPriority) [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
 5: 0x106ffd978 cppgc::internal::Sweeper::SweeperImpl::Start(cppgc::internal::SweepingConfig) [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
 6: 0x106feda98 cppgc::internal::HeapBase::Terminate() [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
 7: 0x1060c52d0 v8::internal::CppHeap::~CppHeap() [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
 8: 0x1060c5368 non-virtual thunk to v8::internal::CppHeap::~CppHeap() [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
 9: 0x10613ba18 v8::internal::Heap::TearDown() [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
10: 0x1060382d8 v8::internal::Isolate::Deinit() [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
11: 0x106037dbc v8::internal::Isolate::Delete(v8::internal::Isolate*) [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
12: 0x105325a74 node::RAIIIsolateWithoutEntering::~RAIIIsolateWithoutEntering() [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
13: 0x105325aa8 node::RAIIIsolateWithoutEntering::~RAIIIsolateWithoutEntering() [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
14: 0x105325b60 node::RAIIIsolate::~RAIIIsolate() [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
15: 0x105325b8c node::RAIIIsolate::~RAIIIsolate() [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
16: 0x1051fdf20 node::BuildCodeCacheFromSnapshot(node::SnapshotData*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&) [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
17: 0x1051fe2c4 node::SnapshotBuilder::Generate(node::SnapshotData*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::optional<std::__1::basic_string_view<char, std::__1::char_traits<char>>>, node::SnapshotConfig const&) [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
18: 0x1051fedac node::SnapshotBuilder::GenerateAsSource(char const*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, node::SnapshotConfig const&, bool) [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
19: 0x104e2c4bc BuildSnapshot(int, char**) [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
20: 0x104e2c220 main [/Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot]
21: 0x1943c6b4c start [/usr/lib/dyld]
/bin/sh: line 1: 39108 Abort trap: 6           /Users/mzasso/git/nodejs/v8-next-update/out/Debug/node_mksnapshot /Users/mzasso/git/nodejs/v8-next-update/out/Debug/gen/node_snapshot.cc

@targos targos marked this pull request as draft April 5, 2025 09:11
@targos targos added the semver-major PRs that contain breaking changes and should be released in the next major version. label Apr 5, 2025
@targos targos force-pushed the v8-136 branch 2 times, most recently from e1ed931 to 7774826 Compare April 6, 2025 11:15
@targos targos changed the title deps: update V8 to 13.6 [help wanted] deps: update V8 to 13.6 Apr 6, 2025
@targos targos added the help wanted Issues that need assistance from volunteers or PRs that need help to proceed. label Apr 6, 2025
@danmcd
Copy link

danmcd commented Apr 6, 2025

I will run a build on illumos/SmartOS. If you fixed the JSDispatchEntry assumptions about high-16-bits in kFreeEntryTag that'll help out a lot.

@danmcd
Copy link

danmcd commented Apr 7, 2025

I will run a build on illumos/SmartOS. If you fixed the JSDispatchEntry assumptions about high-16-bits in kFreeEntryTag that'll help out a lot.

I used some brute-force to clear the high-16-bits in the userland address space so v8 can do its thing. It stopped failing where it used to and started failing somewhere else, but still in node_mksnapshot.

In non-debug/stock:

  LD_LIBRARY_PATH=/home/danmcd/targos-v8-136/out/Release/lib.host:/home/danmcd/targos-v8-136/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../.; mkdir -p /home/danmcd/targos-v8-136/out/Release/obj/gen; "/home/danmcd/targos-v8-136/out/Release/node_mksnapshot" "/home/danmcd/targos-v8-136/out/Release/obj/gen/node_snapshot.cc"

  #  [20437]: node::IsolatePlatformDelegate* node::NodePlatform::ForIsolate(v8::Isolate*) at ../src/node_platform.cc:530
  #  Assertion failed: (data.first) != nullptr

In debug (using --v8-non-optimized-debug --v8-with-dchecks --v8-enable-object-print) I get the same error.

I noticed on #57114 that @bnoordhuis had a suggestion too that might ease the requirement for me to do something drastic (at least in the short term?).

If v8 documents "pointer requirements" somewhere, that'd help me a lot. I'm visiting the v8-dev google group now, so they may give me the clues I need.

EDIT/UPDATE ==> I didn't look carefully before at @targos 's update ==> I'm encountering the same issue.

@targos
Copy link
Member Author

targos commented Apr 9, 2025

#57753 (comment) basically requires a revert of #30909.
I will push it so we can run CI but if the race condition is still possible I don't know what to do.

@targos
Copy link
Member Author

targos commented Apr 9, 2025

/cc @addaleax

@targos
Copy link
Member Author

targos commented Apr 9, 2025

Now it's failing embedtest:

$ out/Debug/embedtest -- 'eval((require("fs").readFileSync("test/fixtures/snapshot/echo-args.js", "utf8")))' arg1 arg2 --embedder-snapshot-blob test/.tmp.0/embedder-snapshot.blob --embedder-snapshot-create
(node:77036) Warning: It's not yet fully verified whether built-in module "vm" works in user snapshot builder scripts.
It may still work in some cases, but in other cases certain run-time states may be out-of-sync after snapshot deserialization.
To request support for the module, use the Node.js issue tracker: https://github.com/nodejs/node/issues
(Use `embedtest --trace-warnings ...` to show where the warning was created)
[1]    77036 segmentation fault  out/Debug/embedtest --  arg1 arg2 --embedder-snapshot-blob
(node:76421) Warning: It's not yet fully verified whether built-in module "vm" works in user snapshot builder scripts.
It may still work in some cases, but in other cases certain run-time states may be out-of-sync after snapshot deserialization.
To request support for the module, use the Node.js issue tracker: https://github.com/nodejs/node/issues
(Use `embedtest --trace-warnings ...` to show where the warning was created)
Process 76421 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xa8)
    frame #0: 0x000000010283dfa8 embedtest`cppgc::internal::ObjectAllocator::in_disallow_gc_scope(this=0x00000000000000a8) const at object-allocator.cc:371:10 [opt]
   368 	}
   369
   370 	bool ObjectAllocator::in_disallow_gc_scope() const {
-> 371 	  return raw_heap_.heap()->IsGCForbidden();
   372 	}
   373
   374 	#ifdef V8_ENABLE_ALLOCATION_TIMEOUT
Target 0: (embedtest) stopped.
warning: embedtest was compiled with optimization - stepping may behave oddly; variables may not be available.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xa8)
  * frame #0: 0x000000010283dfa8 embedtest`cppgc::internal::ObjectAllocator::in_disallow_gc_scope(this=0x00000000000000a8) const at object-allocator.cc:371:10 [opt]
    frame #1: 0x000000010282a25c embedtest`cppgc::internal::ObjectAllocator::AllocateObject(this=0x00000000000000a8, size=40, gcinfo=1) at object-allocator.h:113:3 [opt]
    frame #2: 0x00000001008c5cf4 embedtest`cppgc::internal::MakeGarbageCollectedTraitInternal::AllocationDispatcher<node::contextify::ContextifyScript, void, 8ul>::Invoke(handle=0x00000000000000a8, size=40) at allocation.h:93:14
    frame #3: 0x00000001008c5c8c embedtest`cppgc::MakeGarbageCollectedTraitBase<node::contextify::ContextifyScript>::Allocate(handle=0x00000000000000a8, size=40) at allocation.h:178:12
    frame #4: 0x00000001008c5c14 embedtest`node::contextify::ContextifyScript* cppgc::MakeGarbageCollectedTrait<node::contextify::ContextifyScript>::Call<node::Environment*&, v8::Local<v8::Object>&>(handle=0x00000000000000a8, args=0x000000016fdfc730, args=0x000000016fdfc738) at allocation.h:239:9
    frame #5: 0x00000001008bcd2c embedtest`node::contextify::ContextifyScript* cppgc::MakeGarbageCollected<node::contextify::ContextifyScript, node::Environment*&, v8::Local<v8::Object>&>(handle=0x00000000000000a8, args=0x000000016fdfc730, args=0x000000016fdfc738) at allocation.h:278:7
    frame #6: 0x00000001008bccf8 embedtest`node::contextify::ContextifyScript::New(env=0x000000015280c400, object=Local<v8::Object> @ 0x000000016fdfc738) at node_contextify.cc:978:10
    frame #7: 0x00000001008bbf70 embedtest`node::contextify::ContextifyScript::New(args=0x000000016fdfd018) at node_contextify.cc:1032:41
    frame #8: 0x000000010161ecd8 embedtest`v8::internal::FunctionCallbackArguments::CallOrConstruct(this=0x000000016fdfd088, function=<unavailable>, is_construct=<unavailable>) at api-arguments-inl.h:93:3 [opt]
    frame #9: 0x000000010161dfe8 embedtest`v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<true>(isolate=0x0000000158008000, new_target=DirectHandle<v8::internal::HeapObject> @ x23, fun_data=DirectHandle<v8::internal::FunctionTemplateInfo> @ x20, receiver=<unavailable>, argv=0x000000016fdfd1f8, argc=8) at builtins-api.cc:104:16 [opt]
    frame #10: 0x000000010161d260 embedtest`v8::internal::Builtin_HandleApiConstruct(int, unsigned long*, v8::internal::Isolate*) [inlined] v8::internal::Builtin_Impl_HandleApiConstruct(args=<unavailable>, isolate=0x0000000158008000) at builtins-api.cc:135:3 [opt]
    frame #11: 0x000000010161d1d0 embedtest`v8::internal::Builtin_HandleApiConstruct(args_length=<unavailable>, args_object=<unavailable>, isolate=0x0000000158008000) at builtins-api.cc:126:1 [opt]
    frame #12: 0x000000010010fe94 embedtest`Builtins_CEntry_Return1_ArgvOnStack_BuiltinExit + 84
    frame #13: 0x00000001000604f0 embedtest`Builtins_InterpreterPushArgsThenFastConstructFunction + 752
    frame #14: 0x000000010024f424 embedtest`Builtins_ConstructHandler + 1284
    frame #15: 0x000000010005fb8c embedtest`Builtins_InterpreterEntryTrampoline + 268
    frame #16: 0x00000001000603a0 embedtest`Builtins_InterpreterPushArgsThenFastConstructFunction + 416
    frame #17: 0x000000010024f424 embedtest`Builtins_ConstructHandler + 1284
    frame #18: 0x000000010005fb8c embedtest`Builtins_InterpreterEntryTrampoline + 268
    frame #19: 0x000000010005fb8c embedtest`Builtins_InterpreterEntryTrampoline + 268
    frame #20: 0x000000010005fb8c embedtest`Builtins_InterpreterEntryTrampoline + 268
    frame #21: 0x000000010005fb8c embedtest`Builtins_InterpreterEntryTrampoline + 268
    frame #22: 0x000000010005d14c embedtest`Builtins_JSEntryTrampoline + 172
    frame #23: 0x000000010005cdf0 embedtest`Builtins_JSEntry + 176
    frame #24: 0x0000000101854900 embedtest`v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [inlined] v8::internal::GeneratedCode<unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, long, unsigned long**>::Call(this=<unavailable>, args=<unavailable>, args=65939493879825, args=27756946545385, args=65939493879873, args=<unavailable>, args=0x000000016fdfdae0) at simulator.h:212:12 [opt]
    frame #25: 0x00000001018548ec embedtest`v8::internal::(anonymous namespace)::Invoke(isolate=0x0000000158008000, params=0x000000016fdfd920) at execution.cc:440:22 [opt]
    frame #26: 0x0000000101853b20 embedtest`v8::internal::Execution::Call(isolate=0x0000000158008000, callable=DirectHandle<v8::internal::Object> @ x23, receiver=DirectHandle<v8::internal::Object> @ x22, args=<unavailable>) at execution.cc:530:10 [opt]
    frame #27: 0x000000010157fd74 embedtest`v8::Function::Call(this=0x000000015284c840, isolate=0x0000000158008000, context=<unavailable>, recv=Local<v8::Value> @ x21, argc=1, argv=0x000000016fdfdae0) at api.cc:5445:7 [opt]
    frame #28: 0x00000001006cd3d0 embedtest`node::LoadEnvironment(node::Environment*, std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::function<void (node::Environment*, v8::Local<v8::Value>, v8::Local<v8::Value>)>)::$_0::operator()(this=0x000000016fdfdfd0, info=0x000000016fdfde88) const at environment.cc:556:30
    frame #34: 0x000000010083fa28 embedtest`std::__1::function<v8::MaybeLocal<v8::Value> (node::StartExecutionCallbackInfo const&)>::operator()(this=0x000000016fdfdfc8, __arg=0x000000016fdfde88) const at function.h:989:10
    frame #35: 0x000000010083f2d8 embedtest`node::StartExecution(env=0x000000015280c400, cb=node::StartExecutionCallback @ 0x000000016fdfdfc8) at node.cc:323:30
    frame #36: 0x00000001006c1c38 embedtest`node::LoadEnvironment(env=0x000000015280c400, cb=node::StartExecutionCallback @ 0x000000016fdfe0a8, preload=node::EmbedderPreloadCallback @ 0x000000016fdfe088) at environment.cc:542:10
    frame #37: 0x00000001006c1df8 embedtest`node::LoadEnvironment(env=0x000000015280c400, main_script_source_utf8="const assert = require('assert');assert(require('v8').startupSnapshot.isBuildingSnapshot());globalThis.embedVars = { nön_ascıı: '🏳️‍🌈' };globalThis.require = require;require('vm').runInThisContext(process.argv[2]);", preload=node::EmbedderPreloadCallback @ 0x000000016fdfe6d8) at environment.cc:551:10
    frame #38: 0x000000010066d008 embedtest`RunNodeInstance(platform=0x0000600001c401e0, args=size=7, exec_args=size=0) at embedtest.cc:189:21
    frame #39: 0x000000010066c42c embedtest`main(argc=8, raw_argv=0x000000016fdfeee0) at embedtest.cc:60:7
    frame #40: 0x00000001943c6b4c dyld`start + 600

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Apr 9, 2025

@targos
Copy link
Member Author

targos commented Apr 9, 2025

@nodejs/platform-ppc @nodejs/platform-s390 V8 gn build fails: https://ci.nodejs.org/job/node-test-commit-v8-linux/6492/nodes=rhel8-ppc64le,v8test=v8test/console

@miladfarca
Copy link
Contributor

@targos try updating gn and build the latest.

@targos
Copy link
Member Author

targos commented Apr 9, 2025

@richardlau are you able to do that?

@richardlau
Copy link
Member

@richardlau are you able to do that?

Yes, I'll look at it after the TSC meeting.

@targos
Copy link
Member Author

targos commented Apr 9, 2025

These are the two commits that are probably wrong and I need help to fix:

  • src: adapt to V8's CppHeap API changes: 718fdd2
  • src: dispose isolate before unregistering it : 84d01ac

@danmcd
Copy link

danmcd commented Apr 9, 2025

illumos (on SmartOS) update:

I can build this branch on the hacked-userlimit illumos system. I've yet to try it on the stock-userlimit (0xffff... limit) system. My test failures on the hacked-userlimit seem to match yours.

@richardlau
Copy link
Member

richardlau commented Apr 9, 2025

@richardlau are you able to do that?

Yes, I'll look at it after the TSC meeting.

Updated in nodejs/build#4066.

https://ci.nodejs.org/job/node-test-commit-v8-linux/6493/ ✅ is with the updated gn on 84d01ac (same commit as previously failing job).

@nodejs-github-bot
Copy link
Collaborator

@joyeecheung
Copy link
Member

I suspect that now the use of DisallowJavascriptExecution requires the isolate to be alive, so it cannot be used in a path that's disposing the isolate. I'd suggest that we just remove that DisallowJavascriptExecution scope cc @legendecas

@joyeecheung
Copy link
Member

I checked the implementation of DisallowJavascriptExecutionScope and it relies on checking a field in the Isolate, so it should not be used during Isolate disposal because otherwise on scope closing it's going to attempt to access a field from the freed isolate. Though we can keep it but only for the IsolateData destruction now

diff --git a/src/node_main_instance.cc b/src/node_main_instance.cc
index edbc3c0dc96..2d20b8d3a99 100644
--- a/src/node_main_instance.cc
+++ b/src/node_main_instance.cc
@@ -81,10 +81,10 @@ NodeMainInstance::~NodeMainInstance() {
     // This should only be done on a main instance that owns its isolate.
     // IsolateData must be freed before UnregisterIsolate() is called.
     isolate_data_.reset();
-    isolate_->Dispose();
-    platform_->UnregisterIsolate(isolate_);
-    // TODO(joyeecheung): split Isolate::Free() here?
   }
+  // TODO(joyeecheung): split Isolate::Free() here?
+  isolate_->Dispose();
+  platform_->UnregisterIsolate(isolate_);
 }
 
 ExitCode NodeMainInstance::Run() {

As V8 is moving towards built-in CppHeap creation, change the
management so that the automatic CppHeap creation on Node.js's end
is also enforced at Isolate creation time.

1. If embedder uses NewIsolate(), either they use
  IsolateSettings::cpp_heap to specify a CppHeap that will be owned
  by V8, or if it's not configured, Node.js will create a CppHeap
  that will be owned by V8.
2. If the embedder uses SetIsolateUpForNode(),
  IsolateSettings::cpp_heap will be ignored (as V8 has deprecated
  attaching CppHeap post-isolate-creation). The embedders need to
  ensure that the v8::Isolate has a CppHeap attached while it's
  still used by Node.js, preferably using v8::CreateParams.

See https://issues.chromium.org/issues/42203693 for details. In
future version of V8, this CppHeap will be created by V8 if not
provided, and we can remove our own "if no CppHeap provided,
create one" code in NewIsolate().
@nodejs-github-bot
Copy link
Collaborator

@targos
Copy link
Member Author

targos commented Apr 16, 2025

There's a consistent test failure on macOS that happens only in GHA:

=== release test-error-serdes ===
Path: sequential/test-error-serdes
Error: --- stderr ---
node:internal/streams/readable:90
const FastBuffer = Buffer[SymbolSpecies];
                         ^

RangeError: Maximum call stack size exceeded
    at node:internal/streams/readable:90:26
    at BuiltinModule.compileForInternalLoader (node:internal/bootstrap/realm:398:7)
    at requireBuiltin (node:internal/bootstrap/realm:429:14)
    at node:internal/streams/duplex:39:18
    at BuiltinModule.compileForInternalLoader (node:internal/bootstrap/realm:398:7)
    at requireBuiltin (node:internal/bootstrap/realm:429:14)
    at node:internal/streams/pipeline:16:16
    at BuiltinModule.compileForInternalLoader (node:internal/bootstrap/realm:398:7)
    at requireBuiltin (node:internal/bootstrap/realm:429:14)
    at node:internal/streams/compose:7:22

Node.js v24.0.0-pre
Command: out/Release/node --expose-internals --stack-size=64 --test-reporter=./test/common/test-error-reporter.js --test-reporter-destination=stdout /Users/runner/work/node/node/node/test/sequential/test-error-serdes.js

@legendecas
Copy link
Member

legendecas commented Apr 16, 2025

@joyeecheung I suspect that now the use of DisallowJavascriptExecution requires the isolate to be alive, so it cannot be used in a path that's disposing the isolate. I'd suggest that we just remove that DisallowJavascriptExecution scope
I checked the implementation of DisallowJavascriptExecutionScope and it relies on checking a field in the Isolate, so it should not be used during Isolate disposal

The original idea was to prevent JS execution in draining platform tasks when shutting down: 5db35b4, via platform_->UnregisterIsolate.

The DisallowJavascriptExecution in ~NodeMainInstance is closed before isolate_->Dispose():

{
#ifdef DEBUG
// node::Environment has been disposed and no JavaScript Execution is
// allowed at this point.
// Create a scope to check that no JavaScript is executed in debug build
// and proactively crash the process in the case JavaScript is being
// executed.
// Isolate::Dispose() must be invoked outside of this scope to avoid
// use-after-free.
Isolate::DisallowJavascriptExecutionScope disallow_js(
isolate_, Isolate::DisallowJavascriptExecutionScope::CRASH_ON_FAILURE);
#endif
// This should only be done on a main instance that owns its isolate.
// IsolateData must be freed before UnregisterIsolate() is called.
isolate_data_.reset();
platform_->UnregisterIsolate(isolate_);
}
isolate_->Dispose();
.

From the stack trace in #57753 (comment):

#1  v8::internal::DisallowJavascriptExecution::Close (isolate=0xffffa5600000, was_execution_allowed=true) at ../deps/v8/src/common/assert-scope.cc:98
#2  0x0000aaaac961e8d8 in node::NodeMainInstance::~NodeMainInstance (this=0xffffdbdba958, __in_chrg=<optimized out>)
    at ../src/node_main_instance.cc:87

The line number looks strange to me because DisallowJavascriptExecution::Close should not be on line src/node_main_instance.cc:87, but instead it should be on line 85 when the scope goes out of block.

@danmcd
Copy link

danmcd commented Apr 17, 2025

Great news from illumos-land!

With these two V8 patches, the first from IBM for generic grow-from-bottom-only that exceeds 47 bits:

https://chromium-review.googlesource.com/c/v8/v8/+/6320599/5
danmcd@f0a2fc9

and an illumos addition to the end, because we grow from both the TOP of VA space, and the BOTTOM (IOW we use both sides of the 4-level Virtual Address 48-bit space, not just the lower-47) and need to account for that:

danmcd@ca01a9c

I'm now getting identical results between a stock illumos with these patches, and an altered illumos without these patches but with all 64-bit user processes being limited to 47-bit Virtual Address space.

ADDITIONALLY, not for illumos yet, but it IS available in Oracle Solaris, there's a link-time trick to reserve address space away from anonymous mmap(), and only available to explicit mmap(). We are considering this in illumos, especially when the time comes for 5-level paging with 57-bit Virtual Address space.

Those identical results are still:

--- TIMEOUT ---


[05:55|% 100|+ 4381|-   1]: Done                                              

Failed tests:
out/Release/node --test-reporter=./test/common/test-error-reporter.js --test-reporter-destination=stdout /home/danmcd/targos-v8-136/test/parallel/test-child-process-stdio-reuse-readable-stdio.js

@targos
Copy link
Member Author

targos commented Apr 17, 2025

On the failure in cross-compiler-rhel9-armv7-gcc-12-glibc-2.28 - it looks like the flags/toolchain for cross compilation are incorrect, it's hitting #ifdef __SSE2__ first and not defined(__ARM_NEON__).

@joyeecheung It happens while compiling for the host (we see obj.host in the command), which is Intel arch. We pass -m32 to the compiler, though. So I don't know why __SSE2__ is defined.

Edit: full compiler command is 17:13:59 ccache g++ -m32 -o /home/iojs/build/workspace/node-cross-compile/out/Release/obj.host/v8_base_without_compiler/gen/torque-generated/src/objects/arguments-tq.o /home/iojs/build/workspace/node-cross-compile/out/Release/obj/gen/torque-generated/src/objects/arguments-tq.cc '-D_GLIBCXX_USE_CXX11_ABI=1' '-DNODE_OPENSSL_CONF_NAME=nodejs_conf' '-DICU_NO_USER_DATA_OVERRIDE' '-DV8_GYP_BUILD' '-DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=64' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' '-DV8_TARGET_ARCH_ARM' '-DCAN_USE_ARMV7_INSTRUCTIONS' '-DCAN_USE_VFP3_INSTRUCTIONS' '-DCAN_USE_VFP32DREGS' '-DV8_HAVE_TARGET_OS' '-DV8_TARGET_OS_LINUX' '-DV8_EMBEDDER_STRING="-node.10"' '-DENABLE_DISASSEMBLER' '-DV8_PROMISE_INTERNAL_FIELD_COUNT=1' '-DV8_ENABLE_PRIVATE_MAPPING_FORK_OPTIMIZATION' '-DOBJECT_PRINT' '-DV8_INTL_SUPPORT' '-DV8_ATOMIC_OBJECT_FIELD_WRITES' '-DV8_ENABLE_LAZY_SOURCE_POSITIONS' '-DV8_USE_SIPHASH' '-DNDEBUG' '-DV8_WIN64_UNWINDING_INFO' '-DV8_ENABLE_REGEXP_INTERPRETER_THREADED_DISPATCH' '-DV8_USE_ZLIB' '-DV8_ENABLE_LEAPTIERING' '-DV8_ENABLE_SPARKPLUG' '-DV8_ENABLE_MAGLEV' '-DV8_ENABLE_TURBOFAN' '-DV8_ENABLE_WEBASSEMBLY' '-DV8_ENABLE_JAVASCRIPT_PROMISE_HOOKS' '-DV8_ENABLE_CONTINUATION_PRESERVED_EMBEDDER_DATA' '-DV8_ALLOCATION_FOLDING' '-DV8_ALLOCATION_SITE_TRACKING' '-DV8_ADVANCED_BIGINT_ALGORITHMS' '-DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_STATIC' '-DUCONFIG_NO_SERVICE=1' '-DU_ENABLE_DYLOAD=0' '-DU_STATIC_IMPLEMENTATION=1' '-DU_HAVE_STD_STRING=1' '-DUCONFIG_NO_BREAK_ITERATION=0' '-DHWY_BROKEN_EMU128=0' '-DUSE_EABI_HARDFLOAT=1' -I../deps/v8 -I../deps/v8/include -I/home/iojs/build/workspace/node-cross-compile/out/Release/obj/gen/inspector-generated-output-root -I../deps/v8/third_party/inspector_protocol -I/home/iojs/build/workspace/node-cross-compile/out/Release/obj/gen -I/home/iojs/build/workspace/node-cross-compile/out/Release/obj/gen/inspector-generated-output-root/include -I/home/iojs/build/workspace/node-cross-compile/out/Release/obj/gen/generate-bytecode-output-root -I../deps/icu-small/source/common -I../deps/icu-small/source/i18n -I../deps/icu-small/source/tools/toolutil -I../deps/v8/third_party/zlib -I../deps/v8/third_party/zlib/google -I../deps/v8/third_party/fp16/src/include -I../deps/v8/third_party/highway/src -I../deps/v8/third_party/simdutf -I../deps/v8/third_party/abseil-cpp -pthread -Wno-unused-parameter -Wno-strict-overflow -Wno-return-type -Wno-int-in-bool-context -Wno-deprecated -Wno-stringop-overflow -Wno-stringop-overread -Wno-restrict -Wno-array-bounds -Wno-nonnull -Wno-dangling-pointer -flax-vector-conversions -m32 -m32 -O3 -fno-omit-frame-pointer -fdata-sections -ffunction-sections -O3 -fno-rtti -fno-exceptions -fno-strict-aliasing -std=gnu++20 -Wno-invalid-offsetof -MMD -MF /home/iojs/build/workspace/node-cross-compile/out/Release/.deps//home/iojs/build/workspace/node-cross-compile/out/Release/obj.host/v8_base_without_compiler/gen/torque-generated/src/objects/arguments-tq.o.d.raw -c

@nodejs-github-bot
Copy link
Collaborator

@targos
Copy link
Member Author

targos commented Apr 17, 2025

There's another issue with macos: https://ci.nodejs.org/job/node-test-commit-osx/64642/nodes=osx13-x64/testReport/junit/(root)/parallel/test_worker_memory/


    #  out/Release/node[75970]: virtual node::PerIsolatePlatformData::~PerIsolatePlatformData() at ../src/node_platform.cc:295
    #  Assertion failed: !flush_tasks_

  ----- Native stack trace -----

   1: 0x10427fd86 node::Assert(node::AssertionInfo const&) [/Users/admin/build/workspace/node-test-commit-osx/nodes/osx13-x64/out/Release/node]
   2: 0x1065dcdd0 node::PerIsolatePlatformData::~PerIsolatePlatformData() (.cold.1) [/Users/admin/build/workspace/node-test-commit-osx/nodes/osx13-x64/out/Release/node]
   3: 0x10430edda node::PerIsolatePlatformData::~PerIsolatePlatformData() [/Users/admin/build/workspace/node-test-commit-osx/nodes/osx13-x64/out/Release/node]
   4: 0x10430f680 node::NodePlatform::RegisterIsolate(v8::Isolate*, uv_loop_s*) [/Users/admin/build/workspace/node-test-commit-osx/nodes/osx13-x64/out/Release/node]
   5: 0x10416bd8c node::NewIsolate(v8::Isolate::CreateParams*, uv_loop_s*, node::MultiIsolatePlatform*, node::SnapshotData const*, node::IsolateSettings const&) [/Users/admin/build/workspace/node-test-commit-osx/nodes/osx13-x64/out/Release/node]
   6: 0x1043acad9 node::worker::WorkerThreadData::WorkerThreadData(node::worker::Worker*) [/Users/admin/build/workspace/node-test-commit-osx/nodes/osx13-x64/out/Release/node]
   7: 0x1043a86a2 node::worker::Worker::Run() [/Users/admin/build/workspace/node-test-commit-osx/nodes/osx13-x64/out/Release/node]
   8: 0x1043ad2a4 node::worker::Worker::StartThread(v8::FunctionCallbackInfo<v8::Value> const&)::$_0::__invoke(void*) [/Users/admin/build/workspace/node-test-commit-osx/nodes/osx13-x64/out/Release/node]
   9: 0x7ff8034ab259 _pthread_start [/usr/lib/system/libsystem_pthread.dylib]
  10: 0x7ff8034a6c7b thread_start [/usr/lib/system/libsystem_pthread.dylib]

@targos
Copy link
Member Author

targos commented Apr 17, 2025

a213815 compiled successfully. I'll try to open a V8 CL

Edit: https://chromium-review.googlesource.com/c/v8/v8/+/6469692

@targos
Copy link
Member Author

targos commented Apr 17, 2025

We can see on https://ci.nodejs.org/job/node-test-binary-armv7l/16772/ that the brotli issue is still here.

@anonrig
Copy link
Member

anonrig commented Apr 17, 2025

I see that the following test became flaky: tools/test.py test/parallel/test-fs-stat-bigint.js --repeat=100

=== release test-fs-stat-bigint ===
Path: parallel/test-fs-stat-bigint
node:internal/assert/utils:281
    throw err;
    ^

AssertionError [ERR_ASSERTION]: Number version atimeMs = 1744906266381.947, BigInt version atimeMs = 1744906266380n, Allowable delta = 1
    at verifyStats (/home/yagiz/coding/node/test/parallel/test-fs-stat-bigint.js:70:7)
    at runSyncTest (/home/yagiz/coding/node/test/parallel/test-fs-stat-bigint.js:102:3)
    at Object.<anonymous> (/home/yagiz/coding/node/test/parallel/test-fs-stat-bigint.js:120:3)
    at Module._compile (node:internal/modules/cjs/loader:1734:14)
    at Object..js (node:internal/modules/cjs/loader:1899:10)
    at Module.load (node:internal/modules/cjs/loader:1469:32)
    at Module._load (node:internal/modules/cjs/loader:1286:12)
    at TracingChannel.traceSync (node:diagnostics_channel:322:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:235:24)
    at Module.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:152:5) {
  generatedMessage: false,
  code: 'ERR_ASSERTION',
  actual: false,
  expected: true,
  operator: '=='
}

@danmcd
Copy link

danmcd commented Apr 17, 2025

@targos

I'm curious about how to proceed with the VA48/full-4-level fixes for illumos? I could probably shrink it down to illumos-only, but that might not help AIX with BIG VA spaces, never mind any other 64-bit architectures that might grow into the top-16-bits of VA space that might kneecap V8. (There STILL may be some V8 abuses of the top-16 I've missed, but the ones I have fixes for are immediately helpful to your overarching import-V8 changes here.)

Also, if/when illumos brings in something akin to Oracle Solaris's mapfile (i.e. /usr/lib/ld/map.va47) that clamps down VA ranges per-executable, we can revisit any unsightly changes IF there's a good reason to make assumptions about VA space.

@targos
Copy link
Member Author

targos commented Apr 18, 2025

@danmcd If https://chromium-review.googlesource.com/c/v8/v8/+/6320599 is accepted by the V8 team, the best would be to submit a follow-up patch to them for illumos

@targos
Copy link
Member Author

targos commented Apr 18, 2025

In the mean time, would it be possible to apply the workaround you found to our CI hosts? How can we do it?

@nodejs-github-bot
Copy link
Collaborator

@targos
Copy link
Member Author

targos commented Apr 18, 2025

For this armv7l error: https://ci.nodejs.org/job/node-test-commit-arm/58141/nodes=ubuntu2204-armv7l/

Reproduction on the CI host is:

iojs@5480e1997a99:~/build/workspace/node-test-commit-arm/tools/doc$ /home/iojs/build/workspace/node-test-commit-arm/out/Release/node /home/iojs/build/workspace/node-test-commit-arm/deps/npm/bin/npm-cli.js ci
npm warn cli npm v11.3.0 does not support Node.js v24.0.0-pre. This version of npm supports the following node versions: `^20.17.0 || >=22.9.0`. You can find the latest version at https://nodejs.org/.
-

#
# Fatal error in , line 0
# Check failed: v8_flags.separate_gc_phases && young_gc_while_full_gc_ implies current_.state == Event::State::SWEEPING.
#
#
#
#FailureMessage Object: 0xffab167c
----- Native stack trace -----

@targos
Copy link
Member Author

targos commented Apr 18, 2025

@targos
Copy link
Member Author

targos commented Apr 18, 2025

This can be "fixed" by reverting https://chromium-review.googlesource.com/c/v8/v8/+/6297948. Should we do that and come back to it for the 13.7 update? Note that the flag no longer exists in V8 13.7 so we won't be able to disable it anymore.

@targos
Copy link
Member Author

targos commented Apr 18, 2025

/cc @omerktz FYI.

@targos
Copy link
Member Author

targos commented Apr 18, 2025

#57753 (comment)

I can reproduce locally, but it's rare (2% failures)

By looking at the code in https://github.com/nodejs/node/blob/main/src/node_platform.cc, it seems possible to have PerIsolatePlatformData being destructed in workers before calling the Shutdown method.

@targos
Copy link
Member Author

targos commented Apr 18, 2025

@joyeecheung is it related to this?

      // TODO(joyeecheung): we reversed the order because the task runner has
      // to be available to handle GC tasks posted during isolate disposal.
      // If this flakes comes back again, try splitting Isolate::Delete out
      // so that the pointer is still available as map key after disposal
      // and we delete it after unregistration.
      // Refs: https://github.com/nodejs/node/pull/53086#issuecomment-2128056793
      isolate->Dispose();
      w_->platform_->UnregisterIsolate(isolate);

@danmcd
Copy link

danmcd commented Apr 18, 2025

In the mean time, would it be possible to apply the workaround you found to our CI hosts? How can we do it?

I'll see if it's as simple as scribbling a value into the kernel tunable. I'll ping here when I've done it so a build can spin. You'll have to use the MNX-hosted agents, however. Unfortunately, this isn't something you can just tell Node users to Just Apply.

I don't know how much pressure we can put on V8. There's also (with illumos) a fact that if you're illumos is past April, 2022, you don't need the caddr_t version of madvise(3C) as well. I also hope that maybe we can spin less cumbersome and more modern build agent environments once we get madvise() fixes into V8 that trickle into node.

@danmcd
Copy link

danmcd commented Apr 18, 2025

In the mean time, would it be possible to apply the workaround you found to our CI hosts? How can we do it?

I'll see if it's as simple as scribbling a value into the kernel tunable. I'll ping here when I've done it so a build can spin. You'll have to use the MNX-hosted agents, however. Unfortunately, this isn't something you can just tell Node users to Just Apply.

I'd have to revisit the whole of how the current MNX agents work. That work was done by Brie Bennett, who is no longer associated with Triton, SmartOS, or illumos.

I don't know how much pressure we can put on V8. There's also (with illumos) a fact that if you're illumos is past April, 2022, you don't need the caddr_t version of madvise(3C) as well. I also hope that maybe we can spin less cumbersome and more modern build agent environments once we get madvise() fixes into V8 that trickle into node.

I've followed up on the IBM-contributed Gerrit CR. We'll see if they pay attention or not.

I'm happy to supply something for Node directly that can be backed-out once V8 gets their act together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues and PRs related to build files or the CI. dependencies Pull requests that update a dependency file. help wanted Issues that need assistance from volunteers or PRs that need help to proceed. meta Issues and PRs related to the general management of the project. needs-ci PRs that need a full CI run. semver-major PRs that contain breaking changes and should be released in the next major version. v8 engine Issues and PRs related to the V8 dependency.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants