Remove coordinator and support forking #1
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ruby/library/csv/**/*.rb
The order of iseq may differ from the order of tokens, typically `while`/`until` conditions are put after the body. These orders can match by using line numbers as builtin-indexes, but at the same time, it introduces the restriction that multiple `cexpr!` and `cstmt!` cannot appear in the same line. Another possible idea is to use `RubyVM::AbstractSyntaxTree` and `node_id` instead of ripper, with making BASERUBY 3.1 or later.
…onstant is defined
Support dropping extra arguments passed by `yield` in blocks. For
example `10.times { work }` drops the count argument. This is common
enough that it's about 3% of fallback reasons in `lobsters`.
Only support simple cases where the surplus arguments are at the top of
the stack, that way they just need to be popped, which takes no work.
`[BUG] pthread_mutex_lock: Invalid argument (EINVAL)` doesn't seem like a fault of YJIT? https://github.com/ruby/ruby/actions/runs/7614455181/job/20736754975 https://github.com/ruby/ruby/actions/runs/7615316673/job/20739572487
Bumps [rb-sys](https://github.com/oxidize-rb/rb-sys) from 0.9.86 to 0.9.87. - [Release notes](https://github.com/oxidize-rb/rb-sys/releases) - [Commits](oxidize-rb/rb-sys@v0.9.86...v0.9.87) --- updated-dependencies: - dependency-name: rb-sys dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> rubygems/rubygems@d86482f0b0
When we're compiling begin / rescue / ensure nodes, we need to "wrap" the code in the begin statements correctly. The wrapping is like this: (ensure code (rescue code (begin code))) This patch pulls the each leg in to its own function, then calls the appropriate wrapping function depending on whether there are ensure / rescue legs. Fixes: ruby/prism#2221
Co-Authored-By: Kevin Newton <[email protected]>
Small PR to add a comment when we clear local variable types, so we can be aware that it's happening when looking at the disasm.
Suppose YJIT runs a rb_vm_opt_send_without_block() fallback and the control frame stack looks like: ``` will_tailcall_bar [FINISH] caller_that_used_fallback ``` will_tailcall_bar() runs in the interpreter and sets up a tailcall. Right before JIT_EXEC() in the `send` instruction, the stack will look like: ``` bar [FINISH] caller_that_used_fallback ``` Previously, JIT_EXEC() ran bar() in JIT code, which caused the `FINISH` flag to return to the interpreter instead of to the JIT code running caller_that_used_fallback(), causing code to run twice and probably crash. Recent flaky failures on CI about "each stub expects a particular iseq" are probably due to leaving methods twice in `test_optimizations.rb`. Only run JIT code from the interpreter if a new frame is pushed.
If the first element of an interpolated string node is an embedded statement, CRuby "pre-initializes" the interpolation with a string of known encoding to concat into. This patch replicates thate behaviour in Prism
…y#9744) * YJIT: print warning when disasm options used without a dev build I was confused for a few minutes the other way then --yjit-dump-disasm printed nothing, so I figured this would be useful for end-users (and future me). * Fix lone extraneous space
Sometimes this file get picked up and break Ripper tests:
TestRipper::Generic#test_parse_files:test/ruby
assert_separately failed with error message
pid 63392 exit 0
| test_regexp.rb:2025: warning: character class has duplicated range
https://github.com/ruby/ruby/actions/runs/7699956651/job/20982702553#step:12:103
Thanks to Kokubun for noticing. Follow-up: b0711b1
|
Oops. Wrong repository. |
wks
pushed a commit
to wks/ruby
that referenced
this pull request
Oct 24, 2024
During compilation, we write keyword default values into the iseq, so we should mark it to ensure it does not get GC'd. This might fix issues on ASAN like http://ci.rvm.jp/logfiles/brlog.trunk_asan.20240927-194923 ==805257==ERROR: AddressSanitizer: use-after-poison on address 0x7b7e5e3e2828 at pc 0x5e09ac4822f8 bp 0x7ffde56b0140 sp 0x7ffde56b0138 READ of size 8 at 0x7b7e5e3e2828 thread T0 #0 0x5e09ac4822f7 in RB_BUILTIN_TYPE include/ruby/internal/value_type.h:191:30 angussidney#1 0x5e09ac4822f7 in rbimpl_RB_TYPE_P_fastpath include/ruby/internal/value_type.h:352:19 ruby#2 0x5e09ac4822f7 in gc_mark gc/default.c:4488:9 ruby#3 0x5e09ac51011e in rb_iseq_mark_and_move iseq.c:361:17 ruby#4 0x5e09ac4b85c4 in rb_imemo_mark_and_move imemo.c:386:9 ruby#5 0x5e09ac467544 in rb_gc_mark_children gc.c:2508:9 ruby#6 0x5e09ac482c24 in gc_mark_children gc/default.c:4673:5 ruby#7 0x5e09ac482c24 in gc_mark_stacked_objects gc/default.c:4694:9 ruby#8 0x5e09ac482c24 in gc_mark_stacked_objects_all gc/default.c:4732:12 ruby#9 0x5e09ac48c7f9 in gc_marks_rest gc/default.c:5755:9 ruby#10 0x5e09ac48c7f9 in gc_marks gc/default.c:5870:9 ruby#11 0x5e09ac48c7f9 in gc_start gc/default.c:6517:13
wks
pushed a commit
to wks/ruby
that referenced
this pull request
Jul 1, 2025
…uby#13231) This change addresses the following ASAN error: ``` ==36597==ERROR: AddressSanitizer: heap-use-after-free on address 0x512000396ba8 at pc 0x7fcad5cbad9f bp 0x7fff19739af0 sp 0x7fff19739ae8 WRITE of size 8 at 0x512000396ba8 thread T0 [643/756] 36600=optparse/test_summary #0 0x7fcad5cbad9e in free_fast_fallback_getaddrinfo_entry /home/runner/work/ruby-dev-builder/ruby-dev-builder/ext/socket/raddrinfo.c:3046:22 angussidney#1 0x7fcad5c9fb48 in fast_fallback_inetsock_cleanup /home/runner/work/ruby-dev-builder/ruby-dev-builder/ext/socket/ipsocket.c:1179:17 ruby#2 0x7fcadf3b611a in rb_ensure /home/runner/work/ruby-dev-builder/ruby-dev-builder/eval.c:1081:5 ruby#3 0x7fcad5c9b44b in rsock_init_inetsock /home/runner/work/ruby-dev-builder/ruby-dev-builder/ext/socket/ipsocket.c:1289:20 ruby#4 0x7fcad5ca22b8 in tcp_init /home/runner/work/ruby-dev-builder/ruby-dev-builder/ext/socket/tcpsocket.c:76:12 ruby#5 0x7fcadf83ba70 in vm_call0_cfunc_with_frame /home/runner/work/ruby-dev-builder/ruby-dev-builder/./vm_eval.c:164:15 ... ``` A `struct fast_fallback_getaddrinfo_shared` is shared between the main thread and two child threads. This struct contains an array of `fast_fallback_getaddrinfo_entry`. `fast_fallback_getaddrinfo_entry` and `fast_fallback_getaddrinfo_shared` were freed separately, and if `fast_fallback_getaddrinfo_shared` was freed first and then an attempt was made to free a `fast_fallback_getaddrinfo_entry`, a `heap-use-after-free` could occur. This change avoids that possibility by separating the deallocation of the addrinfo memory held by `fast_fallback_getaddrinfo_entry` from the access and lifecycle of the `fast_fallback_getaddrinfo_entry` itself.
wks
pushed a commit
to wks/ruby
that referenced
this pull request
Jul 1, 2025
[Bug #18119]
When we create classes, it pushes the class to the subclass list of the
superclass. This access needs to be synchronized because multiple Ractors
may be creating classes with the same superclass, which would cause race
conditions and cause the linked list to be corrupted.
For example, we can reproduce with this script crashing:
workers = (0...8).map do
Ractor.new do
loop do
100.times.map { Class.new }
Ractor.yield nil
end
end
end
100.times { Ractor.select(*workers) }
With ASAN enabled, we can see that there are use-after-free errors:
==176013==ERROR: AddressSanitizer: heap-use-after-free on address 0x5030000974f0 at pc 0x62f9e56f892d bp 0x7a503f1ffd90 sp 0x7a503f1ffd88
WRITE of size 8 at 0x5030000974f0 thread T4
#0 0x62f9e56f892c in rb_class_remove_from_super_subclasses class.c:149:24
angussidney#1 0x62f9e58c9dd2 in rb_gc_obj_free gc.c:1262:9
ruby#2 0x62f9e58f6e19 in gc_sweep_plane gc/default/default.c:3450:21
ruby#3 0x62f9e58f686a in gc_sweep_page gc/default/default.c:3535:13
ruby#4 0x62f9e58f12b4 in gc_sweep_step gc/default/default.c:3810:9
ruby#5 0x62f9e58ed2a7 in gc_sweep gc/default/default.c:4058:13
ruby#6 0x62f9e58fac93 in gc_start gc/default/default.c:6402:13
ruby#7 0x62f9e58e8b69 in heap_prepare gc/default/default.c:2032:13
ruby#8 0x62f9e58e8b69 in heap_next_free_page gc/default/default.c:2255:9
ruby#9 0x62f9e58e8b69 in newobj_cache_miss gc/default/default.c:2362:38
...
0x5030000974f0 is located 16 bytes inside of 24-byte region [0x5030000974e0,0x5030000974f8)
freed by thread T4 here:
#0 0x62f9e562f28a in free (miniruby+0x1fd28a) (BuildId: 5ad6d9e7cec8318df6726ea5ce34d3c76d0d0233)
angussidney#1 0x62f9e58ca2ab in rb_gc_impl_free gc/default/default.c:8102:9
ruby#2 0x62f9e58ca2ab in ruby_sized_xfree gc.c:5029:13
ruby#3 0x62f9e58ca2ab in ruby_xfree gc.c:5040:5
ruby#4 0x62f9e56f88e6 in rb_class_remove_from_super_subclasses class.c:152:9
ruby#5 0x62f9e58c9dd2 in rb_gc_obj_free gc.c:1262:9
ruby#6 0x62f9e58f6e19 in gc_sweep_plane gc/default/default.c:3450:21
ruby#7 0x62f9e58f686a in gc_sweep_page gc/default/default.c:3535:13
ruby#8 0x62f9e58f12b4 in gc_sweep_step gc/default/default.c:3810:9
ruby#9 0x62f9e58ed2a7 in gc_sweep gc/default/default.c:4058:13
...
previously allocated by thread T5 here:
#0 0x62f9e562f70d in calloc (miniruby+0x1fd70d) (BuildId: 5ad6d9e7cec8318df6726ea5ce34d3c76d0d0233)
angussidney#1 0x62f9e58c8e1a in calloc1 gc/default/default.c:1472:12
ruby#2 0x62f9e58c8e1a in rb_gc_impl_calloc gc/default/default.c:8138:5
ruby#3 0x62f9e58c8e1a in ruby_xcalloc_body gc.c:4964:12
ruby#4 0x62f9e58c8e1a in ruby_xcalloc gc.c:4958:34
ruby#5 0x62f9e56f906e in push_subclass_entry_to_list class.c:88:13
ruby#6 0x62f9e56f906e in rb_class_subclass_add class.c:111:38
ruby#7 0x62f9e56f906e in RCLASS_SET_SUPER internal/class.h:257:9
ruby#8 0x62f9e56fca7a in make_metaclass class.c:786:5
ruby#9 0x62f9e59db982 in rb_class_initialize object.c:2101:5
wks
pushed a commit
to wks/ruby
that referenced
this pull request
Jul 1, 2025
In commit d42b9ff, an optimization was introduced that can speed up Regexp#match by 15% when it matches with strings of different encodings. This optimization, however, does not work across ractors. To fix this, we only use the optimization if no ractors have been started. In the future, we could use atomics for the reference counting if we find it's needed and if it's more performant. The backtrace of the misbehaving native thread: ``` * frame #0: 0x0000000189c94388 libsystem_kernel.dylib`__pthread_kill + 8 frame angussidney#1: 0x0000000189ccd88c libsystem_pthread.dylib`pthread_kill + 296 frame ruby#2: 0x0000000189bd6c60 libsystem_c.dylib`abort + 124 frame ruby#3: 0x0000000189adb174 libsystem_malloc.dylib`malloc_vreport + 892 frame ruby#4: 0x0000000189adec90 libsystem_malloc.dylib`malloc_report + 64 frame ruby#5: 0x0000000189ae321c libsystem_malloc.dylib`___BUG_IN_CLIENT_OF_LIBMALLOC_POINTER_BEING_FREED_WAS_NOT_ALLOCATED + 32 frame ruby#6: 0x00000001001c3be4 ruby`onig_free_body(reg=0x000000012d84b660) at regcomp.c:5663:5 frame ruby#7: 0x00000001001ba828 ruby`rb_reg_prepare_re(re=4748462304, str=4748451168) at re.c:1680:13 frame ruby#8: 0x00000001001bac58 ruby`rb_reg_onig_match(re=4748462304, str=4748451168, match=(ruby`reg_onig_search [inlined] rbimpl_RB_TYPE_P_fastpath at value_type.h:349:14 ruby`reg_onig_search [inlined] rbimpl_rstring_getmem at rstring.h:391:5 ruby`reg_onig_search at re.c:1781:5), args=0x000000013824b168, regs=0x000000013824b150) at re.c:1708:20 frame ruby#9: 0x00000001001baefc ruby`rb_reg_search_set_match(re=4748462304, str=4748451168, pos=<unavailable>, reverse=0, set_backref_str=1, set_match=0x0000000000000000) at re.c:1809:27 frame ruby#10: 0x00000001001bae80 ruby`rb_reg_search0(re=<unavailable>, str=<unavailable>, pos=<unavailable>, reverse=<unavailable>, set_backref_str=<unavailable>, match=<unavailable>) at re.c:1861:12 [artificial] frame ruby#11: 0x0000000100230b90 ruby`rb_pat_search0(pat=<unavailable>, str=<unavailable>, pos=<unavailable>, set_backref_str=<unavailable>, match=<unavailable>) at string.c:6619:16 [artificial] frame ruby#12: 0x00000001002287f4 ruby`rb_str_sub_bang [inlined] rb_pat_search(pat=4748462304, str=4748451168, pos=0, set_backref_str=1) at string.c:6626:12 frame ruby#13: 0x00000001002287dc ruby`rb_str_sub_bang(argc=1, argv=0x00000001381280d0, str=4748451168) at string.c:6668:11 frame ruby#14: 0x000000010022826c ruby`rb_str_sub ``` You can reproduce this by running: ``` RUBY_TESTOPTS="--name=/test_str_capitalize/" make test-all TESTS=test/ruby/test_m17n.comb ``` However, you need to run it with multiple ractors at once. Co-authored-by: jhawthorn <[email protected]>
wks
pushed a commit
to wks/ruby
that referenced
this pull request
Jul 1, 2025
`name` is used via `RSTRING_PTR` within rb_str_catf, which may allocate
and thus potentially trigger GC. Although `name` is still referenced
by a local variable, the compiler might optimize away the reference
before the GC sees it, especially under aggressive optimization or when
debugging tools like ASAN are used.
This patch adds an explicit `RB_GC_GUARD` to ensure `name` is kept alive
until after the last use.
While it's not certain this is the root cause of the following observed
use-after-poison ASAN error, I believe this fix is indeed needed and
hopefully a likely candidate for preventing the error.
```
==1960369==ERROR: AddressSanitizer: use-after-poison on address 0x7ec6a00f1d88 at pc 0x5fb5bcafcf2e bp 0x7ffcc1178cb0 sp 0x7ffcc1178470
READ of size 61 at 0x7ec6a00f1d88 thread T0
#0 0x5fb5bcafcf2d in __asan_memcpy (/tmp/ruby/build/trunk_asan/ruby+0x204f2d) (BuildId: 6d92c84a27b87cfd253c38eeb552593f215ffb3d)
angussidney#1 0x5fb5bcde1fa5 in memcpy /usr/include/x86_64-linux-gnu/bits/string_fortified.h:29:10
ruby#2 0x5fb5bcde1fa5 in ruby_nonempty_memcpy /tmp/ruby/src/trunk_asan/include/ruby/internal/memory.h:758:16
ruby#3 0x5fb5bcde1fa5 in ruby__sfvwrite /tmp/ruby/src/trunk_asan/sprintf.c:1083:9
ruby#4 0x5fb5bcde1521 in BSD__sprint /tmp/ruby/src/trunk_asan/vsnprintf.c:318:8
ruby#5 0x5fb5bcde0fbc in BSD_vfprintf /tmp/ruby/src/trunk_asan/vsnprintf.c:1215:3
ruby#6 0x5fb5bcdde4b1 in ruby_vsprintf0 /tmp/ruby/src/trunk_asan/sprintf.c:1164:5
ruby#7 0x5fb5bcddd648 in rb_str_vcatf /tmp/ruby/src/trunk_asan/sprintf.c:1234:5
ruby#8 0x5fb5bcddd648 in rb_str_catf /tmp/ruby/src/trunk_asan/sprintf.c:1245:11
ruby#9 0x5fb5bcf97c67 in location_format /tmp/ruby/src/trunk_asan/vm_backtrace.c:462:9
ruby#10 0x5fb5bcf97c67 in location_to_str /tmp/ruby/src/trunk_asan/vm_backtrace.c:493:12
ruby#11 0x5fb5bcf90a37 in location_to_str_dmyarg /tmp/ruby/src/trunk_asan/vm_backtrace.c:795:12
ruby#12 0x5fb5bcf90a37 in backtrace_collect /tmp/ruby/src/trunk_asan/vm_backtrace.c:786:28
ruby#13 0x5fb5bcf90a37 in backtrace_to_str_ary /tmp/ruby/src/trunk_asan/vm_backtrace.c:804:9
ruby#14 0x5fb5bcf90a37 in rb_backtrace_to_str_ary /tmp/ruby/src/trunk_asan/vm_backtrace.c:816:9
ruby#15 0x5fb5bd335b25 in exc_backtrace /tmp/ruby/src/trunk_asan/error.c:1904:15
ruby#16 0x5fb5bd335b25 in rb_get_backtrace /tmp/ruby/src/trunk_asan/error.c:1924:16
```
https://ci.rvm.jp/results/trunk_asan@ruby-sp1/5810304
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related PRs: