Disable child metric registration under parent to prevent OOM#10
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR temporarily disables child metric registration in the VortexMetrics::child_with_tags() method to prevent memory leaks caused by indefinite accumulation of child metrics in long-lived VortexSession instances. The change addresses an OOM issue in high-throughput scenarios where each query creates multiple child VortexMetrics instances (per partition and file) that persist for the session lifetime.
Changes:
- Commented out the line that registers child metrics with the parent, preventing memory accumulation
- Added a TODO comment with a link to the upstream tracking issue
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
sgrebnov
force-pushed
the
sgrebnov/0113-mem-test
branch
from
f3e78cf to
e4b8cd5
Compare
lukekim
approved these changes
Jan 13, 2026
lukekim
pushed a commit
that referenced
this pull request
Jan 14, 2026
lukekim
pushed a commit
that referenced
this pull request
Feb 6, 2026
lukekim
added a commit
that referenced
this pull request
Feb 27, 2026
…Fusion demuxer (#23) * Disable child metric registration under parent to prevent OOM (#10) * Use tokio::sync::oneshot to prevent FuturesUnordered reentrant drop crash (#9) * Don't return an error when we have an unsupported node, bubble up "TRUE" as in keep for that node, up to any `and` or `or` node and handle empty IN list. (#8) * feat: Support retrieving WriterStrategyBuilder from VortexSession (#6) * Fix session get-or-default (vortex-data#5662) The comments described this get-or-default, but instead it was a panic --------- Signed-off-by: Nicholas Gates <nick@nickgates.com> * feat: Support retrieving writer strategy builder from vortex session --------- Signed-off-by: Nicholas Gates <nick@nickgates.com> Co-authored-by: Nicholas Gates <gatesn@users.noreply.github.com> * fix: Handle UncompressedSizeInBytes statistic correctly (#3) * fix: Add log dependency to vortex-io * fix: Add Debug impl for WriteStrategyBuilder and log dep for vortex-io * fix: Update persistent module to use simplified expression handling from PR #8 * revert: Restore spiceai-51 versions of persistent module (incompatible with DF51) * fix: Restore format.rs to spiceai-51 version * Add Case-When Expression and tests (#12) * fix: Ensure CastExpr/CastColumnExpr/ScalarFunctionExpr check children in can_be_pushed_down The can_be_pushed_down function was returning true for CastExpr and CastColumnExpr without checking if their child expressions are convertible. This caused runtime errors when the child contained expressions like CaseExpr that convert() cannot handle. Also fixed ScalarFunctionExpr to recursively check its arguments. Fixes spiceai/spiceai#9037 * Add Case-When Expression and tests * Implement execute() * Add additional tests and fix type issue * Fix toml lint * feat(case_when): implement lazy evaluation to avoid side effects in unevaluated branches This implements proper lazy evaluation in CaseWhen expression to ensure that THEN/ELSE branches are only evaluated for rows where they apply. This is critical for correctness when expressions have side effects like divide-by-zero panics. The implementation: 1. Evaluates conditions in order, tracking which rows have been matched 2. For each condition, computes an effective mask (condition AND NOT matched) 3. Uses filter() to create a scoped array with only matching rows 4. Evaluates THEN expression only on the filtered scope 5. Uses scatter_with_mask() to expand results back to original positions 6. Short-circuits when all rows are matched or all conditions fail This fixes TPC-DS Q73 which has a pattern like: CASE WHEN hd_vehicle_count > 0 THEN hd_dep_count/hd_vehicle_count ELSE NULL END Previously, the division would be evaluated for all rows including those where hd_vehicle_count=0, causing a divide-by-zero panic. Now the division is only evaluated for rows where the condition is true. Added test: test_evaluate_divide_by_zero_protected_by_case_when * Formatting * feat(datafusion): Export DefaultExpressionConvertor (#19) * feat: Add option for writing to target vortex file size * fix: Actually respect target file size * fix: Set default file size to 16mb * fix: Update use of unwrap in tests * test: Update tests * fix: Always write with custom sink * fix: Stream directly into target file * fix: Improve safety arounded ended writer streams with remaining source * feat: add target file size configuration for Vortex file output - Introduced `target_file_size_mb` option to `VortexFormat` for controlling the size of output files in megabytes. - Updated `VortexSink` to handle writing files based on the specified target size, bypassing DataFusion's default row count-based splitting. - Implemented logic to split output files when the buffered data exceeds the target file size during the write process. - Added tests to verify the new target file size configuration. * refactor: clean up case expression handling and improve file size configuration logic --------- Signed-off-by: Nicholas Gates <nick@nickgates.com> Co-authored-by: Sergei Grebnov <sergei.grebnov@gmail.com> Co-authored-by: Luke Kim <80174+lukekim@users.noreply.github.com> Co-authored-by: Nicholas Gates <gatesn@users.noreply.github.com>
lukekim
pushed a commit
that referenced
this pull request
May 18, 2026
## Summary Fix for the second part of: vortex-data#7808 ``` (gdb) bt #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78 #2 __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:89 #3 0x000076a38cc4527e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #4 0x000076a38cc288ff in __GI_abort () at ./stdlib/abort.c:79 #5 0x000076a38cc297b6 in __libc_message_impl (fmt=fmt@entry=0x76a38cdce8d7 "%s\n") at ../sysdeps/posix/libc_fatal.c:134 #6 0x000076a38cca8ff5 in malloc_printerr ( str=str@entry=0x76a38cdd1bf0 "free(): double free detected in tcache 2") at ./malloc/malloc.c:5775 #7 0x000076a38ccab55f in _int_free (av=0x76a38ce03ac0 <main_arena>, p=<optimized out>, have_lock=0) at ./malloc/malloc.c:4541 #8 0x000076a38ccaddce in __GI___libc_free (mem=0x5be5cd9632c0) at ./malloc/malloc.c:3398 #9 0x000076a38eb6807e in alloc::alloc::dealloc (ptr=0x5be5cd9632c0, layout=...) at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/alloc.rs:114 #10 alloc::alloc::{impl#1}::deallocate (self=0x5be5cd95f708, ptr=..., layout=...) at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/alloc.rs:271 #11 0x000076a38ead9a64 in alloc::boxed::{impl#8}::drop<dyn vortex_scan::Partition, alloc::alloc::Global> (self=0x5be5cd95f6f8) at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1666 #12 0x000076a38ead349e in core::ptr::drop_in_place<alloc::boxed::Box<dyn vortex_scan::Partition, alloc::alloc::Global>> () at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:804 #13 0x000076a38e8764de in core::ptr::drop_in_place<vortex_ffi::scan::VxPartitionScan> () at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:804 #14 0x000076a38e876fb8 in core::ptr::drop_in_place<alloc::boxed::Box<vortex_ffi::scan::VxPartitionScan, alloc::alloc::Global>> () at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:804 #15 0x000076a38e87f2f5 in core::mem::drop<alloc::boxed::Box<vortex_ffi::scan::VxPartitionScan, alloc::alloc::Global>> (_x=0x5be5cd95f6f0) at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/mem/mod.rs:961 #16 0x000076a38e84efa7 in vortex_ffi::scan::vx_partition_free (ptr=0x5be5cd95f6f0) at vortex-ffi/src/macros.rs:295 #17 0x00005be5b0c81126 in operator() (__closure=0x7fff2208a8b0) at /home/ubuntu/vortex/vortex-ffi/test/scan.cpp:940 ``` ## Testing Verifying existing behavior is maintained. Signed-off-by: Dergousov Maksim <dergousovmaxim99@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🗣 Description
Temporarily disable child registration in
child_with_tags()to prevent memory growth in long-lived VortexSession instances. The children Vec accumulates indefinitely with strong references never cleared.Root cause:
VortexSession → VortexMetrics (root)
└─> children: RwLock<Vec> ← NEVER CLEARED
Each query creates child VortexMetrics (one per partition × files) that are registered with the session's root metrics and persist for the session lifetime, causing OOM in high-throughput scenarios.
Impact:
Vortex tracking issue:
🔨 Related Issues
🤔 Concerns