-
Notifications
You must be signed in to change notification settings - Fork 4k
Open
Labels
Description
Describe the bug, including details regarding any error messages, version, and platform.
Running this test
require 'bundler/setup'
require 'arrow'
table = Arrow::Table.new(
'foo' => [1, 2],
)
plan = Arrow::ExecutePlan.new
source_node = plan.build_source_node(table)
expresions = [
Arrow::FieldExpression.new('foo'),
Arrow::LiteralExpression.new('hello'),
]
names = [
'foo',
'bar',
]
project_options = Arrow::ProjectNodeOptions.new(expresions, names)
project_node = plan.build_project_node(source_node, project_options)
10_000.times do |i|
Arrow::LiteralExpression.new("test #{i}")
end
sink_node_options = Arrow::SinkNodeOptions.new
plan.build_sink_node(project_node, sink_node_options)
plan.validate
plan.start
plan.wait
reader = sink_node_options.get_reader(project_node.output_schema)
pp reader.read_allI expect the following output:
foo bar
(uint8) (utf8)
0 1 hello
1 2 hello
However the output is something like this:
foo bar
(uint8) (utf8)
0 1
1 2
In this case the bar column became the empty string, but sometimes I get other junk values.
If I comment out the times-block I get consistently the expected result. I suspect the literals gets garbage collected before the plan is executed.
I can reproduce this with both Arrow 22 and main.
Component(s)
Ruby