Skip to content

Commit 32669f3

Browse files
committed
fix(build): Fix greedy bin-packing to respect CMake positional grouping
The previous rebalancing produced groups with unequal sizes (8 files in group0), but CMake's velox_add_grouped_tests takes files positionally in fixed batches of VELOX_TESTS_PER_GROUP=10. This caused IndexLookupJoinTest (378s) and NestedLoopJoinTest (46s) to spill into group0 alongside HashJoinTest (807s), resulting in a 1800s timeout. Fix by running greedy bin-packing with fixed capacity constraints: each group must have exactly 10 files (8 for the last group with 78 total). Now group0 has HashJoinTest + 9 lightest tests (~816s), while groups 1-7 are all ~419-424s each.
1 parent e42803d commit 32669f3

File tree

1 file changed

+75
-73
lines changed

1 file changed

+75
-73
lines changed

velox/exec/tests/CMakeLists.txt

Lines changed: 75 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -37,99 +37,101 @@ target_link_libraries(
3737
)
3838

3939
# Sources are ordered so that heavy tests are spread across groups via greedy
40-
# bin-packing by estimated execution time. With VELOX_TESTS_PER_GROUP=10 this
41-
# produces 8 groups of roughly equal wall-clock time (~8.6 min each), except
42-
# group0 which is dominated by HashJoinTest (~13.7 min). The previous
43-
# alphabetical order concentrated slow tests (HashJoinTest, IndexLookupJoinTest,
44-
# HashTableTest) into the same groups, creating a worst-case of ~18.8 min.
40+
# bin-packing by estimated execution time. With VELOX_TESTS_PER_GROUP=10,
41+
# CMake takes files positionally in batches of 10 (files 1-10 = group0,
42+
# 11-20 = group1, etc.), so each batch of 10 must be independently balanced.
43+
# Group0 is dominated by HashJoinTest (~13.6 min); groups 1-7 are ~7 min each.
44+
# The previous alphabetical order concentrated slow tests (HashJoinTest,
45+
# IndexLookupJoinTest, HashTableTest) into the same groups, creating a
46+
# worst-case of ~18.8 min.
4547
set(
4648
VELOX_EXEC_TEST_SOURCES
47-
# group0 (~822s): HashJoinTest is the single heaviest test; pad with light tests.
49+
# group0 (~816s): HashJoinTest is the single heaviest test; pad with light tests.
4850
HashJoinTest.cpp
49-
ParallelProjectTest.cpp
50-
AsyncConnectorTest.cpp
51-
ProbeOperatorStateTest.cpp
52-
ValuesTest.cpp
53-
PlanNodeStatsTest.cpp
51+
PrintPlanWithStatsTest.cpp
52+
ThreadDebugInfoTest.cpp
5453
WriterFuzzerUtilTest.cpp
5554
UnorderedStreamReaderTest.cpp
56-
# group1 (~523s)
55+
ProbeOperatorStateTest.cpp
56+
PlanNodeStatsTest.cpp
57+
ValuesTest.cpp
58+
ParallelProjectTest.cpp
59+
AsyncConnectorTest.cpp
60+
# group1 (~424s)
5761
IndexLookupJoinTest.cpp
58-
NestedLoopJoinTest.cpp
59-
SpatialIndexTest.cpp
60-
MergerTest.cpp
61-
WindowTest.cpp
62-
ContainerRowSerdeTest.cpp
62+
PrefixSortTest.cpp
63+
HashPartitionFunctionTest.cpp
64+
ExpandTest.cpp
65+
SplitListenerTest.cpp
66+
AssignUniqueIdTest.cpp
67+
HashBitRangeTest.cpp
68+
SplitTest.cpp
69+
ArrowStreamTest.cpp
6370
SqlTest.cpp
64-
PrintPlanWithStatsTest.cpp
65-
WindowFunctionRegistryTest.cpp
66-
TopNTest.cpp
67-
# group2 (~519s)
71+
# group2 (~423s)
6872
TableScanTest.cpp
73+
AggregationTest.cpp
6974
MergeTest.cpp
75+
LimitTest.cpp
76+
TopNTest.cpp
77+
SpatialIndexTest.cpp
78+
PlanNodeSerdeTest.cpp
79+
SpillStatsTest.cpp
80+
WindowFunctionRegistryTest.cpp
81+
PlanNodeToSummaryStringTest.cpp
82+
# group3 (~422s)
83+
MergeJoinTest.cpp
84+
OperatorTraceTest.cpp
85+
LocalPartitionTest.cpp
7086
SortBufferTest.cpp
71-
MarkSortedTest.cpp
72-
PartitionedOutputTest.cpp
73-
SplitListenerTest.cpp
74-
ThreadDebugInfoTest.cpp
87+
ScaleWriterLocalPartitionTest.cpp
88+
ContainerRowSerdeTest.cpp
89+
EnforceSingleRowTest.cpp
90+
MixedUnionWithTableScanTest.cpp
91+
ColumnStatsCollectorTest.cpp
7592
AggregateFunctionRegistryTest.cpp
76-
RoundRobinPartitionFunctionTest.cpp
77-
ArrowStreamTest.cpp
78-
# group3 (~516s)
79-
MergeJoinTest.cpp
93+
# group4 (~422s)
94+
MultiFragmentTest.cpp
8095
OutputBufferManagerTest.cpp
81-
OperatorTraceTest.cpp
82-
MemoryReclaimerTest.cpp
83-
FunctionResolutionTest.cpp
84-
TraceUtilTest.cpp
85-
HilbertIndexTest.cpp
86-
PlanNodeSerdeTest.cpp
96+
StreamingAggregationTest.cpp
97+
MixedUnionTest.cpp
98+
VectorHasherTest.cpp
99+
CountingJoinTest.cpp
100+
MergerTest.cpp
87101
ConcatFilesSpillMergeStreamTest.cpp
88-
AssignUniqueIdTest.cpp
89-
# group4 (~516s)
90-
MultiFragmentTest.cpp
91-
ExchangeClientTest.cpp
92-
AggregationTest.cpp
93-
MarkDistinctTest.cpp
94-
MixedUnionWithTableScanTest.cpp
95-
FilterToExpressionTest.cpp
96-
ExpandTest.cpp
97-
StreamingEnforceDistinctTest.cpp
98-
CustomJoinTest.cpp
99-
PlanNodeToSummaryStringTest.cpp
100-
# group5 (~515s)
102+
AddressableNonNullValueListTest.cpp
103+
RoundRobinPartitionFunctionTest.cpp
104+
# group5 (~421s)
101105
RowContainerTest.cpp
102-
SpillTest.cpp
103-
StreamingAggregationTest.cpp
104-
HashJoinBridgeTest.cpp
105-
PrefixSortTest.cpp
106-
HashPartitionFunctionTest.cpp
106+
HashTableTest.cpp
107+
NestedLoopJoinTest.cpp
108+
MemoryReclaimerTest.cpp
109+
RowNumberTest.cpp
107110
EnforceDistinctTest.cpp
108-
TaskListenerTest.cpp
109-
EnforceSingleRowTest.cpp
110-
HashBitRangeTest.cpp
111-
# group6 (~514s)
111+
MarkSortedTest.cpp
112+
TraceUtilTest.cpp
113+
FilterToExpressionTest.cpp
114+
ScaledScanControllerTest.cpp
115+
# group6 (~421s)
112116
TableWriterTest.cpp
113-
LocalPartitionTest.cpp
117+
ExchangeClientTest.cpp
114118
OrderByTest.cpp
115-
LimitTest.cpp
116-
RowNumberTest.cpp
117-
ScaledScanControllerTest.cpp
118-
VectorHasherTest.cpp
119-
SpillStatsTest.cpp
120-
ColumnStatsCollectorTest.cpp
121-
AddressableNonNullValueListTest.cpp
122-
# group7 (~513s)
123-
SpillerTest.cpp
124-
HashTableTest.cpp
125-
MixedUnionTest.cpp
126-
ScaleWriterLocalPartitionTest.cpp
119+
HashJoinBridgeTest.cpp
127120
FilterProjectTest.cpp
128-
TopNRowNumberTest.cpp
129-
CountingJoinTest.cpp
130121
UnnestTest.cpp
122+
StreamingEnforceDistinctTest.cpp
123+
FunctionResolutionTest.cpp
124+
TaskListenerTest.cpp
131125
PlanNodeToStringTest.cpp
132-
SplitTest.cpp
126+
# group7 (~419s)
127+
SpillerTest.cpp
128+
SpillTest.cpp
129+
WindowTest.cpp
130+
PartitionedOutputTest.cpp
131+
TopNRowNumberTest.cpp
132+
MarkDistinctTest.cpp
133+
CustomJoinTest.cpp
134+
HilbertIndexTest.cpp
133135
)
134136

135137
set(

0 commit comments

Comments
 (0)