Skip to content

Commit 04dcbc2

Browse files
author
Xin Zhao
committed
Do not enable simple_motion_search_prune_rect
av2_simple_motion_search_prune_rect() is not implemented for the AV2 partition set and asserts "Not implemented" since e2d0566. With --disable-ml-partition-speed-features=0 the speed feature turned on, the assert was compiled out under NDEBUG, and the encoder segfaulted: av2_prune_partitions_before_search() calls the function for rectangular block sizes, where convert_bsize_to_idx() returns -1 and sms_tree is NULL because fill_sms_buf() only populates square sizes. Default configurations are bit-identical; the speed feature was already forced off there by av2_disable_ml_based_partition_sf().
1 parent 4bdc679 commit 04dcbc2

1 file changed

Lines changed: 10 additions & 1 deletion

File tree

av2/encoder/speed_features.c

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -296,7 +296,16 @@ static void set_good_speed_features_framesize_independent(
296296
sf->gm_sf.disable_gm_search_based_on_stats = 1;
297297

298298
sf->part_sf.less_rectangular_check_level = 1;
299-
sf->part_sf.simple_motion_search_prune_rect = 1;
299+
// Keep simple_motion_search_prune_rect off:
300+
// av2_simple_motion_search_prune_rect() is not implemented for the AV2
301+
// partition set (see the assert at the top of that function) and faults if
302+
// reached. av2_prune_partitions_before_search() calls it for rectangular
303+
// block sizes too, where convert_bsize_to_idx() returns -1 and sms_tree is
304+
// NULL, since fill_sms_buf() only populates square sizes. Previously this was
305+
// only unreachable because disable_ml_partition_speed_features defaults to 1;
306+
// with --disable-ml-partition-speed-features=0 the encoder segfaulted. Do not
307+
// set this to 1 without reimplementing the model for AV2.
308+
sf->part_sf.simple_motion_search_prune_rect = 0;
300309

301310
sf->inter_sf.disable_wedge_search_var_thresh = 0;
302311
// TODO(debargha): Test, tweak and turn on either 1 or 2

0 commit comments

Comments
 (0)