test: assert IN predicate falls back to Flink instead of disabling the case - #60
test: assert IN predicate falls back to Flink instead of disabling the case#60LuciferYang wants to merge 1 commit into
Conversation
…e case lance-format#55 disabled testInPredicatePushDown so the new CI pipeline could go green: it asserted that an IN predicate gets pushed down, but convertCallExpression() has no branch for BuiltInFunctionDefinitions.IN and returns null, so applyFilters() routes the predicate to remainingFilters. Disabling left IN as the only push-down path with no coverage at all. Assert the current fallback instead (acceptedFilters 0, remainingFilters 1), which keeps CI green and keeps the path under test; the assertions turn red once IN push-down is implemented, which is the reminder to update them. Also fix the class Javadoc, which listed IN and BETWEEN as covered. BETWEEN has no test and is not implemented in the source either.
|
Closing this in favour of #21, which fixes the source instead of the test. I opened this without checking for existing work, and #21 ( This PR would also have put #21 into conflict, since both touch the same assertions in The Javadoc correction here (the class listed |
What
Re-enables
LanceReadOptimizationsTest.testInPredicatePushDown, which #55 disabled, by asserting what the connector actually does with anINpredicate today.Why
#55 needed a green pipeline, and this test was the one red case: it asserted that
INgets pushed down, but the source never implemented that.convertCallExpression()has no branch forBuiltInFunctionDefinitions.IN, so it returnsnullandapplyFilters()routes the predicate intoremainingFilters. The quickest way to get CI green was@Disabled, with re-working the test called out as a follow-up. This is that follow-up.Disabling had a cost:
testInPredicatePushDownwas the only test touchingBuiltInFunctionDefinitions.IN, so theINpath ended up with no coverage at all. Nothing would catch it if a later refactor madeINthrow, or accepted it for push-down when the filter translation still can't express it.How
Assert the fallback rather than the unimplemented feature:
CI stays green and the path stays under test. When
INpush-down does land, these assertions turn red, which is the reminder to flip them back toacceptedFilters == 1. The reasoning is in a comment on the test so the next reader doesn't have to re-derive it.Also corrected the class Javadoc, which listed
INandBETWEENas covered.BETWEENhas no test and no implementation either (the source just has a// BETWEEN (not supported yet)comment).Test plan
mvn -pl lance-flink-1.18 test -Dtest=LanceReadOptimizationsTest-> 24 run, 0 failures, 0 skipped (was 24 run / 1 skipped)mvn -B -ntp -am -pl lance-flink-1.18 verify-> BUILD SUCCESS (178 run, 0 failures, 17 skipped)mvn -B -ntp -am -pl lance-flink-1.19 verify-> BUILD SUCCESSmvn -B -ntp -am -pl lance-flink-1.20 verify-> BUILD SUCCESSThe remaining 17 skips are
LanceCatalogS3Test$MinioIntegrationTests, gated on MinIO env vars.Still open from the #55 discussion
Wiring
*ITCaseinto CI needs work before the pipeline part is worth doing, since the ITs don't pass today:maven-failsafe-pluginadded locally, the ITs run post-packageagainst the shaded jar and hitNoSuchMethodErroronLanceTypeConverter.toArrowSchema(RowType)andLanceNamespace.connect(String, Map, BufferAllocator), which looks like relocated and non-relocated Arrow signatures mixed on one classpath.LanceNamespaceCatalog.createTable, which doesn't send the Arrow IPC schema stream, soDirectoryNamespacerejects it withInvalidInputException code=13.