You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Gluten implemented some logic to convert call expr as subfield filter. To avoid duplication, we would like to use the existing 'leafCallToSubfieldFilter' logic in Velox. One incompatibility we found is current implementation is matching against Presto function names, while Spark function names could be different. The other one is 'isnotnull' is frequently used in Spark and it is not currently supported. These are the drafted changes we would like to propose: rui-mo@7340903.
The text was updated successfully, but these errors were encountered:
Yeah leafCallToSubfieldFilter is Presto specific (similar to Tokenizer). Mixing in Spark names probably is not a good way to do it. A related question is how do we parse this expression in ExprCompiler, do we have a isnotnull function in sparksql?
Description
Gluten implemented some logic to convert call expr as subfield filter. To avoid duplication, we would like to use the existing 'leafCallToSubfieldFilter' logic in Velox. One incompatibility we found is current implementation is matching against Presto function names, while Spark function names could be different. The other one is 'isnotnull' is frequently used in Spark and it is not currently supported. These are the drafted changes we would like to propose: rui-mo@7340903.
The text was updated successfully, but these errors were encountered: