-
Notifications
You must be signed in to change notification settings - Fork 282
[FEA] Support contains( strCol, expr ) #14539
Copy link
Copy link
Open
Labels
? - Needs TriageNeed team to review and classifyNeed team to review and classifyfeature requestNew feature or requestNew feature or request
Description
It would be good if spark-rapids would support the STRING contains() with a column or expression as the second argument.
See https://spark.apache.org/docs/latest/api/sql/index.html#contains.
Currently, when running queries using contains() such as the following:
SELECT COUNT(1) FROM fact WHERE contains( val, CAST( id AS STRING ) )We see the operator fall off the GPU as follows:
!Expression <Contains> Contains(cast(val#1L as string), cast(id#0L as string)) cannot run on GPU because search only supports StringType if it is a literal value
This then necessitates row-column transposition to get the data to/from the CPU operator.
Putting this operator on the GPU should enable some speedup.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
? - Needs TriageNeed team to review and classifyNeed team to review and classifyfeature requestNew feature or requestNew feature or request