Skip to content

NoSuchMethodErrors on cidr_match queries #115

@mstair

Description

@mstair

Getting following error on attempts to run cidr_match queries against following CREATE TABLE security_log (user_id STRING, remote_host_ip STRING). Cluster is 11.3 LTS ML (includes Apache Spark 3.3.0, Scala 2.12)

from dbl_transpiler import spl

display(spl("""
index=security_log remote_host_ip=109.177.0.0/16
"""))

...

Py4JJavaError: An error occurred while calling z:com.databricks.labs.transpiler.spl.Transpiler.toDataFrame.
: java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.expressions.Cast$.apply$default$4()Z
	at org.apache.spark.sql.CidrMatch.aton(CidrMatch.scala:43)
	at org.apache.spark.sql.CidrMatch.ipAddress(CidrMatch.scala:24)
	at org.apache.spark.sql.CidrMatch.cidrMatch(CidrMatch.scala:22)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions