Creating an Iceberg table with a geometry column

[edigonzales]

Hi

I'm trying to create an Iceberg table with a geometry column just link in this example:

import org.apache.sedona.sql.utils.SedonaSQLRegistrator
SedonaSQLRegistrator.registerAll(spark)

val stmt = """
    CREATE TABLE local.myschema.geotable (id string, geom geometry)
    USING iceberg
    TBLPROPERTIES('format-version'='3');
"""

spark.sql(stmt).show()

But I get this error:

25/04/06 16:59:18 WARN UDTRegistration: Cannot register UDT for org.geotools.coverage.grid.GridCoverage2D, which is already registered.
25/04/06 16:59:18 WARN SimpleFunctionRegistry: The function rs_union_aggr replaced a previously registered function.
25/04/06 16:59:18 WARN UDTRegistration: Cannot register UDT for org.locationtech.jts.geom.Geometry, which is already registered.
25/04/06 16:59:18 WARN UDTRegistration: Cannot register UDT for org.apache.sedona.common.geometryObjects.Geography, which is already registered.
25/04/06 16:59:18 WARN UDTRegistration: Cannot register UDT for org.locationtech.jts.index.SpatialIndex, which is already registered.
25/04/06 16:59:18 WARN SimpleFunctionRegistry: The function st_envelope_aggr replaced a previously registered function.
25/04/06 16:59:18 WARN SimpleFunctionRegistry: The function st_intersection_aggr replaced a previously registered function.
25/04/06 16:59:18 WARN SimpleFunctionRegistry: The function st_union_aggr replaced a previously registered function.
java.lang.UnsupportedOperationException: User-defined types are not supported
  at org.apache.iceberg.spark.SparkTypeVisitor.visit(SparkTypeVisitor.java:52)
  at org.apache.iceberg.spark.SparkTypeVisitor.visit(SparkTypeVisitor.java:37)
  at org.apache.iceberg.spark.SparkSchemaUtil.convert(SparkSchemaUtil.java:126)
  at org.apache.iceberg.spark.SparkCatalog.createTable(SparkCatalog.java:238)
  at org.apache.spark.sql.connector.catalog.TableCatalog.createTable(TableCatalog.java:223)

I start the spark-shell with this command:

spark-shell \
  --conf spark.serializer=org.apache.spark.serializer.KryoSerializer \
  --conf spark.kryo.registrator=org.apache.sedona.core.serde.SedonaKryoRegistrator \
  --conf spark.sql.extensions=org.apache.sedona.sql.SedonaSqlExtensions \
  --conf spark.sql.catalog.local=org.apache.iceberg.spark.SparkCatalog \
  --conf spark.sql.catalog.local.type=hadoop \
  --conf spark.sql.catalog.local.warehouse=file:///Users/stefan/tmp/lakehouse \
  --packages org.apache.sedona:sedona-spark-shaded-3.5_2.12:1.7.1,org.datasyslab:geotools-wrapper:1.7.1-28.5,org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.8.1

Any ideas what's wrong?

Stefan

[jiayuasu]

@edigonzales This is an upcoming feature of Sedona 1.8.0. Please stay tuned about the release!