Description
Version
main branch
Describe what's wrong
use spark iceberg catalog error in spark sql client
Error message and/or stacktrace
With the enterprise playground, login to the spark container, and then run:
`
cd /opt/spark && /bin/bash bin/spark-sql
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
25/03/06 02:50:08 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
25/03/06 02:50:11 WARN GravitinoDriverPlugin: Skip registering catalog_mysql because jdbc-mysql is not supported yet.
25/03/06 02:50:11 WARN GravitinoDriverPlugin: Skip registering catalog_postgres because jdbc-postgresql is not supported yet.
Spark master: local[*], Application Id: local-1741229411158
25/03/06 02:50:25 WARN SparkSQLCLIDriver: WARNING: Directory for Hive history file: /home/spark does not exist. History will not be available during this session.
spark-sql (default)> use catalog_iceberg;
25/03/06 02:50:40 ERROR SparkSQLDriver: Failed in [use catalog_iceberg]
org.apache.iceberg.jdbc.UncheckedSQLException: Failed to connect: jdbc:mysql://mysql:3306/db
at org.apache.iceberg.jdbc.JdbcClientPool.newClient(JdbcClientPool.java:57)
at org.apache.iceberg.jdbc.JdbcClientPool.newClient(JdbcClientPool.java:30)
at org.apache.iceberg.ClientPoolImpl.get(ClientPoolImpl.java:125)
at org.apache.iceberg.ClientPoolImpl.run(ClientPoolImpl.java:56)
at org.apache.iceberg.ClientPoolImpl.run(ClientPoolImpl.java:51)
at org.apache.iceberg.jdbc.JdbcCatalog.initializeCatalogTables(JdbcCatalog.java:155)
at org.apache.iceberg.jdbc.JdbcCatalog.initialize(JdbcCatalog.java:141)
Caused by: java.sql.SQLException: No suitable driver found for jdbc:mysql://mysql:3306/db
at java.sql/java.sql.DriverManager.getConnection(Unknown Source)
at java.sql/java.sql.DriverManager.getConnection(Unknown Source)
at org.apache.iceberg.jdbc.JdbcClientPool.newClient(JdbcClientPool.java:55)
... 105 more
Failed to connect: jdbc:mysql://mysql:3306/db
`
But, the mysql driver jar is on spark's jar folder:
`
find . -name mysql*
./jars/mysql-connector-java-8.0.27.jar
How to reproduce
- create Iceberg catalog with JDBC catalog backend
- run spark sql
Additional context
No response
Activity