[BUG] Found unsupported timezone

[res-life]

Describe the bug

For new JVM version, we find unsupported timezone.

From Niranjan:

25/11/13 00:18:55 ERROR GpuTimeZoneDB: cache timezone info cache failed
java.lang.IllegalStateException: load timezone DB cache failed!
	at com.nvidia.spark.rapids.jni.GpuTimeZoneDB.loadData(GpuTimeZoneDB.java:414)
	at com.nvidia.spark.rapids.jni.GpuTimeZoneDB.cacheDatabaseImpl(GpuTimeZoneDB.java:184)
	at com.nvidia.spark.rapids.jni.GpuTimeZoneDB.lambda$cacheDatabaseAsync$0(GpuTimeZoneDB.java:105)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.IllegalStateException: Unsupported midnight end of day for DST rules.
	at com.nvidia.spark.rapids.jni.GpuTimeZoneDB.lambda$loadData$3(GpuTimeZoneDB.java:377)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at com.nvidia.spark.rapids.jni.GpuTimeZoneDB.loadData(GpuTimeZoneDB.java:373)
	... 3 more

It's related to: https://github.com/NVIDIA/spark-rapids-jni/blob/main/src/main/java/com/nvidia/spark/rapids/jni/GpuTimeZoneDB.java#L373-L377

Steps/Code to reproduce bug

Expected behavior

Environment details (please complete the following information)

Additional context

[nartal1]

On JVM-17, below time is unsupported timezone
JAVA_HOME = /usr/lib/jvm/java-17-openjdk-amd64/

spark-shell:

scala> :paste
// Entering paste mode (ctrl-D to finish)

import java.time.ZoneId
import java.time.zone.ZoneOffsetTransitionRule
import java.util.TimeZone
import scala.collection.JavaConverters._

TimeZone.getAvailableIDs().foreach { tz =>
  val rules = ZoneId.of(tz, ZoneId.SHORT_IDS).getRules().getTransitionRules().asScala
  rules.foreach { rule =>
    if (rule.isMidnightEndOfDay()) {
      println(s"Timezone: $tz has a rule with midnight end of day: $rule")
    }
  }
}



// Exiting paste mode... now interpreting.
warning: 1 deprecation (since 2.13.0); for details, enable `:setting -deprecation` or `:replay -deprecation`
Timezone: Asia/Amman has a rule with midnight end of day: TransitionRule[Gap +02:00 to +03:00, THURSDAY on or before last day of FEBRUARY at 24:00 WALL, standard offset +02:00]
import java.time.ZoneId
import java.time.zone.ZoneOffsetTransitionRule
import java.util.TimeZone
import scala.collection.JavaConverters._

[res-life]

In Java 17, timezone of Asia/Amman does not have transition rules. What's your Java version?

[res-life]

@nartal1 Please help update the Environment details.

[nartal1]

Hi @res-life , Below are the details. Please let me know if you are looking for some other info as well.

workstation:~/apache-spark-4.0-released/spark-4.0.0-bin-hadoop3$ echo $JAVA_HOME 
/usr/lib/jvm/java-17-openjdk-amd64/

workstation:~/apache-spark-4.0-released/spark-4.0.0-bin-hadoop3$ java -version
openjdk version "17.0.2" 2022-01-18
OpenJDK Runtime Environment (build 17.0.2+8-86)
OpenJDK 64-Bit Server VM (build 17.0.2+8-86, mixed mode, sharing)

workstation:~/apache-spark-4.0-released/spark-4.0.0-bin-hadoop3$ ./bin/spark-shell 
25/11/13 02:28:28 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 4.0.0
      /_/
         
Using Scala version 2.13.16 (OpenJDK 64-Bit Server VM, Java 17.0.2)
Type in expressions to have them evaluated.
Type :help for more information.
25/11/13 02:28:31 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
25/11/13 02:28:32 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
Spark context Web UI available at http://10.110.46.105:4041
Spark context available as 'sc' (master = local[*], app id = local-1763029712184).
Spark session available as 'spark'.

scala> java.time.ZoneId.of("Asia/Amman").getRules().getTransitionRules()
val res0: java.util.List[java.time.zone.ZoneOffsetTransitionRule] = [TransitionRule[Gap +02:00 to +03:00, THURSDAY on or before last day of FEBRUARY at 24:00 WALL, standard offset +02:00], TransitionRule[Overlap +03:00 to +02:00, FRIDAY on or after OCTOBER 25 at 00:00 STANDARD, standard offset +02:00]]

[nartal1]

The error goes away when JVM is upgraded to openjdk version 17.0.12

[res-life]

It only happens on openjdk version 17.0.2, so this should have a low priority.