Skip to content

Commit 89c3c81

Browse files
LantaoJintdas
authored andcommitted
[DELTA-OSS-EXTERNAL] Fix integer overflow of numUpdatedRows
`numUpdatedRows` can overflow integer when update a large table. ``` java.lang.NumberFormatException: For input string: "4029988707" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:583) at java.lang.Integer.parseInt(Integer.java:615) at scala.collection.immutable.StringLike$class.toInt(StringLike.scala:272) at scala.collection.immutable.StringOps.toInt(StringOps.scala:29) at org.apache.spark.sql.delta.DeltaOperations$Update.transformMetrics(DeltaOperations.scala:190) at org.apache.spark.sql.delta.files.SQLMetricsReporting$class.getMetricsForOperation(SQLMetricsReporting.scala:62) at org.apache.spark.sql.delta.OptimisticTransaction.getMetricsForOperation(OptimisticTransaction.scala:80) ``` Closes #428 Signed-off-by: liwensun <[email protected]> Author: Alan Jin <[email protected]> #10064 is resolved by liwensun/4eq64o5z. GitOrigin-RevId: c4c77a7044d75896fc5ed8b766bd97ac87929d81
1 parent 1320bf9 commit 89c3c81

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/main/scala/org/apache/spark/sql/delta/DeltaOperations.scala

+1-1
Original file line numberDiff line numberDiff line change
@@ -159,7 +159,7 @@ object DeltaOperations {
159159
strMetrics += "numCopiedRows" -> "0"
160160
} else {
161161
strMetrics += "numCopiedRows" -> (
162-
numOutputRows - strMetrics("numUpdatedRows").toInt).toString
162+
numOutputRows - strMetrics("numUpdatedRows").toLong).toString
163163
}
164164
strMetrics
165165
}

0 commit comments

Comments
 (0)