Skip to content

Commit fbdf9ea

Browse files
committed
[DELTA-OSS-EXTERNAL] Allow Python vacuum(<integer>)
`vacuum(0)` was throwing the following error because the Java API has Double and Py4j was searching for integer. The fix is to convert the python parameter to float before calling java. ``` Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/private/var/folders/0q/c_zjyddd4hn5j9jkv0jsjvl00000gp/T/spark-ff856d5c-6a62-45bd-b9a4-0a0fea0acd09/userFiles-de551310-4bed-4267-be1c-b3c36556d6ff/io.delta_delta-core_2.11-0.4.0.jar/delta/tables.py", line 210, in vacuum File "/usr/local/Cellar/spark/spark-2.4.3-bin-hadoop2.7/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__ File "/usr/local/Cellar/spark/spark-2.4.3-bin-hadoop2.7/python/pyspark/sql/utils.py", line 63, in deco return f(*a, **kw) File "/usr/local/Cellar/spark/spark-2.4.3-bin-hadoop2.7/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 332, in get_return_value py4j.protocol.Py4JError: An error occurred while calling o49.vacuum. Trace: py4j.Py4JException: Method vacuum([class java.lang.Integer]) does not exist at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318) at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326) at py4j.Gateway.invoke(Gateway.java:274) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:238) at java.lang.Thread.run(Thread.java:748) ``` Closes #198 Author: Tathagata Das <[email protected]> #6557 is resolved by tdas/j0ag9ic6. GitOrigin-RevId: 7a171b8c110d40d23b18bcfffc2c5b8e59d6c0f4
1 parent d28c72d commit fbdf9ea

File tree

2 files changed

+2
-1
lines changed

2 files changed

+2
-1
lines changed

python/delta/tables.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -207,7 +207,7 @@ def vacuum(self, retentionHours=None):
207207
if retentionHours is None:
208208
return DataFrame(jdt.vacuum(), self._spark._wrapped)
209209
else:
210-
return DataFrame(jdt.vacuum(retentionHours), self._spark._wrapped)
210+
return DataFrame(jdt.vacuum(float(retentionHours)), self._spark._wrapped)
211211

212212
@since(0.4)
213213
def history(self, limit=None):

python/delta/tests/test_deltatable.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -289,6 +289,7 @@ def test_vacuum(self):
289289
self.__createFile('bac.txt', 'abcdf')
290290
self.assertEqual(True, self.__checkFileExists('abc.txt'))
291291
dt.vacuum() # will not delete files as default retention is used.
292+
dt.vacuum(1000) # test whether integers work
292293

293294
self.assertEqual(True, self.__checkFileExists('bac.txt'))
294295
retentionConf = "spark.databricks.delta.retentionDurationCheck.enabled"

0 commit comments

Comments
 (0)