Skip to content

Commit 8436b28

Browse files
committed
fix: Clean up leaked progress bars on session stop
_execution_progress_bar entries are only removed when operations complete with done=True. If operations are interrupted or fail before the handler is called with done=True, their tqdm objects leak. Now clears all remaining progress bars during stop().
1 parent c6636e1 commit 8436b28

1 file changed

Lines changed: 8 additions & 0 deletions

File tree

google/cloud/dataproc_spark_connect/session.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1259,6 +1259,14 @@ def stop(self, terminate: Optional[bool] = None) -> None:
12591259
>>> spark.stop(terminate=False)
12601260
"""
12611261
with DataprocSparkSession._lock:
1262+
# Clean up any remaining progress bars from interrupted operations
1263+
for pbar in DataprocSparkSession._execution_progress_bar.values():
1264+
try:
1265+
pbar.close()
1266+
except Exception:
1267+
pass
1268+
DataprocSparkSession._execution_progress_bar.clear()
1269+
12621270
if DataprocSparkSession._active_s8s_session_id is not None:
12631271
# Determine if we should terminate the server-side session
12641272
if terminate is None:

0 commit comments

Comments
 (0)