-
Notifications
You must be signed in to change notification settings - Fork 29
Description
Issue
When telemetry extraction fails (e.g. during chip reset), the telemetry update thread dies silently without showing any error message or traceback. This makes debugging very difficult.
Current behavior: Heartbeat stops, telemetry freezes, no error shown
Expected behavior: Exception and traceback displayed when telemetry fails
Reproduction
- Start tt-smi and navigate to the Telemetry tab
- In another terminal, reset a chip:
tt-smi -r 0 - Observe: heartbeat stops updating, but no error message appears
Root Cause
The telemetry thread uses threading.Thread(daemon=True) which silently terminates on exceptions.
Current code (tt_smi/tt_smi.py, line ~574):
thread = threading.Thread(target=update_telem, name="telem_thread")
thread.setDaemon(True) # Daemon threads die silently on exception
thread.start()Suggested Fix
Use Textual's Worker API, which properly displays exceptions:
# Simple fix: add thread=True parameter
worker = self.run_worker(
update_telem,
thread=True,
exit_on_error=True, # Show traceback and exit (helpful for debugging)
name="telem_worker"
)
TELEM_THREADS.append(worker)For production use, set exit_on_error=False to continue on transient errors (though you'll want error handling inside update_telem() in that case).
Reference
From Textual docs:
A Worker that encounters an exception will, by default, display a traceback and exit the app. This behavior is designed to help you quickly identify issues during development.