Skip to content

[BUG FIX]: hf_hub_download crashes when stderr lacks a real file descriptor#4065

Open
tobocop2 wants to merge 5 commits intohuggingface:mainfrom
tobocop2:fix/tqdm-bad-fd-safety
Open

[BUG FIX]: hf_hub_download crashes when stderr lacks a real file descriptor#4065
tobocop2 wants to merge 5 commits intohuggingface:mainfrom
tobocop2:fix/tqdm-bad-fd-safety

Conversation

@tobocop2
Copy link
Copy Markdown

@tobocop2 tobocop2 commented Apr 7, 2026

Problem

hf_hub_download crashes with ValueError: bad value(s) in fds_to_keep in any environment where sys.stderr is not backed by a real file descriptor.

The download itself is fine. Only tqdm's progress bar initialization fails, but because _create_progress_bar has no safety net, it takes down the entire download.

Affected environments

Any runtime where sys.stderr is not backed by a real file descriptor:

  • Textual TUI apps
  • Jupyter notebooks
  • pytest capture
  • WSGI/ASGI web servers (gunicorn, uvicorn)
  • Sandboxed/containerized runtimes
  • IDE embedded terminals

Root cause

_create_progress_bar calls cls(disable=disable, name=name, **kwargs) with no exception handling. When stderr's fileno() returns -1, tqdm's __init__ tries to create a multiprocessing lock, which spawns a resource tracker subprocess via fork_exec(), which rejects the invalid fd.

Crash Demo

Crash demo

Fix

Add a _SafeTqdm subclass that uses a threading.RLock instead of a multiprocessing lock, and wrap both cls() calls in _create_progress_bar with try/except (OSError, ValueError) to fall back to _SafeTqdm.

Fixes #4066


Note

Medium Risk
Changes progress bar construction and fallback behavior across download paths; while scoped to error handling, it affects user-visible progress output and warning emission in diverse runtimes (Jupyter/TUIs/pytest).

Overview
Prevents hf_hub_download/snapshot_download from crashing when tqdm cannot initialize its multiprocessing lock (e.g. sys.stderr.fileno() is invalid) by wrapping _create_progress_bar construction in try/except (OSError, ValueError).

Introduces _SafeTqdm (threading RLock-based) as a fallback: HF’s default bar falls back with a warning and continues without progress, while custom tqdm_class fallbacks are forced disable=True to avoid corrupting caller output (e.g. TUIs). Adds regression tests covering bad fileno() and exploding custom tqdm initialization.

Reviewed by Cursor Bugbot for commit cb1cb66. Bugbot is set up for automated code reviews on this repo. Configure here.

@tobocop2 tobocop2 force-pushed the fix/tqdm-bad-fd-safety branch from f08d2e6 to 6eff9fb Compare April 7, 2026 21:52
@tobocop2 tobocop2 changed the title fix: handle tqdm crash when stderr lacks a real file descriptor fix: handle hf_hub_download crash when stderr lacks a real file descriptor Apr 7, 2026
_create_progress_bar crashes with ValueError when stderr.fileno()
returns -1 (Textual, Jupyter, pytest) or raises OSError. The crash
is in tqdm's multiprocessing lock init, not the download itself.

Add _SafeTqdm fallback that uses a threading lock instead, and
wrap both cls() calls in try/except (OSError, ValueError).
@tobocop2 tobocop2 force-pushed the fix/tqdm-bad-fd-safety branch from 6f4b2a4 to 43399d3 Compare April 8, 2026 03:52
@tobocop2 tobocop2 changed the title fix: handle hf_hub_download crash when stderr lacks a real file descriptor fix: hf_hub_download crashes when stderr lacks a real file descriptor Apr 8, 2026
@tobocop2 tobocop2 changed the title fix: hf_hub_download crashes when stderr lacks a real file descriptor [BUG FIX]: hf_hub_download crashes when stderr lacks a real file descriptor Apr 8, 2026
tobocop2 added 3 commits April 9, 2026 10:37
The custom-class branch of _create_progress_bar was falling back to
_SafeTqdm(**kwargs) without passing disable=True, so after a crash in
the caller's class the fallback rendered a visible tqdm bar into the
caller's output stream. For TUIs that had opted into custom progress
via tqdm_class, this meant ANSI escapes leaking into the app.

Pass disable=True on the fallback, matching the HF-subclass branch
which already passes disable=disable. Add a regression test covering
the crash-in-__init__ path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: hf_hub_download crashes with ValueError when stderr lacks a real file descriptor

1 participant