Skip to content

[BUG]: hf_hub_download crashes with ValueError when stderr lacks a real file descriptor #4066

@tobocop2

Description

@tobocop2

Problem

hf_hub_download crashes with ValueError: bad value(s) in fds_to_keep in any environment where sys.stderr is not backed by a real file descriptor.

The download itself is fine. Only tqdm's progress bar initialization fails, but because _create_progress_bar has no safety net, it takes down the entire download.

Affected environments

Any runtime where sys.stderr is not backed by a real file descriptor:

  • Textual TUI apps
  • Jupyter notebooks
  • pytest capture
  • WSGI/ASGI web servers (gunicorn, uvicorn)
  • Sandboxed/containerized runtimes
  • IDE embedded terminals

Root cause

_create_progress_bar calls cls(disable=disable, name=name, **kwargs) with no exception handling. When stderr's fileno() returns -1, tqdm's __init__ tries to create a multiprocessing lock, which spawns a resource tracker subprocess via fork_exec(), which rejects the invalid fd.

Reproduction

Add to tests/test_file_download.py in the CachedDownloadTests class and run with pytest tests/test_file_download.py -k test_hf_hub_download_survives_bad_fileno:

import os
from unittest.mock import patch

from huggingface_hub import constants, hf_hub_download
from huggingface_hub.utils import SoftTemporaryDirectory

from .testing_utils import DUMMY_MODEL_ID


def test_hf_hub_download_survives_bad_fileno(self):
    """hf_hub_download should not crash when stderr.fileno() returns -1."""

    class FakeStderr:
        def write(self, s):
            pass

        def flush(self):
            pass

        def isatty(self):
            return True

        def fileno(self):
            return -1

    with SoftTemporaryDirectory() as tmpdir:
        with patch("sys.stderr", FakeStderr()):
            filepath = hf_hub_download(
                DUMMY_MODEL_ID,
                filename=constants.CONFIG_NAME,
                cache_dir=tmpdir,
            )
        self.assertTrue(os.path.exists(filepath))

Demo

Crash demo

Proposed fix

See #4065.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions