feat: auto-detect model download hub between Hugging Face and ModelScope - #5265
Open
OliverBryant wants to merge 4 commits into
Open
feat: auto-detect model download hub between Hugging Face and ModelScope#5265OliverBryant wants to merge 4 commits into
OliverBryant wants to merge 4 commits into
Conversation
Add an "auto" download_hub option and make automatic detection the default: probe whether the Hugging Face endpoint is reachable (honoring HF_ENDPOINT mirrors and HTTP(S)_PROXY proxies) and pick huggingface if so, otherwise fall back to modelscope. The probe result is cached per process and its timeout is tunable via XINFERENCE_HUB_DETECT_TIMEOUT. Priority: explicit --download_hub > XINFERENCE_MODEL_SRC env (which now also accepts "auto") > connectivity detection. A local model_path skips detection since no download is needed.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces automatic detection of the model download hub (Hugging Face vs. ModelScope) by probing Hugging Face endpoint connectivity. It adds support for an "auto" download hub option, caches the detection result, and allows configuring the probe timeout. Feedback on the changes includes addressing a bug in resolve_download_hub when XINFERENCE_MODEL_SRC is set to "auto" and download_hub is None, catching broader exceptions during the connectivity probe to prevent unexpected crashes, and adding a test case to verify the resolution logic under the "auto" environment variable setting.
- resolve_download_hub now resolves XINFERENCE_MODEL_SRC="auto" to a concrete hub itself instead of relying on the legacy fallback inside download_from_modelscope (review feedback), with a test for it - probe failures now swallow any exception, not just RequestException, so a malformed HF_ENDPOINT/proxy config cannot break a model launch - pass a narrowly-typed resolved hub to downstream create_*_model_instance functions to fix mypy arg-type errors in CI
qinxuye
reviewed
Aug 2, 2026
qinxuye
reviewed
Aug 2, 2026
qinxuye
reviewed
Aug 2, 2026
- treat HTTP error responses (403/407 blocking proxies, 5xx broken mirrors) from the reachability probe as unreachable, since downloads would fail anyway; documented and covered by tests - default XINFERENCE_MODEL_SRC to "auto" in the Docker Compose deployment (and .env.example / docs) so the main deployment path gets auto detection instead of a pinned huggingface source - image/video/embedding/rerank matchers no longer let the XINFERENCE_MODEL_SRC fallback override an explicitly requested download_hub; the env is consulted only when no hub is given, matching the documented priority order, with an end-to-end priority test
qinxuye
reviewed
Aug 3, 2026
Offline / air-gapped Compose deployments set HF_HUB_OFFLINE=1 and read model weights from a pre-populated local Hugging Face cache. Probing there necessarily fails and auto detection would fall back to modelscope, whose spec bypasses that cache and attempts a real download. When HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE is enabled, detection now skips the probe and selects huggingface directly, so the offline profile keeps working with the default XINFERENCE_MODEL_SRC=auto. Documented in sources.rst and offline.env.example (including how to pin modelscope via .env when the local cache is a ModelScope one), with offline regression tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What do these changes do?
Currently users have to manually choose between Hugging Face and ModelScope as the model download source (via
--download_huborXINFERENCE_MODEL_SRC), and the implicit default is guessed from the system locale (zh_CN→ ModelScope). This guess is often wrong: users in China with a working proxy still get ModelScope, while users whose Hugging Face access is blocked but locale is notzh_CNget Hugging Face and the download fails.This PR adds an
autodownload hub option and makes automatic detection the default, so users no longer need to pick a source:auto_detect_download_hub()inxinference/model/utils.py: probes whether the Hugging Face endpoint is reachable (honoringHF_ENDPOINTmirrors andHTTP(S)_PROXYproxies) with a short timeout, pickshuggingfaceif reachable and falls back tomodelscopeotherwise. The result is cached per process so the probe runs at most once, and the timeout is tunable viaXINFERENCE_HUB_DETECT_TIMEOUT(default 3s).download_hub="auto"is accepted through the whole launch chain (REST API → supervisor → worker →create_model_instance) and resolved to a concrete hub before spec matching.download_hubis not specified at all, detection also runs — unless the user pinned a source viaXINFERENCE_MODEL_SRCor provided a localmodel_path(no download needed, no probe).XINFERENCE_MODEL_SRC=autois also supported.autoalongside the existing options.models/sources/sources.rst,getting_started/environments.rst).Behavior priority: explicit
--download_hub>XINFERENCE_MODEL_SRCenv > automatic connectivity detection. The locale heuristic is kept as the fallback insidedownload_from_modelscope()for code paths that never receive a resolved hub.Verification
xinference/model/tests/test_utils.pycovering detection (reachable/unreachable/cached),resolve_download_hubpriorities, andXINFERENCE_MODEL_SRC=auto— all pass.xinference/model/tests/test_utils.pypasses locally (one pre-existing network-flaky download test passed on re-run).pre-commit(black/ruff/isort/mypy/codespell) passes on the changed files; frontend file passes Prettier and ESLint.huggingface;resolve_download_hub("auto")/resolve_download_hub(None)resolve accordingly and explicit hubs pass through untouched.