Fix local from_pretrained when CONFIG_NAME contains subdirectories

[ijiraq]

ModelHubMixin.from_pretrained used CONFIG_NAME in os.listdir(model_id) for local directories. listdir only returns top-level names, so nested paths such as codecs/XXXX/config.json are never detected; remote hf_hub_download already supported such filenames. This is a correction to make local behave like remote.

here's a tag to help get to the heart of the issue: local from_pretrained + CONFIG_NAME containing /

Use os.path.isfile(os.path.join(model_id, CONFIG_NAME)) instead.

Adds regression test with temporary nested config path.

---

Note

Low Risk
Low risk bugfix limited to local config discovery in ModelHubMixin.from_pretrained, plus a regression test. Main risk is unintended behavior change for local directory loads if a file exists at the joined path.

Overview
Fixes ModelHubMixin.from_pretrained local loading to correctly detect configs when constants.CONFIG_NAME includes subdirectories by switching from a top-level os.listdir check to os.path.isfile(os.path.join(dir, CONFIG_NAME)), and updates the warning message accordingly.

Adds a regression test (with a small dummy model) that temporarily sets CONFIG_NAME to a nested path and verifies from_pretrained can load config.json from that nested location in a local folder.

^{Reviewed by Cursor Bugbot for commit 6835fbc. Bugbot is set up for automated code reviews on this repo. Configure here.}

[Wauplin]

Hi @ijiraq did you ran into an actual issue with ModelHubMixin or is it just a theoretical issue? As far as I see, loading only top-level config files is precisely the expected behavior.

[ijiraq]

@Wauplin This was a real world problem experienced with AION when using the aion.codex.CodecManager. This model set of pretrained weights and also weights for a codec. Is the loading of the codex weights that runs into problem when attempting to use a local repository:

The request happens here;

https://github.com/PolymathicAI/AION/blob/bb93be306b4cc4ce005459a6bf260170dd50fab8/aion/codecs/manager.py#L53

It appears to me that huggingface_hub does the correct action (looking model subdirectory codex/images for config.json in this case) when doing a remote request but not on locally cached file somehow.

[Wauplin]

Oooh, I see. Sorry I haven't read correctly at first. So what you are doing is monkey-patching constants.CONFIG_NAME at runtime from config.json to a nested <path>/<to>/<folder>/config.json, correct? This is actually not something we want to support. Monkeypatching constant values can have pretty bad (and unnoticed) side effects. Besides, not relying on a root config.json file will make the download counter ineffective.

What I would suggest you to do is to drop PyTorchHubMixin entirely and rewrite some from_pretrained and push_to_hub class methods using huggingface_hub primitives (hf_hub_download, snapshot_download, upload_folder, etc.). The mixin class is supposed to help project getting started but it is not meant to be flexible for complex project. Hence why we also provide the low-level building blocks.

Please let me know if you have any questions