DownloadConfig is not passed to LocalEvaluationModuleFactory when loading from local file, causing uncontrollable download behavior

[Very-White]

In evaluate/src/evaluate/loading.py lines 621-624:

elif os.path.isfile(combined_path):
    return LocalEvaluationModuleFactory(
        combined_path, download_mode=download_mode, dynamic_modules_path=dynamic_modules_path
    ).get_module()

The DownloadConfig from evaluation_module_factory is not passed to LocalEvaluationModuleFactory. This causes line 421:

local_imports = _download_additional_modules(
    name=self.name,
    base_path=str(Path(self.path).parent),
    imports=imports,
    download_config=self.download_config,
)

to use the default download_config. Consequently, in _download_additional_modules (line 248), if url_or_filename is an external URL, it will use the default download_config when entering the cached_path function.

Subsequently, when entering the get_from_cache function in evaluate/src/evaluate/utils/file_utils.py, use_etag=True and local_files_only=False. This skips line 438:

if os.path.exists(cache_path) and not force_download and not use_etag:

because use_etag=True. Then it proceeds to the if not local_files_only branch on line 447. If my server can't connect to Hugging Face at this point, it will get stuck here.

I also can't modify the download configuration by changing the DownloadConfig in the load function because it's not passed to the LocalEvaluationModuleFactory function. Nor can I use already downloaded files in the cache because the line if os.path.exists(cache_path) and not force_download and not use_etag: is skipped.

Is this a bug? If not, how can I use the evaluate.load function when connections to the Hugging Face server are prone to timeouts?

[caoyang-sufe]

try evaluate.load(path, download_config = DownloadConfig(use_etag = False))

network error derives from checking etag, not connecting to huggingface (if you have local evaluate repository)

[Very-White]

try evaluate.load(path, download_config = DownloadConfig(use_etag = False))

network error derives from checking etag, not connecting to huggingface (if you have local evaluate repository)

The DownloadConfig from evaluation_module_factory is not passed to LocalEvaluationModuleFactory. Your command will not have any effect.