Skip to content

HTTP Error 403 when using Calculator.downloadModels #26

@ricardonpa

Description

@ricardonpa

Root cause: pysmartdl2.SmartDL uses HTTP range requests to support multi‑threaded downloads, which means it sends Range: bytes=… headers when fetching files. On Zenodo’s current infrastructure, these ranged requests to file URLs are rejected with HTTP 403 Forbidden, even though a plain, single GET using urllib.request.urlretrieve works fine for the same URL. As a result, Calculator.downloadModels fails with a 403 when using SmartDL, but succeeds when the same URLs are downloaded via urllib without range headers.

>>> Calculator.downloadModels("SIPFENN_Krajewski2022_NN30")
Fetching: SIPFENN_Krajewski2022_NN30
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ricardonpa/Library/CloudStorage/GoogleDrive-ricardo.npa@gmail.com/Shared drives/Development/Codes/pySIPFENN/pysipfenn/core/pysipfenn.py", line 290, in downloadModels
    downloadObject.start()
  File "/Users/ricardonpa/miniconda3/envs/pysipfenn/lib/python3.11/site-packages/pysmartdl2/pysmartdl2.py", line 272, in start
    urlObj = urllib.request.urlopen(req, timeout=self.timeout, context=self.context)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ricardonpa/miniconda3/envs/pysipfenn/lib/python3.11/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ricardonpa/miniconda3/envs/pysipfenn/lib/python3.11/urllib/request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/ricardonpa/miniconda3/envs/pysipfenn/lib/python3.11/urllib/request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/Users/ricardonpa/miniconda3/envs/pysipfenn/lib/python3.11/urllib/request.py", line 563, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ricardonpa/miniconda3/envs/pysipfenn/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/Users/ricardonpa/miniconda3/envs/pysipfenn/lib/python3.11/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

We can either patch pysmartdl2.SmartDL to avoid range requests or bypass it in pySIPFENN using requests or urllib.request.

Here’s a minimal fix using urllib.request that preserves the main behavior of the current implementation: main...ricardonpa:pySIPFENN:main

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions