Errors out after a handful of photographs

[alper]

Progress is very slow because every time it errors out after a bit with this error message:

ERROR:root:IO error saving photo: <urlopen error [Errno 8] nodename nor servname provided, or not known>
Traceback (most recent call last):
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/urllib3/connection.py", line 204, in _new_conn
    sock = connection.create_connection(
        (self._dns_host, self.port),
    ...<2 lines>...
        socket_options=self.socket_options,
    )
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/urllib3/util/connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
               ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alpercugun/.local/share/uv/python/cpython-3.13.5-macos-aarch64-none/lib/python3.13/socket.py", line 977, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    response = self._make_request(
        conn,
    ...<10 lines>...
        **response_kw,
    )
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/urllib3/connectionpool.py", line 488, in _make_request
    raise new_e
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/urllib3/connectionpool.py", line 464, in _make_request
    self._validate_conn(conn)
    ~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
    conn.connect()
    ~~~~~~~~~~~~^^
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/urllib3/connection.py", line 759, in connect
    self.sock = sock = self._new_conn()
                       ~~~~~~~~~~~~~~^^
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/urllib3/connection.py", line 211, in _new_conn
    raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: HTTPSConnection(host='api.flickr.com', port=443): Failed to resolve 'api.flickr.com' ([Errno 8] nodename nor servname provided, or not known)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/requests/adapters.py", line 644, in send
    resp = conn.urlopen(
        method=request.method,
    ...<9 lines>...
        chunked=chunked,
    )
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/urllib3/connectionpool.py", line 841, in urlopen
    retries = retries.increment(
        method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2]
    )
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/urllib3/util/retry.py", line 535, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.flickr.com', port=443): Max retries exceeded with url: /services/rest/ (Caused by NameResolutionError("HTTPSConnection(host='api.flickr.com', port=443): Failed to resolve 'api.flickr.com' ([Errno 8] nodename nor servname provided, or not known)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/bin/flickr_download", line 12, in <module>
    sys.exit(main())
             ~~~~^^
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/flickr_download/flick_download.py", line 541, in main
    download_user(
    ~~~~~~~~~~~~~^
        args.download_user,
        ^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        args.metadata_store,
        ^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/flickr_download/flick_download.py", line 340, in download_user
    download_set(
    ~~~~~~~~~~~~^
        photoset.id, get_filename, size_label, skip_download, save_json, metadata_store
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/flickr_download/flick_download.py", line 121, in download_set
    download_list(
    ~~~~~~~~~~~~~^
        pset, pset.title, get_filename, size_label, skip_download, save_json, metadata_store
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/flickr_download/flick_download.py", line 170, in download_list
    do_download_photo(
    ~~~~~~~~~~~~~~~~~^
        dirname,
        ^^^^^^^^
    ...<7 lines>...
        metadata_db=conn,
        ^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/flickr_download/flick_download.py", line 219, in do_download_photo
    fname = photo._getOutputFilename(fname, size_label)
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/flickr_api/objects.py", line 1319, in _getOutputFilename
    photo_file = self.getPhotoFile(size_label)
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/flickr_api/objects.py", line 1312, in getPhotoFile
    size_label = self._getLargestSizeLabel()
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/flickr_api/objects.py", line 1258, in _getLargestSizeLabel
    sizes = {k:v for k,v in self.getSizes().items() if v["media"] == self.media}
                            ~~~~~~~~~~~~~^^
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/flickr_api/objects.py", line 1229, in getSizes
    self.__dict__["sizes"] = self._getSizes(**args)
                             ~~~~~~~~~~~~~~^^^^^^^^
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/flickr_api/reflection.py", line 256, in call
    r = method_call.call_api(method=flickr_method, **method_args)
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/flickr_api/method_call.py", line 141, in call_api
    resp = CACHE.get(cachekey) or requests.post(request_url, args,
                                  ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
        timeout=get_timeout())
        ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/Users/alpercugun/.cache/uv/archive-v0/-HW_okz4uK3pnQuogiO-r/lib/python3.13/site-packages/requests/adapters.py", line 677, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.flickr.com', port=443): Max retries exceeded with url: /services/rest/ (Caused by NameResolutionError("HTTPSConnection(host='api.flickr.com', port=443): Failed to resolve 'api.flickr.com' ([Errno 8] nodename nor servname provided, or not known)"))

I'm running the latest with:

uvx --from git+https://github.com/beaufour/flickr-download flickr_download

[beaufour]

I think you are having some network issues. The error is Failed to resolve 'api.flickr.com which isn't a problem with the library

[alper]

It's odd though that I don't have such network issues anywhere else or for the first so many pictures.

Could this be some kind of rate limiting?

[beaufour]

Hmm, it would be odd if rate limiting resulted in a DNS error. I added some more error handling now, and it'll then skip the download with an error and continue to the next photo.