Skip to content

Enhance parallel use of stac Accessor #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
konstntokas merged 2 commits into from
Jul 9, 2026
Merged

Enhance parallel use of stac Accessor #25

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
30bd2f9
read yfor review
konstntokas Jul 9, 2026
4a1ed75
change back to 50 day max for Sen2
konstntokas Jul 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion xcube_multistore/accessors/stac.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,13 @@
# SOFTWARE.

import datetime
import random
import time
from collections.abc import Iterable

import xarray as xr
from pystac_client.exceptions import APIError
from rasterio.errors import RasterioIOError
from requests.exceptions import ConnectionError, Timeout
from urllib3.exceptions import ProtocolError

Expand Down Expand Up @@ -56,6 +58,7 @@
ConnectionError,
Timeout,
ProtocolError,
RasterioIOError,
ConnectionResetError,
)
_NO_ITEMS_FOUND_PREFIX = "No items found in collection"
Expand Down Expand Up @@ -138,7 +141,8 @@ def _open_and_store_with_retry(
)
)

time.sleep(2**attempt)
delay = (5 * 2**attempt) + random.uniform(0, 2)
time.sleep(delay)

@staticmethod
def _split_time_range(data_id: str, open_params: dict) -> list[tuple[str, str]]:
Expand Down
Loading