diff --git a/docs/docs/async.md b/docs/docs/async.md new file mode 100644 index 000000000..d228e3f23 --- /dev/null +++ b/docs/docs/async.md @@ -0,0 +1,19 @@ +# Async Usage + +Icechunk includes an optional asynchronous interface for orchestrating repos and sessions. However, the Icechunk core is fully asynchronous and delivers full parallelism and performance whether you choose to use the synchronous or asynchronous interface. Most users, particularly those doing interactive data science and analytics, should use the synchronous interface. + +## When to use async + +The async interface allows for icechunk operations to run concurrently, without blocking the current thread while waiting for IO operations. The most common reason to use async is that you are developing a backend service in which work may be happening across multiple Icechunk repositories simultaneously. + +## Using the async interface + +You can call both sync and async methods on a `Repository`, `Session`, or `Store` as needed. (Of course, to use the async methods, you must be within a async function.) Methods that support async are named with an `_async` postfix: + +```python exec="on" session="async_usage" source="material-block" +import icechunk + +async def get_branches(storage: icechunk.Storage) -> set[str]: + repo = await icechunk.Repository.open_async(storage) + return await repo.list_branches_async() +``` diff --git a/docs/docs/howto.md b/docs/docs/howto.md index b977e18e7..1a5be95e7 100644 --- a/docs/docs/howto.md +++ b/docs/docs/howto.md @@ -271,3 +271,11 @@ expired = repo.expire_snapshots(older_than=expiry_time) ```python results = repo.garbage_collect(expiry_time) ``` + +### Usage in async contexts + +Most methods in Icechunk have an async counterpart, named with an `_async` postfix. For more info, see [Async Usage](./async.md). + +```python +results = await repo.garbage_collect_async(expiry_time) +``` diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 10e85da19..56480c240 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -157,6 +157,7 @@ plugins: # These redirects work with use_directory_urls: true (default) 'icechunk-python/quickstart/index.md': 'quickstart.md' 'icechunk-python/reference/index.md': 'reference.md' + 'icechunk-python/async/index.md': 'async.md' 'icechunk-python/configuration/index.md': 'configuration.md' 'icechunk-python/storage/index.md': 'storage.md' 'icechunk-python/version-control/index.md': 'version-control.md' @@ -214,6 +215,7 @@ nav: - Xarray: xarray.md - Tuning Performance: performance.md - Virtual Datasets: virtual.md + - Async Usage: async.md - Icechunk for Git Users: icechunk-for-git-users.md - Creating Icechunk Datasets: - Global Raster Data cube: ingestion/glad-ingest.ipynb