S3 support

[TLCFEM]

Since h5py support different drivers including S3, in principle it is possible to read H5 files from remote storage.

https://docs.h5py.org/en/stable/high/file.html#file-drivers

There is a need to store files on S3 and interact with h5grove/h5web, to avoid temporarily downloading desired files to local storage first, it would be beneficial if h5grove directly supports it.

The following two approaches can be proposed.

1. Change get methods to post in order to provide potential S3 credentials in the request body, then configurations can be passed to h5py.File via h5py_options.
2. Keep existing get methods, provide an additional configuration on the server side, when the file path is a S3 url, the credentials can be automatically applied.

I can contribute to the implementation once the best approach is determined.

[loichuder]

Strictly speaking, I am not against the idea but we may creeping up on HSDS. Have you tried using it?

Both approches look okay to me, 1. may seem cleaner. What do you think @axelboc ?

[axelboc]

I would follow the answer you gave about Zarr support: #107 (comment)

At its core, h5grove provides an HTTP API schema and a bunch of utilities to help implement this API. We provide one implementation for reading HDF5 files from disc, but I don't think we should expand the scope to include object storage or alternative formats. The problem is code ownership and long-term maintenance—we can only own and maintain what is currently in scope. That being said, I'm all for facilitating these additional implementations, so if there are any blockers we are more than open to help resolve them.

[TLCFEM]

I think it still makes a difference compared to zarr.
zarr is to some extend parallel to hdf5, indeed supporting it is nontrivial and expands the scope of this project.

S3 is natively supported by h5py, enabling it requires merely a few additional parameters passed to h5py.File.
Maintenance thus should not be an issue as this project does not implement the logic of resolving S3 files.

[t20100]

You are uploading hdf5 files to s3, right?

In which context/how do you want to use h5web/h5grove to access it? Custom webapp, jupyterlab extension, ...?

[TLCFEM]

You are uploading hdf5 files to s3, right?

In which context/how do you want to use h5web/h5grove to access it? Custom webapp, jupyterlab extension, ...?

Yes, we want to store hdf5 files in S3 storage and accessing it using h5web/h5grove.