Skip to content

Commit fc6dcbd

Browse files
committed
Refresh the S3 store comparison for the cloud store's new capabilities
The paragraph predates this branch: the cloud store now has parallel multipart transfers in both directions with per-direction tuning, and handles S3-compatible endpoints via endpoint_url.
1 parent 7135622 commit fc6dcbd

1 file changed

Lines changed: 15 additions & 13 deletions

File tree

lib/galaxy/config/sample/object_store_conf.sample.yml

Lines changed: 15 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -156,21 +156,23 @@ backends:
156156
files_dir: /old-fs/galaxy/files
157157

158158

159-
# There are now four ways to access S3 related services. Two are
160-
# suitable just for AWS services (aws_s3 & cloud), one is
161-
# more suited for non-AWS S3 compatible services (generic_s3),
162-
# and finally boto3 gracefully handles either scenario.
159+
# There are now four ways to access S3 related services: aws_s3 is
160+
# suitable just for AWS services, generic_s3 is more suited for
161+
# non-AWS S3 compatible services, and boto3 and cloud gracefully
162+
# handle either scenario (cloud via its aws provider with an
163+
# endpoint_url).
163164
#
164165
# boto3 is built on the newest and most widely used Python client
165-
# outside of Galaxy. It has advanced transfer options and is likely
166-
# the client you should use for new setup. generic_s3 and aws_s3
167-
# have existed in Galaxy for longer and could perhaps be considered
168-
# more battle tested. Both boto3 and generic_s3 have been tested
169-
# with multiple non-AWS APIs including minio and GCP. The cloud
170-
# implementation is based on CloudBridge and is still supported
171-
# and has been recently tested - the downside is mostly the advanced
172-
# multi-threaded processing options of boto3 are not available
173-
# and it has not been battle tested like aws_s3.
166+
# outside of Galaxy and has advanced transfer options. generic_s3
167+
# and aws_s3 have existed in Galaxy for longer and could perhaps be
168+
# considered more battle tested. Both boto3 and generic_s3 have been
169+
# tested with multiple non-AWS APIs including minio and GCP. The
170+
# cloud implementation is based on CloudBridge and supports parallel
171+
# multipart uploads and downloads with per-direction tuning like
172+
# boto3; its distinguishing feature is that the same store type also
173+
# speaks Azure Blob, Google Cloud Storage, and OpenStack Swift
174+
# natively. For a pure S3 setup boto3 remains the most battle-tested
175+
# choice; choose cloud for multi-cloud deployments or native Swift.
174176

175177
#
176178
# Sample AWS S3 Object Store configuration (newest boto3 client)

0 commit comments

Comments
 (0)