how to use with non-AWS s3 providers?

[rabernat]

Many object storage services implement the S3 protocol without actually being AWS S3. For example, Wasabi and Open Storage Network (OSN). For these services, you define a custom endpoint URL for S3.

I would like to try cloud-files with OSN. For the AWS CLI, I would use this configuration

[plugins]
endpoint = awscli_plugin_endpoint

[profile osn]
aws_access_key_id=<key>
aws_secret_access_key=<secret>
s3 =
    endpoint_url = https://ncsa.osn.xsede.org
s3api =
    endpoint_url = https://ncsa.osn.xsede.org

However, I'm having trouble figuring out how to make this work. I read the wiki and looked at the code, but I couldn't figure it out. It is supported?

Thanks a lot for your help and for providing this useful package.

[william-silversmith]

Hey Ryan! Glad you're finding this useful.

We use a kind of non-standard way of doing this, but I think it makes a lot of sense.

cf = CloudFiles("s3://https://ncsa.osn.xsede.org/BUCKET", secrets=SECRETS)

This functionality isn't super battle tested, so please report any issues or any apparent design flaws.

[rabernat]

Awesome, will give it a try.

[william-silversmith]

The thing that immediately occurs to me is that there isn't a great way to set a per-host secrets file. I think it will probably pick up your default aws-secret.json though.