Add support for MiniO storage backend

[alfeg]

This PR is yet another attempt to address #621, #551, and #271

Added additional settings to S3StorageOptions:

• ForcePathStyle = false setting to true will make AWSSDK.net compatible with MiniO installations
• ServiceUrl - ability to override S3 service location
• UseHttp - ability to use HTTP protocol for ServiceUrl

All those settings are real settings from actual AmazonS3Config from official AWSSDK library

There is already another PR's to address those issues: #644 and #272

My PR is much less intrusive, only adding ability to pass settings that are already exists in AwsSdk library

[viceice]

@loic-sharma Can you please add this? I like BaGet but would like to use MinIO as storage backend in my kubernetes cluster instead of nfs.

[loic-sharma]

It'd be less confusing to folks if we separate the AWS S3 and MinIO samples:

Suggested change

	"ServiceUrl": "",
	"UseHttp": false,
	"ForcePathStyle": false

[viceice]

Sound good

[loic-sharma]

Hello @alfeg, thank you for the contribution! Your changes look good. I suggested some minor tweaks but I've never used MinIO. Could you let me know what you think?

@viceice and @DTeuchert Feel free to jump in too if you have comments!

[loic-sharma]

Could we add the data annotation attributes that @DTeuchert had used in #272?

-       [Required]
+       [RequiredIf(nameof(ServiceUrl), null)]
        public string Region { get; set; }

+       [RequiredIf(nameof(Region), null)]
        public string ServiceUrl { get; set; }

[DTeuchert]

Yes in my understanding we need a service url for a private / on premise solutions or a region.

[loic-sharma]

From my understanding the region can be set using the service URL. If so should we follow @DTeuchert's pattern in #272:

Suggested change

	RegionEndpoint = RegionEndpoint.GetBySystemName(options.Region),
	RegionEndpoint = (options.Region != null)
	? RegionEndpoint.GetBySystemName(options.Region)
	: null,
	ServiceURL = options.ServiceUrl

This will let us delete lines 47-50 below:

-               if (!string.IsNullOrWhiteSpace(options.ServiceUrl))
-               {
-                   config.ServiceURL = options.ServiceUrl;
-               }

[viceice]

I think the AWS SDK requires a region independent from service url, but im not fully sure.

[alfeg]

Hello @alfeg, thank you for the contribution! Your changes look good. I suggested some minor tweaks but I've never used MinIO. Could you let me know what you think?

Hello,

I'm OK with all proposed changes, but cannot test them in any way for next two weeks. I'm on vacation without laptop and with poor internet :)

[viceice]

Sounds good

[DTeuchert]

On Monday I can test this request with a ceph cluster as s3 backend.

[krmxd]

are there any plans on following up on this ?

[viceice]

you should check

• https://github.com/bagetter/BaGetter

[pishangujeniya]

@loic-sharma Any plans on this?

[loic-sharma]

No. I'd consider checking out BaGetter instead, it is an active fork of BaGet.

[pishangujeniya]

No. I'd consider checking out BaGetter instead, it is an active fork of BaGet.

that is also not having support of MinIO.