Some notes about working in progress GCS support

[ClSlaid]

related issue: #7
Currently, OpenDAL does not support GCS, and I've been working on it this week.

Design Notes:

GCS support will be implemented as an independent service, controlled by a feature switch of "services-gcs". All other services will not be affected.

All communications with GCS will be via its XML HTTP API.

Authentication and encryptions

With the dependency of reqsign, we could choose a scope and authenticate with the credential, which was read from the path or from base64 &strs. No access key id and access key are needed.

The scope of auth should be read-write by default. If happened to write on a read-only scoped bucket, may a is read_only error be returned directly to save time?

GCS also supports server-side encrypting with customer-managed encryption keys (CMEK) and customer-supplied encryption keys (CSEK). Only CSEK will be implemented for simpleness.

Won't support region detection

Having skewed twice through GCS's documents, I did not find how the actual region of the storage will affect the URI to use. So it won't be supported.

Won't support virtual hosted-style endpoints

Clients can name their buckets with domains, but in this case, GCS won't support virtual hosted-style endpoints for these buckets. For simplicity, virtual hosted-style endpoints will not be supported.

Customizable endpoint

GCS supports CNAME mapping, clients could map their domains to c.www.googleapi.com. The endpoint should be customizable, just in case.

Working inside a single bucket

Being an object storage service like AWS S3, GCS offers a flat view of buckets and objects. Detailed interactions with GCS should stay consistent with the S3 service, thanks to @Xuanwo who made such a perfect example for this. :)

OpenDAL's GCS service won't care about bucket management, it just operates inside the bucket.

[Xuanwo]

Why not using its native json api?

I thought the xml api is compatible with s3 api, we can use service s3 directly for accessing through xml api.

[ClSlaid]

It's both ok I think, we can use JSON API, certainly.

[Xuanwo]

They are different. JSON API is more powerful than XML API.

[ClSlaid]

Update:
Won't support virtual hosted-style endpoints, and fixed some typo.

[Xuanwo]

Please don't try to map all services-s3 features to gcs.

gcs is gcs, although it provides a compatible layer to s3. We only need to support gcs native features in services-gcs. Users who want to behave as s3 can use service-s3directly.