-
Notifications
You must be signed in to change notification settings - Fork 26
Description
Summary
Add support for passing GCS-specific query parameters (specifically predefinedAcl) when uploading objects via put/put_async to Google Cloud Storage.
Motivation
When uploading objects to GCS buckets that use legacy ACLs (not uniform bucket-level access), it's common to need to set the object's access control at upload time. Currently, obstore's put_async only supports the attributes parameter which maps to HTTP headers (Content-Type, Cache-Control, etc.), but doesn't provide a way to pass GCS-specific query parameters like predefinedAcl.
This forces users to make a separate API call after upload to set the ACL, which:
- Adds latency (extra round-trip)
- Creates a race condition window where the object exists but isn't publicly accessible
- Requires using a separate GCS client library (e.g.,
gcloud-aio-storage) just for ACL management - Can cause errors due to timing issues
Current Workaround
from gcloud.aio.storage import Storage
# Upload with obstore
store = GCSStore(bucket=bucket_name)
await store.put_async(path=object_name, file=stream, attributes={"Content-Type": content_type})
# Separate call to set ACL using different library
aio_client = Storage(session=session)
await aio_client.patch_metadata(
bucket=bucket_name,
object_name=object_name,
metadata={},
params={"predefinedAcl": "publicRead"},
)Proposed Solution
Add an optional options or gcs_options parameter to put/put_async that allows passing GCS-specific query parameters:
await store.put_async(
path=object_name,
file=stream,
attributes={"Content-Type": content_type},
# New parameter for GCS/Provider specific options
options={"predefinedAcl": "publicRead"},
)Alternatively could be supported in attributes?
GCS API Reference
The predefinedAcl parameter is documented in the GCS JSON API for object insertion:
-
Objects: insert - https://cloud.google.com/storage/docs/json_api/v1/objects/insert
- Query parameter:
predefinedAcl(string) - Acceptable values:
authenticatedRead,bucketOwnerFullControl,bucketOwnerRead,private,projectPrivate,publicRead
- Query parameter:
-
Predefined ACLs overview - https://cloud.google.com/storage/docs/access-control/lists#predefined-acl