Datafile get_metadata requires bucket read permissions

[thclark]

Bug report

What is the current behavior?

Retrieval of object metadata requires excess permissions.

To reproduce:

1. Create a service account
2. Create a bucket (say my-bucket name) and put a file (say my-file.txt) in it
3. Give the service account 'Storage Object Viewer' permission for the bucket
4. Attempt to get the metadata for the object using the client with this service account

metadata = GoogleCloudStorageClient("my-project-id").get_metadata("my-bucket-name", "my-file.txt")

Result:

 Forbidden('GET https://storage.googleapis.com/storage/v1/b/lake-mast-timeseries?projection=noAcl&prettyPrint=false: dev-thclark@windquest.iam.gserviceaccount.com does not have storage.buckets.get access to the Google Cloud Storage bucket.')

This arises because the bucket is retrieved prior to the metadata for the object. Not sure if this is avoidable.

What is the expected behavior?

I'd expected get_metadata to require only object view permissions.

Workaround

For now I'm adding Storage Legacy Bucket Reader permission to the service accounts that need to do this. But, this enables them to list all datafiles in the bucket. This may not be desired behaviour.

Your environment

• Library Version: 0.1.16
• Platform Linux
• OS type and version: ubuntu buster
• Python version: 3.8.6
• pip version: 21.1.1
• google-cloud-storage version: 1.38.0

DECISION REQUIRED

We need to have a review of permissions that should be attached to services and document the various combinations required for typical service roles. Then we can decide whether it's necessary to fix this or not.