Skip to content

[FEATURE] Support prefix to allow using same bucket #85

@pushkar-anand

Description

@pushkar-anand

Is your feature request related to a problem? Please describe.

Currently, when using ESOP to back up to cloud storage (like S3, GCP, Azure), the path structure within the bucket is largely dictated by clusterId/datacenterId/nodeId. While this is a good default, it lacks flexibility when organizations need to:

  • Share a single bucket across multiple applications, environments (dev, staging, prod), or tenants, and want to isolate ESOP backups for each under a specific top-level prefix.
  • Adhere to specific organizational naming conventions or folder structures within their cloud storage buckets that don't neatly map to just clusterId/datacenterId/nodeId.
  • Simplify IAM permission management by granting access based on common prefixes for different teams or services.

Describe the solution you'd like
I propose that ESOP supports and enhances the --storage-location parameter to allow for the inclusion of user-defined prefixes directly within its string value.

If the command is:
esop backup --storage-location=s3://application-bucket/backups/cassandra/clusterA/dc1/node1 --snapshot-tag=backup_ ...

ESOP should reliably interpret:

  • Bucket: application-bucket
  • User-defined prefix: backups/cassandra
  • ESOP clusterId: clusterA
  • ESOP datacenterId: dc1
  • ESOP nodeId: node1

The topology can be uploaded to s3://application-bucket/backups/cassandra and all snapshot files can under s3://application-bucket/backups/cassandra/clusterA/dc1/node1.

Additional context

  • Formally supporting prefixes within --storage-location would greatly enhance the organizational capabilities of ESOP backups in shared cloud storage environments.
  • It would allow users to adhere to specific organizational naming conventions or folder structures within their cloud storage buckets more reliably.

I would be happy to contribute to the implementation if this feature request is accepted.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions